I have followed the documentation provided for the Python comtypes module (http://pythonhosted.org/comtypes/server.html) and built a simple COM server.
It seems to be working correctly. I can see that the typelib and interface are registered (using Oleviewer) and from a Python script I can call the (single) method that adds two integers.
However, I cannot make it work with Excel (Office 2007, running on Win7). The following simple function does not work (execution stops when MyMethod is called)
Function test(a, b)
Dim x As New MyTypeLib.MyObject
ab = x.MyMethod(a, b)
Debug.Print "d" & CStr(ab)
test = ab
End Function
Is there any way to debug this?
More comments
There is indeed light at the end of the tunnel. The suggestion to use VBScript has been helpful. The following short script does work (providing independent confirmation that the COM server works).
Dim x,ab
Set x = CreateObject("MyTypeLib.MyObject")
ab = x.MyMethod(1, 2)
MsgBox CStr(ab)
However, VBScript is calling the server using late binding and I was trying to get the early binding to work.
So, I changed my Excel VBA function to use late binding and expected things to work, but they do not.
Here is the modified VBA:
Function test(a, b)
Dim x As Object
Set x = CreateObject("MyTypeLib.MyObject")
gadd = x.MyMethod(1, 5)
End Function
So my server can be used with late binding from Python and VBScript, but not Excel!
Using Process Monitor I can see that in both late and early binding cases Excel tries unsuccessfully to load msvcr90.dll, even though this DLL is installed on the system.
Here is a screenshot from Process Monitor, showing where Office Excel 2007 on 64-bit Windows 7 Home Edition starts to try to load msvcr90.dll
When I posted this question, I needed to know a bit more about how to debug COM servers (well a lot more actually!). It turns out that there were several problems that stopped my server from working. I think finding out about the tools is probably as much of interest as the actual problems, so this answer will try to cover both.
Initially, it was helpful to use VBScript to provide an independent verification that the server was registered and working (the simple code is shown above). VBScript uses late binding and I wanted early binding, but it was nevertheless helpful because I then realised that Excel did not work with the late binding VBA code either (on machine A)!
Something odd happened too with VBScript. When I first wrote scripts they ran by default (clicking on the file in Windows Explorer), but then for no apparent reason they stopped working. I found that I needed to specify the full path to the 32-bit version (C:\WINDOWS\SysWOW64\cscript.exe) for them to work.
Also worth noting is that by using cscript.exe, instead of wscript.exe, debugging messages embedded in my Python modules appear on the Windows CMD window.
The next breakthrough came by using the Dependency Walker program depends.exe. This is often mentioned in posts like mine where people are trying to figure out why DLLs are not working.
Dependency Walker is available from it's own website, but that version does not work properly with Windows 7 (see Profiling x86 executable with Dependency Walker hangs on Windows 7 x64) so I found that I needed to install WDK 8.1 (from here: http://msdn.microsoft.com/en-US/windows/hardware/gg454513).
DW is very powerful and a bit overwhelming at first. However, by reading its logging output it became clear that Excel 2007 uses msvcr80.dll and that the server's attempt to load msvcr90.dll was failing (even though the DLL was properly installed on the machine). This was not a problem on machine B because Excel 2013 uses msvcr90.dll.
Now, in these tests I was creating the server using Python comtypes interactively (there is no server DLL involved). But there is also the possibility of creating a DLL by using the py2exe tool. I was trying to do this too, the DLL failed to register using regsvr32.
DW helped debug this problem too. Happily, in the logging output I noticed a stack trace of a Python exception. It was the GetModule() in the example. It turns out that this call is needed only once, to create some Python classes that are then cached on the machine. Removing this command fixed the problem.
Also, although it may not have caused a problem, the official current version of py2exe is 0.6.9 but this caused some files that are no longer available to be bundled (specifically zlib.pyd). The unofficial version 0.6.10 (available here: http://www.lfd.uci.edu/~gohlke/pythonlibs/#py2exe) solved that issue.
So I now have the possibility of compiling a COM server DLL, in which I can include the required msvcr90.dll, and, yes, it works on both machines :-)
In summary then. Windows 7, with its mixture of 32-bit and 64-bit libraries is a nightmare. VBScript helps, by providing an independent environment for testing a server. Dependency Walker really is amazing, but takes ages to learn to use (other useful tools in the WDK include oleviewer, process-monitor and process-explorer).
Related
We have a sizeable, legacy Python application written in Python 2.7 64 bit on Windows 10 64 bit - with GUI in wxPython, heavy numerical computations via NumPy, 2D plotting with MatPlotLib, 3D with VTK and so on, which we distribute as py2exe compiled binary after converting all our own Python files to pyd (Windows dlls) using Cython.
Since about a month ago, we started experiencing a super weird behavior of the file open dialog on Windows - i.e., a dialog that appears when the user asks to load a model/file from disk. The issue is, all the filenames are gone, we can only see the icons for folders/files, no text at all.
Please see picture below:
As you can see, the central list which usually contains icons and filenames now displays only icons. The only way to get the filenames back is to right-click in the main window and select "Refresh" - although sometimes navigating to a different folder makes the filenames show up.
This behavior does not show up while using the Python code, only in the compiled app. And it does not happen with other software (Microsoft Office, Notepad++, anything else, they all use the native Windows file dialog like our app and they all work all right).
I have been fighting with this for days and days, and it's not that obvious what is going on as the executable distribution folder has hundreds of dlls and standard Python modules and sub-folders containing the Python standard library plus all 3rd party libraries... the possible interactions between those dlls and Windows native ones are uncountable.
We also have zero control on Windows updates, when and where and how, and zero access privileges on our machines as they are locked down corporate beasts.
Now, for the questions:
Has anybody ever seen this behavior before in an app? If yes, can you point us to a possible solution?
If no one has seen this before, do you know if there is a way to force a refresh of an open file dialog, by whatever mean necessary - whether it is a MFC command, a Python one via win32gui, a system call, a C hack, anything?
Thank you in advance from a hair-pulling fellow programmer.
EDIT
After some more pain, I found out that py2exe had included a Windows DLL (PROPSYS.dll) from my machine, which was incompatible with all other machines (they have all sort of architectures). Excluding that DLL from the build system has fixed the problem.
Thanks go to #Simon Mourier for the invaluable suggestion to look at the DLL used by the process. Thank you!!! If you post your comment as an answer I’ll accept it as solution.
In Windows, when an application fails in a strange way, one idea is to check what are the .dll loaded in that application's process and try to spot anything that looks strange.
This is especially true when you use the Common Dialog (Open, Save As) provider by the Windows Shell since doing that potentially bring in a lot of 3rd parties .dll, loading them in the process.
One of the best tool to check that is Process Explorer from Sysinternals.
Here is a screenshot that shows external .dll opened in standard Notepad just because I used the File Open dialog box. We can see Intel's extensions, Adobe extensions, OneDrive, etc.
It was not the issue here, but still, it apparently helped spot the problem.
I'm working on an (Windows and Mac) application that uses Python as an embedded scripting language.
The application includes an internal text editor, implemented using Scintilla, and I'm using Jedi for autocompletion, which generally works great.
However, when attempting autocompletion on a computer that does not have a separate installation of Python, Jedi raises an error:
jedi.api.environment.InvalidPythonEnvironment:
Could not get version information for 'python':
FileNotFoundError(2, 'The system cannot find the file specified', None, 2, None)
Digging into the code, I can see that the underlying code that is throwing the FileNotFoundError is when Jedi attempts to run python using subprocess.Popen. Python is not installed on the computer, so this fails.
I can also reproduce the same issue on a computer that does have Python installed by editing my Path environment variable not to include the location of python.exe.
Ideally, we don't want users of our application to have to install Python just to get autocompletion working.
My questions:
Is it possible to get Jedi not to spawn subprocesses, and instead run its code inside the same instance of Python within which it itself is running? I couldn't find anything about this in the documentation or the source code that deals with Environments, and extrapolating from the discussion here I suspect the answer might be no.
Is it possible somehow to get Jedi to use the same python37.dll that our application is using for its functionality, instead of looking for a .exe file that does not exist?
Is there any way we could make some kind of minimal Python installation within our existing app installation that uses the same DLLs/Python Lib etc? How could I go about doing this?
Is there any other way to get Jedi autocompletion working in our app without requiring the user to install Python, or including a full Python installer as part of our build process?
Is it possible to get Jedi not to spawn subprocesses, and instead run its code inside the same instance of Python within which it itself is running? I couldn't find anything about this in the documentation or the source code that deals with Environments, and extrapolating from the discussion here I suspect the answer might be no.
This is definitely possible. All the tools are there. There are discussions ongoing here: https://github.com/davidhalter/jedi-vim/issues/870.
IMO a patch to Jedi is needed that uses an jedi.api.environment.InterpreterEnvironment in some cases like yours. It's definitely possible, it's just buggy at the moment.
So I wrote a Python script which does some simple stuff. It was originally going to run on a Unix server but due to crappy network security settings which TPTB refuse to change, we need to run it on a Windows server instead. However, the administrators of said Windows server refuse to do anything helpful like install Python.
What are my options for running a Python script on Windows without Python?
Consideration 1:
Something like Py2Exe - I found this after a quick Google search and it seems promising. From what I can tell, it'll generate a bunch of files but we can just xcopy that directory to our Windows machine and it will be completely isolated and not have any external dependencies. Does anyone have any insight on how well this works? Obviously, it depends on my Python script but fortunately this script is quite simple and only uses built in Python libraries such as urllib2 and urlparse.
Consideration 2:
We can assume the Windows server has at least some version of the .NET Framework installed too, which brings IronPython to mind. I've never used this before, but I've always wanted to. From what I can tell, it will compile Python code into CLS compliant IL code which can be run natively under the .NET runtime. However, does this require additional .NET libraries to be installed on the server? Can I just bundle those DLLs with my program? Or, does it require I rewrite my Python script to call into .NET Framework specific classes instead of using things like urllib2 or urlparse?
Thanks!
PS - The ironic part: I actually barely know Python and I'm a .NET expert, but I wrote the script in Python because I was told it would run on a Unix server. Had I known we'd end up running this on a Windows server, I'd have written the thing in C# to begin with in about 1/10th of the time. Fail.
Will they let you copy executables onto the server at all? If so then you should be able to do a non-admin installation of Python or use Portable Python which can just be copied into a folder without any installation at all.
Nothing wrong with Py2exe, but it does mean you then have to build the script into a fresh executable each time you update it. Also Py2exe has a slightly longer startup time than a Python interpreter because it has to extract the Python dlls into a temporary folder each time it runs; that only matters of course if you run your script a lot.
I'm working on a Python application. Sometimes the interpreter crashes when in a third party C++ DLL.
I'm thinking about writing a Python extension that installs a handler for unhandled structured exceptions (Windows) in order to write a minidump to the disk and log the stack trace of every Python thread.
Two questions:
Does a Python extension with a similar purpose already exist? According to my own Google search, nothing seems to be publicly available, but maybe I didn't search enough.
Is it feasible to implement something like this? (I'm experienced in C++ and Windows programming, but have never implemented a Python extension...)
Check out FaultHandler on PyPI.
I recently wanted to do the same thing and created minidumper to do so, and did a small write-up of it here.
I once read about minimal python installation without a lot of the libraries that come with the python default installation but could not find it on the web...
What I want to do is to just pack a script with the python stuff required to execute it and make portable.
Does any one know about something like that?
Thanks
Micro Python is actively maintained and has been ported to a bunch of microcontrollers.
For other small implementations, you might also want to check out tinypy or PyMite.
If you don't care about size, but really just want an easy way to distribute a python program, consider PyInstaller or one of the others on this list.
Portable python might do what you want. It's a python installation for USB thumb drives.
There's now finally Micro Python, claiming to be full reimplementation of Python 3 core, fitting even into medium-size 32bit microcontrollers. API will be different of course, so C modules will require porting. Project is funded via KickStarter, source code will be released some time after the campaign (request for consideration was made to author to not delay release of the source, to help bootstrap Micro Python community sooner).
http://micropython.org/
You can also look for already installed instances.
OpenOffice / LibreOffice
Look at the environment variable UNO_PATH or into the default install directories, for example for Windows and LO5
%ProgramFiles(x86)%\LibreOffice 5\program\python.exe
Gimp
look into the default install directories, for example for Windows
C:\Program Files\GIMP 2\Python
and so on...