We have a sizeable, legacy Python application written in Python 2.7 64 bit on Windows 10 64 bit - with GUI in wxPython, heavy numerical computations via NumPy, 2D plotting with MatPlotLib, 3D with VTK and so on, which we distribute as py2exe compiled binary after converting all our own Python files to pyd (Windows dlls) using Cython.
Since about a month ago, we started experiencing a super weird behavior of the file open dialog on Windows - i.e., a dialog that appears when the user asks to load a model/file from disk. The issue is, all the filenames are gone, we can only see the icons for folders/files, no text at all.
Please see picture below:
As you can see, the central list which usually contains icons and filenames now displays only icons. The only way to get the filenames back is to right-click in the main window and select "Refresh" - although sometimes navigating to a different folder makes the filenames show up.
This behavior does not show up while using the Python code, only in the compiled app. And it does not happen with other software (Microsoft Office, Notepad++, anything else, they all use the native Windows file dialog like our app and they all work all right).
I have been fighting with this for days and days, and it's not that obvious what is going on as the executable distribution folder has hundreds of dlls and standard Python modules and sub-folders containing the Python standard library plus all 3rd party libraries... the possible interactions between those dlls and Windows native ones are uncountable.
We also have zero control on Windows updates, when and where and how, and zero access privileges on our machines as they are locked down corporate beasts.
Now, for the questions:
Has anybody ever seen this behavior before in an app? If yes, can you point us to a possible solution?
If no one has seen this before, do you know if there is a way to force a refresh of an open file dialog, by whatever mean necessary - whether it is a MFC command, a Python one via win32gui, a system call, a C hack, anything?
Thank you in advance from a hair-pulling fellow programmer.
EDIT
After some more pain, I found out that py2exe had included a Windows DLL (PROPSYS.dll) from my machine, which was incompatible with all other machines (they have all sort of architectures). Excluding that DLL from the build system has fixed the problem.
Thanks go to #Simon Mourier for the invaluable suggestion to look at the DLL used by the process. Thank you!!! If you post your comment as an answer I’ll accept it as solution.
In Windows, when an application fails in a strange way, one idea is to check what are the .dll loaded in that application's process and try to spot anything that looks strange.
This is especially true when you use the Common Dialog (Open, Save As) provider by the Windows Shell since doing that potentially bring in a lot of 3rd parties .dll, loading them in the process.
One of the best tool to check that is Process Explorer from Sysinternals.
Here is a screenshot that shows external .dll opened in standard Notepad just because I used the File Open dialog box. We can see Intel's extensions, Adobe extensions, OneDrive, etc.
It was not the issue here, but still, it apparently helped spot the problem.
Related
I'm working on a 2d game in pygame, and I want it to be available on the web. In my head, a fun way to challenge myself is to use web assembly to make the game available on the web. So I'm wondering if there is any way to take executable binary (.exe) and somehow "compile" .wasm if that's still sayable at this point) to
Running a .exe via WebAssembly (WASM) in the browser will be extremely slow and inefficient but is still possible with a Wasm PC emulator ( eg v86 ) + an OS ( eg ReactOS ).
But if you can gain access to game sourcecode then
what you could do is using CPython 3.11 which supports WebAssembly target ( with the help of Emscripten a C/C++ compiler targeting WASM ).
Add to that the latest development version of PyGame and minor changes to the game main loop source code.
You can now run directly the game if you pack both sourcecode and assets with the emscripten File Packager instead of a .exe you will have a .wasm (cpython) + one .data (assets+code) and some javascript glue to load them.
Please note that currently there is no straightforward tool to do that at once, but situation will most likely evolve very quickly after release of CPython 3.11.
Personally, I don't have much experience with combining .py and .wasm files, but I do know some stuff regarding running Pygame:
First of all, you can run basic Pygame files here. I scoured the internet in search of an independent Pygame running engine, and this is the only one I could find.
Second, you can use PyInstaller to convert your Pygame file itself to .exe (it packages it nicely, but can't be signed, so some computers might think it's a virus). I have no idea how to run this specifically on the web, but if you can manage to find a web platform that can run it with a window, then you can maybe try and run that .exe with Javascript and connect the .wasm to that.
Third, I'm not sure there's a way to run .exe files with a separate window in web assembly, but I'm not too familiar with wasm.
I hope that helped!
I have followed the documentation provided for the Python comtypes module (http://pythonhosted.org/comtypes/server.html) and built a simple COM server.
It seems to be working correctly. I can see that the typelib and interface are registered (using Oleviewer) and from a Python script I can call the (single) method that adds two integers.
However, I cannot make it work with Excel (Office 2007, running on Win7). The following simple function does not work (execution stops when MyMethod is called)
Function test(a, b)
Dim x As New MyTypeLib.MyObject
ab = x.MyMethod(a, b)
Debug.Print "d" & CStr(ab)
test = ab
End Function
Is there any way to debug this?
More comments
There is indeed light at the end of the tunnel. The suggestion to use VBScript has been helpful. The following short script does work (providing independent confirmation that the COM server works).
Dim x,ab
Set x = CreateObject("MyTypeLib.MyObject")
ab = x.MyMethod(1, 2)
MsgBox CStr(ab)
However, VBScript is calling the server using late binding and I was trying to get the early binding to work.
So, I changed my Excel VBA function to use late binding and expected things to work, but they do not.
Here is the modified VBA:
Function test(a, b)
Dim x As Object
Set x = CreateObject("MyTypeLib.MyObject")
gadd = x.MyMethod(1, 5)
End Function
So my server can be used with late binding from Python and VBScript, but not Excel!
Using Process Monitor I can see that in both late and early binding cases Excel tries unsuccessfully to load msvcr90.dll, even though this DLL is installed on the system.
Here is a screenshot from Process Monitor, showing where Office Excel 2007 on 64-bit Windows 7 Home Edition starts to try to load msvcr90.dll
When I posted this question, I needed to know a bit more about how to debug COM servers (well a lot more actually!). It turns out that there were several problems that stopped my server from working. I think finding out about the tools is probably as much of interest as the actual problems, so this answer will try to cover both.
Initially, it was helpful to use VBScript to provide an independent verification that the server was registered and working (the simple code is shown above). VBScript uses late binding and I wanted early binding, but it was nevertheless helpful because I then realised that Excel did not work with the late binding VBA code either (on machine A)!
Something odd happened too with VBScript. When I first wrote scripts they ran by default (clicking on the file in Windows Explorer), but then for no apparent reason they stopped working. I found that I needed to specify the full path to the 32-bit version (C:\WINDOWS\SysWOW64\cscript.exe) for them to work.
Also worth noting is that by using cscript.exe, instead of wscript.exe, debugging messages embedded in my Python modules appear on the Windows CMD window.
The next breakthrough came by using the Dependency Walker program depends.exe. This is often mentioned in posts like mine where people are trying to figure out why DLLs are not working.
Dependency Walker is available from it's own website, but that version does not work properly with Windows 7 (see Profiling x86 executable with Dependency Walker hangs on Windows 7 x64) so I found that I needed to install WDK 8.1 (from here: http://msdn.microsoft.com/en-US/windows/hardware/gg454513).
DW is very powerful and a bit overwhelming at first. However, by reading its logging output it became clear that Excel 2007 uses msvcr80.dll and that the server's attempt to load msvcr90.dll was failing (even though the DLL was properly installed on the machine). This was not a problem on machine B because Excel 2013 uses msvcr90.dll.
Now, in these tests I was creating the server using Python comtypes interactively (there is no server DLL involved). But there is also the possibility of creating a DLL by using the py2exe tool. I was trying to do this too, the DLL failed to register using regsvr32.
DW helped debug this problem too. Happily, in the logging output I noticed a stack trace of a Python exception. It was the GetModule() in the example. It turns out that this call is needed only once, to create some Python classes that are then cached on the machine. Removing this command fixed the problem.
Also, although it may not have caused a problem, the official current version of py2exe is 0.6.9 but this caused some files that are no longer available to be bundled (specifically zlib.pyd). The unofficial version 0.6.10 (available here: http://www.lfd.uci.edu/~gohlke/pythonlibs/#py2exe) solved that issue.
So I now have the possibility of compiling a COM server DLL, in which I can include the required msvcr90.dll, and, yes, it works on both machines :-)
In summary then. Windows 7, with its mixture of 32-bit and 64-bit libraries is a nightmare. VBScript helps, by providing an independent environment for testing a server. Dependency Walker really is amazing, but takes ages to learn to use (other useful tools in the WDK include oleviewer, process-monitor and process-explorer).
I'm a self-taught, amateur, purely recreational programmer. I don't understand all the fancy programming lingo, and I certainly don't have any good resources, apart from this website, where I can go for help. (i.e., Please dumb it down for me!) I would imagine my question here is somewhat common, but I honestly couldn't find any answers on Google or this website, probably because I don't know the proper terminology to search for.
~~~
Having said that, I feel I have a pretty solid grasp on the basics of Python. And now, I've created an application that I'd like to share with a friend. My application accesses JPEG image files on my computer using a directory path that I've written into the code itself. However, I'd like my friend to be able to store these image files anywhere on their computer, not necessarily in the file folder that I've been using.
I assume the best way to accomplish this is to allow my friend to choose the directory path for themselves and then to write their chosen directory path to a file at a predetermined location on their computer. My application would then have that file's location prewritten into its code. This way, it would be trivially easy to open the file at the predetermined location, and then that file would point my application to my friend's chosen directory path.
1.) Are any of my intuitions here misguided? Are there better ways of doing this?
2.) If you think my general approach is a reasonable one, then is there a good/common place on the computer where applications typically store their directory paths upon installation?
Any advice - or any recommended resources - would be very much appreciated! Thanks!
Well, the standard way to do this is a lot more complicated and platform-specific:
On traditional Unix, this is pretty simple; you create a text file in some simpler format (e.g., that used by ConfigParser, named, say, ~/.myprogram.cfg, and you write a line to it that looks like image_path=/path/to/images.
On most modern Linux systems, or any other FreeDesktop/XDG-based system, you should (at least for GUI apps) instead use a special directory looked up in the environment as XDG_CONFIG_HOME, falling back to ~/.config, instead of using ~.
On Windows, the standard place to store stuff like this is the Windows Registry (e.g., by using winreg), by creating a key for your program and storing a value with name image_path and value /path/to/images there.
On Mac, the standard place to store stuff like this is in the NSUserDefaults database (e.g., by using PyObjC, which isn't part of the stdlib but does come built-in with Apple's pre-installed Python) by opening the default domain for your program and adding a value with key image_path and value… well, you probably want a Cocoa bookmark (maybe even a security-scoped one), not a path.
That probably all sounds way, way too complicated.
One option is to use a library that wraps this all up for you. If you're already using a heavy-duty framework like, say, Qt, it probably has functionality built-in to do that. Otherwise, it may take a lot of searching to find something.
A simpler alternative is to just pretend everything is like traditional Unix. That will work on Windows and Mac. It will be slightly annoying on some Windows versions that your config file will be visible in their home directory, but not a huge deal. It means you won't get some of the bonus features that Mac provides, like being able to magically follow the directory if the user moves it somewhere else on his hard drive, or remembering the settings if he reinstalls OS X and migrates his old settings, but again, usually that's fine.
In between the extremes, you can pretend everything is like Linux, using a special, and unobtrusive, location for the files on Windows and Mac just as you do there. Both platforms have APIs to look up special directories, called "application data" on Windows and "application support" on Mac. Using PyWin32 or PyObjC, respectively, these are pretty easy to look up. (For example, see this answer.) Then you just create a subdirectory there named My App on Windows, or com.mydomain.myapp on Mac, and store the file there.
I have written a program. I don't know if it is important how it is written but you can find it here: http://pastebin.com/Z3ZvVPV8 Basically, it asks you to assign values to variables and will perform calculations depending on what variables you chose, and prints the answer.
I would like to know how I can make the program run in a window other than cmd (I am using Windows Vista 32bit). I don't need much at all in terms of GUI, just a window that is a bit more user friendly/easier to look at when they are using the program.
EDIT: To those suggesting using IDLE, while that would work for me, if others want to use the program they would have to download it, so I was hoping for a way for that not to happen.
Python comes with a sort of default GUI package TkInter you can use it.
Also there is a lot of other GUI packages available.
The Python standard library offers a lot of ways to implemt simple (but also rather complex) GUIs. I'd like to point you at the documentation of TK (tool kit for graphical interfaces) http://docs.python.org/library/tk.html where you will find also some useful example of use.
Py2Exe is a viable option if you really don't need a gui. This will make it run and look like a command prompt, but it will be an .exe file. Here is a quick quote from thier page: "py2exe is a Python Distutils extension which converts Python scripts into executable Windows programs, able to run without requiring a Python installation."
Another alternative is to get Portable Python. Here is a quote from thier webpage: "Portable Python is a Python® programming language preconfigured to run directly from any USB storage device, enabling you to have, at any time, a portable programming environment. Just download it, extract to your portable storage device or hard drive and in 10 minutes you are ready to create your next Python® application." After packaging the portable python and your .py or .pyc file then create a .bat file that runs the portable python "Python-Portable.exe" with the correct command line parameters for loading your script. Be sure to use relative paths in the batch file in case they are running it from a flash drive, or something other than the same location as you.
NOTE: This is really not a good way to do this as thier download page states: "Installed size: based on selected packages, between 49MB and 480MB". Also be sure to read the the current Python Software Foundation License, as that is what Portable Python is released under, and it may or may not be legal to package it in a closed source project. I haven't really looked at the license myself to be able to tell you. If you are releasing it as open source, then there would not be an issue though. As a quick side note, if you need that .bat file to be a .exe file then you can use a .bat to .exe converter battoexe.com is one. This is really going the long way about doing the whole thing, but it is an option.
Sources:
Working with Python on and off for 7 years now, a lot that using a portable version on a flash drive, and also dealing with Batch files much longer.
I am about to start a personal project using python and I will be using it on both Linux(Fedora) and Windows(Vista), Although I might as well make it work on a mac while im at it. I have found an API for the GUI that will work on all 3. The reason I am asking is because I have always heard of small differences that are easily avoided if you know about them before starting. Does anyone have any tips or suggestions that fall along these lines?
In general:
Be careful with paths. Use os.path wherever possible.
Don't assume that HOME points to the user's home/profile directory.
Avoid using things like unix-domain sockets, fifos, and other POSIX-specific stuff.
More specific stuff:
If you're using wxPython, note that there may be differences in things like which thread certain events are generated in. Don't assume that events are generated in a specific thread. If you're calling a method which triggers a GUI-event, don't assume that event-handlers have completed by the time your method returns. (And vice versa, of course.)
There are always differences in how a GUI will appear. Layouts are not always implemented in the exact same way.
Some things I've noticed in my cross platform development in Python:
OSX doesn't have a tray, so application notifications usually happen right in the dock. So if you're building a background notification service you may need a small amount of platform-specific code.
os.startfile() apparently only works on Windows. Either that or Python 2.5.1 on Leopard doesn't support it.
os.normpath() is something you might want to consider using too, just to keep your paths and volumes using the correct slash notation and volume names.
icons are dealt with in fundamentally different ways in Windows and OSX, be sure you provide icons at all the right sizes for both (16x16, 24x24, 32x32, 48x48, 64x64, 128x128 and 256x256) and be sure to read up on setting up icons with wx widgets.
You should take care of the Python version you are developing against. Especially, on a Mac, the default version of Python installed with the OS, is rather old (of course, newer versions can be installed)
Don't use the OS specific libraries
Take special care of 'special' UI elements, like taskbar icons (windows), ...
Use forward slashes when using paths, avoid C:/, /home/..., ... Use os.path to work with paths.
Some filename problems: This.File and this.file are different files on Linux, but point to the same file on Windows. Troublesome if you manage some file repository and access it from both platforms. Less frequent related problem is that of names like NUL or LPT being files on Windows.
Binary distribution code (if any) would likely use py2exe on Win, py2app on Mac and wouldn't be present on Linux.