I'm using py2app to make a wxPython frozen binary that uses matplotlib (whew). The resulting executable is quite large, so I'd like to whittle it down if possible.
One obvious way of doing this is to exclude unneeded matplotlib backends. I've tried running my script with python setup.py py2app --matplotlib-backends - which should compile only backends that are used in the script, as documented here: https://pythonhosted.org/py2app/options.html. The resulting executable is indeed a lot smaller, but it's broken. I also tried python setup.py py2app --matplotlib-backends "WXAgg" which failed the same way. Here is most of the traceback (awkwardly copied from Console, sorry).
import wx
File "wx/__init__.pyc", line 45, in <module>
File "wx/_core.pyc", line 4, in <module>
File "wx/_core_.pyc", line 14, in <module>
File "wx/_core_.pyc", line 10, in __load
ImportError: dlopen(/Users/***/***/***/dist/magic_gui.app/Contents/Resources/lib/python2.7/lib-dynload/wx/_core_.so, 2): Library not loaded: #rpath/lib/libwx_osx_cocoau-3.0.dylib
Referenced from: /Users/***/***/***/dist/magic_gui.app/Contents/Resources/lib/python2.7/lib-dynload/wx/_core_.so
Reason: image not found
Maybe I can explicitly tell it to include libwx_osx_cocoau-3.0.dylib. Or, maybe I need to explicitly include another backend (although I am using wxPython, so the WXAgg backend should be right).
I have two questions: first, is there a way I can include only required matplotlib backends without breaking my program? Second, are there other ways I could/should consider to make my program smaller with py2app?
Related
I am using Imatest IT (v5.0.1) and need to use Imatest IT tests with my Python code, however, my Python code also interfaces with MATLAB’s (2017b) Python API and there seems to be a conflict.
You can see below that simply trying to import ImatestLibrary after importing the matlab.engine results in an exception.
Just importing one or the other causes no problem.
import matlab.engine
from imatest.it import ImatestLibrary
Exception caught during initialization of Python interface. Details: DLL load failed: The specified procedure could not
be found.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python34\lib\site-packages\imatest\it.py", line 6, in <module>
import imatest.library
File "C:\Python34\lib\site-packages\imatest\library\__init__.py", line 279, in <module>
_pir.import_cppext()
File "C:\Python34\lib\site-packages\imatest\library\__init__.py", line 272, in import_cppext
self.cppext_handle = importlib.import_module("matlabruntimeforpython" + self.interpreter_version)
File "C:\Python34\lib\importlib\__init__.py", line 109, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: DLL load failed: The specified procedure could not be found.
Is there a way to overcome this issue?
According to the documentation of the Imatest Python interface regarding importing imatest.it:
Behind the scenes, the ImatestLibrary constructor will start up the Matlab MCR Runtime and load all the IT libraries into memory.
This suggests that imatest itself meddles with the state of the MATLAB engine. While the order of imports usually doesn't matter, occasionally certain imports should happen before others for proper initialization, when modules imported later rely on modules imported earlier (for instance, importing matplotlib first and setting a backend before importing pyplot that would use said backend).
So my only suggestion is to try switching the order of imports to see if that helps. If it doesn't you're out of luck (as far as Stack Overflow is concerned): both MATLAB and Imatest are expensive proprietary (and even closed source) products so you should file bug reports to either or both of them, because nobody else will be able to tell if and how the conflict could be fixed.
Which version of python are you using? It's possible it's not one of Matlabs supported versions for 2017b, which are limited. I forget exactly which but I think 2.7, 3.3, 3.4, and possibly 3.5
The workaround to this I used was to create new processes using Python's multiprocessing module. The processes would use either the MATLAB libraries or the Imatest libraries. An extra layer of code but it works.
I had a simple app up and running that used the PyGui package. After I decided I had everything working I refactored my code and basically broke it up over multiple files, but I am fairly sure I did not change the location of any files. Now when I try to run my app It gives me the following error:
Traceback (most recent call last):
File "blobedit.py", line 16, in <module>
from GUI import Application, ScrollableView, Document, Window, Cursor, rgb
File "/Users/<myusername>/Desktop/BlobEdit/GUI/__init__.py", line 54, in <module>
raise ImportError("Unable to find an implementation of PyGUI for this installation")
ImportError: Unable to find an implementation of PyGUI for this installation
I have tried moving files around for weeks and can't find the problem. Any help would be much appreciated!
You have a directory GUI and inside it __init__.py, this makes the directory an importable package.
Unfortunately, PyGUI also provides a GUI package. Due to the way Python searches for packages to import, your GUI has come up first and this is what is causing problems.
The simple way to solve it is to rename your GUI directory /Users/Destkop/BlobEdit/GUI to something else.
I'm bundling a pygame app using py2app. The bundling works and the resulting bundle runs on my mac just fine. It also used to run on another person's mac just fine. However, recently I've started getting this error (from the console) when trying to run the bundle on his computer:
Traceback (most recent call last):
File "/Users/.../tmp/withconsole.app/Contents/Resources/__boot__.py", line 316, in <module>
_run()
File "/Users/.../tmp/withconsole.app/Contents/Resources/__boot__.py", line 311, in _run
exec(compile(source, path, 'exec'), globals(), globals())
File "/Users/.../tmp/withconsole.app/Contents/Resources/withconsole.py", line 18, in <module>
pdb.set_trace()
File "bdb.pyc", line 53, in trace_dispatch
File "bdb.pyc", line 88, in dispatch_return
File "pdb.pyc", line 190, in user_return
File "pdb.pyc", line 210, in interaction
File "cmd.pyc", line 142, in cmdloop
File "pdb.pyc", line 279, in onecmd
File "cmd.pyc", line 218, in onecmd
File "pygame/macosx.pyc", line 10, in <module>
File "pygame/sdlmain_osx.pyc", line 14, in <module>
File "pygame/sdlmain_osx.pyc", line 10, in __load
ImportError: dlopen(/Users/.../tmp/withconsole.app/Contents/Resources/lib/python2.7/lib-dynload/pygame/sdlmain_osx.so, 2): Symbol not found: _OBJC_CLASS_$_NSObject
Referenced from: /Users/.../tmp/withconsole.app/Contents/Resources/lib/python2.7/lib-dynload/pygame/sdlmain_osx.so
Expected in: /usr/lib/libobjc.A.dylib
in /Users/.../tmp/withconsole.app/Contents/Resources/lib/python2.7/lib-dynload/pygame/sdlmain_osx.so
2013-11-09 06:19:50.794 withconsole[2797:1c03] ogclient Error
I even tried running an older bundle that used to work, yet now it no longer works! I'm 95% certain of that, anyway.
In any case, what's the issue and how can I fix it?
A standalone app from py2app creates its baked-in Python interpreter from whichever version of Python you use for py2app.
If you use, say, a Python built to run on any OS X 10.6 or later, the application (so long as it doesn't use any post-10.6 features) will also run on any OS X 10.6 or later. If you use a Python built to run on one particular 10.8 machine, the application will very likely not work on anything earlier than 10.8, and may not even work on other 10.8+ machines.
On top of that, if you're using any C extension modules that you built against C libraries that you built yourself, if those C libraries were built for your particular machine, they'll probably cause the same problem.
You built your Python interpreter with Homebrew. And maybe your SDL as well, which pygame depends on. And possibly other things. By default, Homebrew builds everything to run on your particular machine, not to be redistributable. Hence the problem.
To build an app that can run on 10.7 machines, you need to either build Python, SDL, etc. against the 10.7 SDK, or build it against a later API and specify -mmacosx-version-min=10.7. (There are a few other issues that can come up, but I'm pretty sure none of them affect CPython, so let's keep it simple.)
That will automatically set things up so any C extension modules you build use the same SDK and version settings, so you shouldn't have to worry about those. But you do have to worry about any C libraries you built that those extension modules depend on. For example, pygame links against libSDL, and if you built SDL for your local machine only, pygame isn't going to work on someone else's machine.
It's possible to get Homebrew to create SDK builds, but not always that easy. It's easier to just build Python manually and pass the right --configure flags. But it's even easier to take a binary that someone else has already built for portability and use that.
The official Python installers at python.org are built for 10.6+. The official SDL development libraries are built for 10.5+. Hopefully the same is true for other things you're depending on. If so, just brew uninstall (or, if you're wary and want to be able to undo this later, brew unlink) everything, run the binary installers, reinstall all the Python modules you need for the new Python, and py2app with it.
I wrote this all in general terms. From the output, it looks like the specific problem you're hitting on this particular run on your friend's particular 10.7 machine is that the SDL wrappers are depending on features of the ObjC runtime that didn't exist in 10.7. So, it's possible that just using a 10.7-friendly SDL, and your existing Python, would be sufficient. But I think you're better off using 10.7-friendly everything if at all possible.
If you're wondering what exactly the SDK does, the SDK Compatibility Guide in Apple's official docs explain the technical side of it pretty well, and there are a zillion blog posts like this one that explain the practical parts, but I'll try to summarize.
Each SDK is just a directory that has its own /usr/include, /usr/lib, /System/Library/Frameworks, and a few other things inside. When you build against an SDK, you end up compiling against its header files instead of your system's, and linking against its dylibs instead of your system's. This prevents you from using any new features added since 10.6 (with a nice compiler error, instead of a successful build that then won't start on half your users' machines). And it makes sure that, if you don't use any such new features, your program doesn't have load-time dependencies on anything that didn't exist in 10.6.
While attempting to "freeze" a Python (v2.78) app which uses PyQt4 (v4.10.4) and Matplotlib (v1.4.2) under Windows XP SP3, using PyInstaller (v2.1), I am running into an annoying problem.
At the very beginning of my program, I have the following lines:
import matplotlib
matplotlib.use("Qt4Agg")
matplotlib.rcParams["backend.qt4"] = "PyQt4"
from matplotlib.backends.backend_qt4agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.figure import Figure
When I run PyInstaller using the line:
pyinstaller --noconsole --icon=app.ico App.py
the application is correctly frozen into the "dist" folder and it runs without any hassles. The problem is, a lot of unwanted files are included into the distribution folder! By these "unwanted" files, I refer to the several "wx" modules which are obviously useless to my application (since it uses Qt and not Wx for the UI). If I manually delete these files from the dist folder, the app still runs very well.
So, my big question is: how do I get rid of these unwanted files when freezing the app with PyInstaller?
I could find several possible solutions for similar problems with py2exe and cx_Freeze, but nothing that refers to PyInstaller (and these is my preferred "compiler" because it has been the only cross-platform one that seems to work very well under Windows and Linux).
Thanks in advance for any assistance you can provide!
EDIT: Also, when running PyInstaller, I got an ugly error message:
43592 INFO: Processing hook hook-matplotlib
Traceback (most recent call last):
File "<string>", line 3, in <module>
File "C:\Python27\lib\site-packages\matplotlib\backends\backend_webagg.py", li
ne 30, in <module>
raise RuntimeError("The WebAgg backend requires Tornado.")
RuntimeError: The WebAgg backend requires Tornado.
But of course I am not using any WebAgg backend (and have no idea what the heck is "Tornado"!).
By the end of the freezing process, I got a warning:
265046 WARNING: lib not found: gdiplus.dll dependency of C:\Python27\lib\site-pa
ckages\wx-2.8-msw-unicode\wx\wxmsw28u_core_vc.dll
I also would like to get rid of both messages, during the freezing process (which I presume I will be, when the problem of getting rid of unwanted backends is solved)!
"
I'm bundling a pygame app using py2app. The bundling works and the resulting bundle runs on my mac just fine. It also used to run on another person's mac just fine. However, recently I've started getting this error (from the console) when trying to run the bundle on his computer:
Traceback (most recent call last):
File "/Users/.../tmp/withconsole.app/Contents/Resources/__boot__.py", line 316, in <module>
_run()
File "/Users/.../tmp/withconsole.app/Contents/Resources/__boot__.py", line 311, in _run
exec(compile(source, path, 'exec'), globals(), globals())
File "/Users/.../tmp/withconsole.app/Contents/Resources/withconsole.py", line 18, in <module>
pdb.set_trace()
File "bdb.pyc", line 53, in trace_dispatch
File "bdb.pyc", line 88, in dispatch_return
File "pdb.pyc", line 190, in user_return
File "pdb.pyc", line 210, in interaction
File "cmd.pyc", line 142, in cmdloop
File "pdb.pyc", line 279, in onecmd
File "cmd.pyc", line 218, in onecmd
File "pygame/macosx.pyc", line 10, in <module>
File "pygame/sdlmain_osx.pyc", line 14, in <module>
File "pygame/sdlmain_osx.pyc", line 10, in __load
ImportError: dlopen(/Users/.../tmp/withconsole.app/Contents/Resources/lib/python2.7/lib-dynload/pygame/sdlmain_osx.so, 2): Symbol not found: _OBJC_CLASS_$_NSObject
Referenced from: /Users/.../tmp/withconsole.app/Contents/Resources/lib/python2.7/lib-dynload/pygame/sdlmain_osx.so
Expected in: /usr/lib/libobjc.A.dylib
in /Users/.../tmp/withconsole.app/Contents/Resources/lib/python2.7/lib-dynload/pygame/sdlmain_osx.so
2013-11-09 06:19:50.794 withconsole[2797:1c03] ogclient Error
I even tried running an older bundle that used to work, yet now it no longer works! I'm 95% certain of that, anyway.
In any case, what's the issue and how can I fix it?
A standalone app from py2app creates its baked-in Python interpreter from whichever version of Python you use for py2app.
If you use, say, a Python built to run on any OS X 10.6 or later, the application (so long as it doesn't use any post-10.6 features) will also run on any OS X 10.6 or later. If you use a Python built to run on one particular 10.8 machine, the application will very likely not work on anything earlier than 10.8, and may not even work on other 10.8+ machines.
On top of that, if you're using any C extension modules that you built against C libraries that you built yourself, if those C libraries were built for your particular machine, they'll probably cause the same problem.
You built your Python interpreter with Homebrew. And maybe your SDL as well, which pygame depends on. And possibly other things. By default, Homebrew builds everything to run on your particular machine, not to be redistributable. Hence the problem.
To build an app that can run on 10.7 machines, you need to either build Python, SDL, etc. against the 10.7 SDK, or build it against a later API and specify -mmacosx-version-min=10.7. (There are a few other issues that can come up, but I'm pretty sure none of them affect CPython, so let's keep it simple.)
That will automatically set things up so any C extension modules you build use the same SDK and version settings, so you shouldn't have to worry about those. But you do have to worry about any C libraries you built that those extension modules depend on. For example, pygame links against libSDL, and if you built SDL for your local machine only, pygame isn't going to work on someone else's machine.
It's possible to get Homebrew to create SDK builds, but not always that easy. It's easier to just build Python manually and pass the right --configure flags. But it's even easier to take a binary that someone else has already built for portability and use that.
The official Python installers at python.org are built for 10.6+. The official SDL development libraries are built for 10.5+. Hopefully the same is true for other things you're depending on. If so, just brew uninstall (or, if you're wary and want to be able to undo this later, brew unlink) everything, run the binary installers, reinstall all the Python modules you need for the new Python, and py2app with it.
I wrote this all in general terms. From the output, it looks like the specific problem you're hitting on this particular run on your friend's particular 10.7 machine is that the SDL wrappers are depending on features of the ObjC runtime that didn't exist in 10.7. So, it's possible that just using a 10.7-friendly SDL, and your existing Python, would be sufficient. But I think you're better off using 10.7-friendly everything if at all possible.
If you're wondering what exactly the SDK does, the SDK Compatibility Guide in Apple's official docs explain the technical side of it pretty well, and there are a zillion blog posts like this one that explain the practical parts, but I'll try to summarize.
Each SDK is just a directory that has its own /usr/include, /usr/lib, /System/Library/Frameworks, and a few other things inside. When you build against an SDK, you end up compiling against its header files instead of your system's, and linking against its dylibs instead of your system's. This prevents you from using any new features added since 10.6 (with a nice compiler error, instead of a successful build that then won't start on half your users' machines). And it makes sure that, if you don't use any such new features, your program doesn't have load-time dependencies on anything that didn't exist in 10.6.