I am trying to convert my Python program into a stand-alone executable for Windows and Linux. It seemed that PyInstaller was a good solution for this task. It got converted very easily under Windows, however I cannot make it working with Linux (was probing it on two different Ubuntu machines). The original Python program could be reduced to the simple two-line code:
from numpy import log
print '%8.6f' % log(5)
Which is working fine when called like:
anton#op:~/bin$ python test_numpy.py
1.609438
After compilation in PyInstaller, it gives me that error:
anton#op:~/bin/dist/test_numpy$ ./test_numpy
Traceback (most recent call last):
File "<string>", line 3, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 270, in load_module
exec(bytecode, module.__dict__)
File "/home/anton/bin/build/test_numpy/out00-PYZ.pyz/numpy", line 153, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 270, in load_module
exec(bytecode, module.__dict__)
File "/home/anton/bin/build/test_numpy/out00-PYZ.pyz/numpy.add_newdocs", line 13, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 270, in load_module
exec(bytecode, module.__dict__)
File "/home/anton/bin/build/test_numpy/out00-PYZ.pyz/numpy.lib", line 8, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 270, in load_module
exec(bytecode, module.__dict__)
File "/home/anton/bin/build/test_numpy/out00-PYZ.pyz/numpy.lib.type_check", line 11, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 270, in load_module
exec(bytecode, module.__dict__)
File "/home/anton/bin/build/test_numpy/out00-PYZ.pyz/numpy.core", line 6, in <module>
ImportError: cannot import name multiarray
So the problem is related to numpy library been not imported correctly.
I have tried adding hidden hooks as pyinstaller suggested, but nothing helps and I am always getting this error. Any ideas?
I was finally able to resolve this problem.
This is what I have found on hithub:
The executable that PyInstaller builds is not fully static, in that it still depends on the system libc. Under Linux, the ABI of GLIBC is backward compatible, but not forward compatible. So if you link against a newer GLIBC, you can't run the resulting executable on an older system. The supplied binary bootloader should work with older GLIBC. However, the libpython.so and other dynamic libraries still depends on the newer GLIBC. The solution is to compile the Python interpreter with its modules (and also probably bootloader) on the oldest system you have around, so that it gets linked with the oldest version of GLIBC.
So it appeared that the current stable PyInstaller version could be outdated. As it was not practical to downgrade all the packages, I upgraded the PyInstaller to the latest development release (3.0.dev2) and voilà! Everything is working now with all the dependencies correctly processed without any custom hooks.
Hope that could help someone.
Related
I use pyinstaller to package my python program into an executable program(exe).
my programs versions:
Pyinstaller :3.3.1
Python :3.6.1
Windows:10-10.0.16299-SP0
I can run the data_processing_gui.py file with the command "python data_processing_gui.py" in my command-line window without any errors.
I used this command to package my python program (which its name is data_processing_gui.py):
pyinstaller -F -p C:\Users\fanyu\Anaconda3\Lib;C:\Users\fanyu\Anaconda3\Lib\site-packages;C:\Users\fanyu\Anaconda3\Lib\site-packages\PyQt5\Qt\bin;C:\Users\fanyu\Anaconda3\Lib\site-packages\pydicom;C:\Users\fanyu\Anaconda3\Lib\site-packages\radiomics;C:\Users\fanyu\Anaconda3\Lib\site-packages\pywt;C:\Users\fanyu\Anaconda3\Lib\site-packages\pywt\_extensions;C:\Users\fanyu\Anaconda3\Lib\site-packages\tensorflow;C:\Users\fanyu\Anaconda3\Lib\site-packages\SimpleITK;C:\Users\fanyu\Anaconda3\Lib\site-packages\PIL;C:\Users\fanyu\Anaconda3\Lib\site-packages\pandas;C:\Users\fanyu\Anaconda3\Lib\site-packages\nrrd;C:\Users\fanyu\Anaconda3\Lib\site-packages\numpy -i C:\Users\fanyu\Desktop\work\package\image\doctor.ico data_processing_gui.py
There is no error when I run code above in windows' command-line window, but when I run the target program(data_progressing_gui.exe), there is a ModuleNotFoundError, and the error information is:
Traceback (most recent call last): File "data_processing_gui.py",
line 12, in File
"c:\users\fanyu\anaconda3\lib\site-packages\PyInstaller\loader\pyimod03_importers.py",
line 631, in exec_module
exec(bytecode, module.dict) File "site-packages\radiomics_init_.py", line 15, in File
"c:\users\fanyu\anaconda3\lib\site-packages\PyInstaller\loader\pyimod03_importers.py",
line 631, in exec_module
exec(bytecode, module.dict) File "site-packages\radiomics\imageoperations.py", line 6, in
File
"c:\users\fanyu\anaconda3\lib\site-packages\PyInstaller\loader\pyimod03_importers.py",
line 631, in exec_module
exec(bytecode, module.dict) File "site-packages\pywt_init_.py", line 17, in File
"c:\users\fanyu\anaconda3\lib\site-packages\PyInstaller\loader\pyimod03_importers.py",
line 631, in exec_module
exec(bytecode, module.dict) File "site-packages\pywt_functions.py", line 17, in File
"c:\users\fanyu\anaconda3\lib\site-packages\PyInstaller\loader\pyimod03_importers.py",
line 714, in load_module
module = loader.load_module(fullname) File "pywt/_extensions/_pywt.pyx", line 1, in init pywt._extensions._pywt
(pywt_extensions_pywt.c:31281) ModuleNotFoundError: No module named
'pywt._extensions._cwt' [7564] Failed to execute script
data_processing_gui
It seems like the program can't find the specified package path (pywt._extensions._cwt), however, I've added the path in the pyinstaller's parameter list.
At first, I thought maybe there was something wrong with my pyinstaller, so I tried to reinstall it. Then I write a simple python program to test it, it runs correctly.
I've tried a lot but just can't solve it, so I am here to ask for help,I will appreciate it if anyone can give me any advice.
I find a solution for my problem in the answers to the following question ModuleNotFoundError: No module named 'pandas._libs.tslibs.timedeltas'
I can't explain why but it does solve my problem.
The solution is:
After generating the ".spec" file using pyinstaller,add the missing module into the ".spec" file at "hiddenimport=[]",like this:
hiddenimport=["pywt","pywt._estentions._cwt"]
Then use this file to generate your exe file:
pyinstaller my.spec
When I use the mayavi (4.3.1) to visualize a mesh, it pronpts the following warning:
WARNING: Imported VTK version (5.1) does not match the one used
to build the TVTK classes (6.3). This may cause problems.
Please rebuild TVTK.
My system:
MS Windows 7 64bits, Anaconda 4.3.1 (latest version),
<code>
$ conda list qt
# packages in environment at C:\ProgramData\Anaconda2:
#
pyqt 4.11.4 py27_7 anaconda
qt 4.8.7 vc9_9 [vc9] anaconda
qtawesome 0.4.3 py27_0
qtconsole 4.2.1 py27_0 anaconda
qtpy 1.2.1 py27_0
$ conda list vtk
# packages in environment at C:\ProgramData\Anaconda2:
#
vtk 5.10.1 py27_0 anaconda
$ conda list mayavi
# packages in environment at C:\ProgramData\Anaconda2:
#
mayavi 4.5.0 <pip>
</code>
I have searched the forums, but most answers regarded it as a bug in VTK 4 years ago, for example, TVTK Error in Mayavi (Python) . I am not sure it has an determined solution. In my computer, indeed there is another version of VTK (version>6) but it is in C++ and has nothing with python and Mayavi.
The error I encountered occurred when I run the Maya sample code, as follows (The warning message is extremely confusing since my VTK is 5.10 (as above) and it is strange where is the version 5.1),
<code>
>>> runfile('D:/flex_8state/code/spherical_harmonics.py', wdir='D:/flex_8state/code')
********************************************************************************
WARNING: Imported VTK version (5.1) does not match the one used
to build the TVTK classes (6.3). This may cause problems.
Please rebuild TVTK.
********************************************************************************
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\ProgramData\Anaconda2\lib\site-packages\spyder\utils\site\sitecustomize.py", line 866, in runfile
execfile(filename, namespace)
File "C:\ProgramData\Anaconda2\lib\site-packages\spyder\utils\site\sitecustomize.py", line 87, in execfile
exec(compile(scripttext, filename, 'exec'), glob, loc)
File "D:/flex_8state/code/spherical_harmonics.py", line 36, in <module>
mlab.figure(1, bgcolor=(1, 1, 1), fgcolor=(0, 0, 0), size=(400, 300))
File "C:\ProgramData\Anaconda2\lib\site-packages\mayavi\tools\figure.py", line 85, in figure
engine.new_scene(name=name, size=size)
File "C:\ProgramData\Anaconda2\lib\site-packages\apptools\scripting\recordable.py", line 45, in _wrapper
result = func(*args, **kw)
File "C:\ProgramData\Anaconda2\lib\site-packages\mayavi\core\engine.py", line 452, in new_scene
viewer = self.scene_factory(**factory_kwargs)
File "C:\ProgramData\Anaconda2\lib\site-packages\mayavi\core\ui\mayavi_scene.py", line 89, in viewer_factory
viewer.open()
File "C:\ProgramData\Anaconda2\lib\site-packages\pyface\i_window.py", line 166, in open
self._create()
File "C:\ProgramData\Anaconda2\lib\site-packages\pyface\ui\qt4\application_window.py", line 121, in _create
contents = self._create_contents(self.control)
File "C:\ProgramData\Anaconda2\lib\site-packages\tvtk\tools\ivtk.py", line 404, in _create_contents
self.scene = self._scene_factory(parent)
File "C:\ProgramData\Anaconda2\lib\site-packages\mayavi\core\ui\mayavi_scene.py", line 68, in mayavi_scene_factory
s = MayaviScene(parent, stereo=p['stereo'])
File "C:\ProgramData\Anaconda2\lib\site-packages\tvtk\pyface\ui\qt4\decorated_scene.py", line 59, in __init__
super(DecoratedScene, self).__init__(parent, **traits)
File "C:\ProgramData\Anaconda2\lib\site-packages\tvtk\pyface\ui\qt4\scene.py", line 325, in __init__
self.picker = picker.Picker(self)
File "C:\ProgramData\Anaconda2\lib\site-packages\tvtk\pyface\picker.py", line 260, in __init__
configure_input(self.p_mapper, self.p_source)
File "C:\ProgramData\Anaconda2\lib\site-packages\tvtk\common.py", line 88, in configure_input
inp.input = op.output
File "C:\ProgramData\Anaconda2\lib\site-packages\traits\trait_handlers.py", line 104, in _read_only
name, class_of( object ) )
traits.trait_errors.TraitError: The 'input' trait of a PolyDataMapper instance is 'read only'.
</code>
After 3-day hard striving, I have found a workaround and I posted here which would be helpful to the similar victims. I strongly suggested the developers of Mayavi/TVTK pay more attention about this bug.
This post helped me much, https://github.com/ContinuumIO/anaconda-issues/issues/1510 . The same initial installation and then update (my initial post) the VTK from 5 to 6, as listed in https://repo.continuum.io/pkgs/free/win-64/ . The intrinsic error rooted in altered SetInputConnection/SetInputData interface across python/VTK versions.
Major concerns
In my notebook PC, the same installation (as the above post) indeed
works. I don't why.
In my PC, before reach the workaround, when I
directly installed VTK-6 by conda, it didn't work and prompted
"...missing modules" (sorry I didn't record these errors).
I have a gui that I created with wx and a bunch of other libraries like matlabplot, win32api etc...
I have compiled the python code into an executable using py2exe.
The executable gets created but when I try to run it, it produces an error.
Traceback (most recent call last):
File "gui.py", line 30, in <module>
File "zipextimporter.pyc", line 82, in load_module
File "guiupdater.pyc", line 22, in <module>
File "zipextimporter.pyc", line 82, in load_module
File "paramiko\__init__.pyc", line 30, in <module>
File "zipextimporter.pyc", line 82, in load_module
File "paramiko\transport.pyc", line 32, in <module>
File "zipextimporter.pyc", line 82, in load_module
File "cryptography\hazmat\backends\__init__.pyc", line 7, in <module>
File "zipextimporter.pyc", line 82, in load_module
File "pkg_resources\__init__.pyc", line 70, in <module>
File "pkg_resources\extern\__init__.pyc", line 61, in load_module
ImportError: The 'appdirs' package is required; normally this is bundled with this package so if you get this warning, consult the packager of your distribution.
I had previously compiled the executable and had it running in a different environment. I'm on a new environment now, but I have all the same dependencies installed etc...
I did some quick research and lots of people say reverting their setuptools to 19.2 version fixed it for them but I am relunctent to do that because my setuptools is at version 28.2. I feel like it would do more harm than good...
Does anyone know how to fix this error?
I had the same problem and as I did not want to go far back with versions I had to do some research...
Problem is that while processing pkg_resources.extern all necessary modules from (private?) default package pkg_resources._vendor are not found (delayed load).
Compare contents of pkg_resources._vendor at source (site-packages) and inside library.zip that is generated with executable. Whatever is missing in zip can be included by options for py2exe.
options={"py2exe": {"includes": ["pkg_resources._vendor.appdirs",...], ...
It's an issue introduced with newer versions of setuptools. Either:
add 'pkg_resources' to setuptools.setup packages option
or
Downgrade to version 19.2
pip install setuptools==19.2
I'm trying to freeze with either cx_freeze or pyInstaller a TraitsUI program that makes use of Chaco, Traits, TraitsUI and to a lesser extent mayavi (could actually be taken out). I need this to run on mac, linux, ubuntu so am avoiding py2exe. I've intentionally uninstalled pyqt and pyside so that only the wx backend is available.
Using cx_freeze, I encountered and reported a bug, so pyInstaller seems to be getting me the furthest. It generated an .exe file, but when I run the file I get warnings and an error. To generate the .exe file I did:
pyinstaller --onefile --name=pameapp pamemain.py
pyinstaller pameapp.exe
Upon running, the warnings are:
Warning: Unable to import the wx backend for pyface due to traceback: Traceback (most recent call last):
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/pyface.toolkit", line 45, in _init_toolkit
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/pyface.toolkit", line 31, in import_toolkit
ImportError: No module named wx.init
Warning: Unable to import the qt4 backend for pyface due to traceback: Traceback (most recent call last):
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/pyface.toolkit", line 45, in _init_toolkit
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/pyface.toolkit", line 31, in import_toolkit
ImportError: No module named init
Warning: Unable to import the null backend for pyface due to traceback: Traceback (most recent call last):
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/pyface.toolkit", line 45, in _init_toolkit
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/pyface.toolkit", line 31, in import_toolkit
ImportError: No module named null.init
In addition, I get the following error:
Traceback (most recent call last):
File "<string>", line 8, in <module>
File "/home/glue/anaconda/envs/fibersim/lib/python2.7/site-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
exec(bytecode, module.__dict__)
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/traitsui.api", line 35, in <module>
File "/home/glue/anaconda/envs/fibersim/lib/python2.7/site-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
exec(bytecode, module.__dict__)
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/traitsui.editors", line 22, in <module>
File "/home/glue/anaconda/envs/fibersim/lib/python2.7/site-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
exec(bytecode, module.__dict__)
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/traitsui.editors.api", line 10, in <module>
File "/home/glue/anaconda/envs/fibersim/lib/python2.7/site-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
exec(bytecode, module.__dict__)
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/traitsui.editors.code_editor", line 36, in <module>
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/traitsui.editors.code_editor", line 48, in ToolkitEditorFactory
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/traits.traits", line 487, in __call__
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/traits.traits", line 1191, in Color
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/traitsui.toolkit_traits", line 7, in ColorTrait
File "/home/glue/Desktop/fibersim/pame/build/pameapp/out00-PYZ.pyz/traitsui.toolkit", line 137, in toolkit
traits.trait_errors.TraitError: Could not find any UI toolkit called 'null'
pyface probably has a lot of dynamic imports, so this must be a problem encountered by anyone who tries to freeze ETS. Is there a single fix to get this working?
Considering the older stuff floating around the web on success rates to freezing an ETS program, is it even worth further investment? To us, having people avoid configuring python for this application would be a huge plus.
Thanks.
Pyface.toolkit you are using is using dynamic import:
def import_toolkit(tk):
try:
# Try and import the toolkit's pyface backend init module.
be = 'pyface.ui.%s.' % tk
__import__(be + 'init')
except:
raise
return be
So pyinstaller is not able to detect this. Check with python -v b.py 2>&1 | grep imp what is the specific import and add hook file (hook-.py) with proper hiddenimports list.
However I've did some tests in local and I am not able to do working test schema with __import__. It's strange as I did some debugging and I found that hiddenimport is added but still not usable.
I have Mercurial 1.8.1, Python 2.6.6 installed on Win 2k8 R2 running on a vm. I have tried installing from msi, source and using tortisehg. Command-line Hg works fine but I get the same error when running the hgweb.cgi:
Traceback (most recent call last):
File ".\hgweb.cgi", line 17, in
application = hgweb(config)
File "mercurial\hgweb\__init__.pyc", line 26, in hgweb
File "mercurial\hgweb\hgwebdir_mod.pyc", line 61, in __init__
File "mercurial\hgweb\hgwebdir_mod.pyc", line 70, in refresh
File "mercurial\ui.pyc", line 35, in __init__
File "mercurial\demandimport.pyc", line 75, in __getattribute__
File "mercurial\demandimport.pyc", line 47, in _load
File "mercurial\util.pyc", line 576, in
File "mercurial\demandimport.pyc", line 85, in _demandimport
File "mercurial\windows.pyc", line 21, in
File "mercurial\demandimport.pyc", line 75, in __getattribute__
File "mercurial\demandimport.pyc", line 47, in _load
File "mercurial\osutil.pyc", line 12, in
File "mercurial\osutil.pyc", line 10, in __load
ImportError: DLL load failed: The specified module could not be found.
The other answers I have found on SO and elsewhere pointed me to try installing from source, dropping the pure osutil into the install, or installing an older version. I have tried them all.
This is especially frustrating because I have other, similar non-vm machines running fine but have been unable to find the disconnect.
Ideas?
I had the same error using following system configuration
Python-2.6.6 installed as msi
mercurial-1.8.2-x86 installed as msi
IIS7
I solved this problem simply:
Python has been installed early
Uninstall Mercurial msi package
Download and install "Mercurial-1.8.2 (32-bit py2.6)" installer from mercurial website which is marked as "This is recommended for hgweb setups".
copyed content of C:\Python26\Lib\site-packages\mercurial\ to the directory used in IIS7 website setup.
Till now all is working. Hope this will help.
Whenever I have less than descriptive error messages that tell me something is going on at the system level but not what, I use Sysinternals' Procmon to tell me what's going with the registry and filesystem. It's verbose, and getting the filter to show just the process of interest takes some learning, but you can export the results to Excel and skim them for suspicious-looking results. Pay particular attention to failures, of course.
Give it a try and see what DLL is being searched for.