I am distributing an app that uses the Python/C API. I have all standard python modules in python31.zip which is basically an archive of the Lib folder in the python install directory. Here is the problem - most common modules like sys and io work fine. BUT tkinter does not. I get an error "cannot find module _tkinter". I really need tkinter in my project. I'm using Windows if that helps.
I don't the best way to bundle tkinter with your app, but I do know why you're getting the error you are. The relevant section of the zipimport module documentation:
Any files may be present in the ZIP archive, but only files .py and .py[co] are available for import. ZIP import of dynamic modules (.pyd, .so) is disallowed. Note that if an archive only contains .py files, Python will not attempt to modify the archive by adding the corresponding .pyc or .pyo file, meaning that if a ZIP archive doesn’t contain .pyc files, importing may be rather slow.
The module _tkinter is a c-extension / shared library. It can't be imported from a zip file.
why don't you use py2exe to bundle your application as an executable? It should take care of all the dependencies, and will include whatever you need.
Related
I am using PyInstaller to create a single-file executable. Is it possible for my script to perform an import such that i) the imported module is imported from the same directory as the exe (i.e. it's not packaged into the exe) and ii) that imported module can import other modules that were packaged into the exe?
The background here is that the imported module contains configuration that the user should be able to modify. This may include creation of custom derived classes and use of enums from the packaged modules.
I haven't found any advice on this, though it's a difficult search because there are so many similar topics that use basically the same keywords.
The following steps allow a Python module (named external_module here) outside of an executable created by PyInstaller to be imported and for that module to import modules that were bundled into the executable.
Add excludes=['external_module'] to the Analysis object used in the PyInstaller spec. This prevents external_module.py being bundled into the executable.
Add sys.path.append(os.path.dirname(sys.executable)) where external_module is imported in your application. This allows it to be imported from the directory the executable is in, which is different to the directory that the application will run in (due to being decompressed to a temporary folder). See below for my recommended method of achieving this.
Make sure any imports performed by external_module.py are also performed by one of the bundled modules before external_module.py is imported. The interpreter will not resolve the external module's imports against bundled modules, but will use ones that already exist in sys.modules.
In order to set up the paths correctly you can use the following:
if getattr(sys, 'frozen', False):
app_path = os.path.dirname(sys.executable)
sys.path.append(app_path)
else:
app_path = os.path.dirname(os.path.abspath(__file__))
frozen is only available in generated executables, not when running as a script directly. This snippet will add the executable's location to sys.path if required as well as giving you easy access to the executable or script's location for use in code.
As an example of the final bullet point, consider the following.
# bundled_module1.py
import external_module
# bundled_module2.py
# module content
# external_module.py
import bundled_module2
This will fail in external_module.py because bundled_module2 can't be found. However, the following modification to bundled_module1.py will work:
# bundled_module1.py
import bundled_module2
import external_module
This will be fine if there are a limited set of bundled modules that the external one should be able to import. It may get unwieldy for larger sets.
Given that the documentation states that the interpreter will resolve imports against modules bundled into the executable, this feels like a possible bug. Interoperating with modules outside of the executable isn't explicitly called out though.
Type in Pyinstaller -h. It will give you info about pyinstaller and tell you about --runtime-hook. I presume adding this to the executable should work. There's actually a whole page of documentation for this. Surprised you could not find that.
Anyway,
The docs say put in:
pyinstaller --additional-hooks-dir=. myscript.py.
I presume then something like pyinstaller --additional-hooks-dir=C:\pathtofolder myscript.py should work in theory. Yet to test it. Tell us how it goes and what made kinks made it work for you.
Lastly, if you want to be hipster try integrating cython for speed and obfuscation. Fair warning, cython is not as user friendly as pyinstaller appears to be. I have yet to use it successfully.
Why is the import of *.so files from ZIP files disallowed in Python?
The documentation (https://docs.python.org/2/library/zipimport.html) is very clear:
Any files may be present in the ZIP archive, but only files .py and .py[co] are available for import. ZIP import of dynamic modules (.pyd, .so) is disallowed.
But the documentation doesn't name any reason for this strange limitation. Is is because importing from ZIP files is generally discouraged in Python? Or is it because of security reasons? If so, which ones? Is the any official statement about this?
From PEP 273, Subdirectory Equivalence:
You can't satisfy dynamic modules from a zip file. Dynamic modules
have extensions like .dll, .pyd, and .so. They are operating system
dependent, and probably can't be loaded except from a file. It might
be possible to extract the dynamic module from the zip file, write it
to a plain file and load it. But that would mean creating temporary
files, and dealing with all the dynload_*.c, and that's probably not a
good idea.
My interpretation is that this decision was made to avoid having the interpreter extract the dynamic module and save it to disk, as the OS's dynamic library facilities don't allow for loading from within a zip (See Windows' LoadLibraryA and Linux's dlopen).
While it looks like it's not technically impossible, a reason why doing the work to implement this functionality may not be worthwhile is the platform-dependence of these libraries. A .zip containing a dynamic module that is distributed with the code that relies on it may not work in a 32-bit environment if it was created from a 64-bit environment, and wouldn't work in Linux if it was created in Windows. Disallowing dynamic modules in .zips may be a conscious decision to ensure that .zips containing Python modules will work cross-platform.
I use ctypes on Windows to access a DLL. Generally, I wrap the DLL with a .py of the same name and add additional support for the DLL such as ctypes structs that are used with some of the functions. I load the DLL use the following technique.
# Determine the absolute path to the DLL that resides in the same directory as this module.
absolute_path = os.path.join(os.path.dirname(__file__), 'monty.dll')
interface = ctypes.CDLL(absolute_path)
The module is part of a package that I maintain and I'd like to use Py2exe with bundle_files set to 1 to create fewer files. I am happy to have library.zip sitting alongside my .exe.
So the obvious problem is that monty.py will be trying to access monty.dll and it will expect that path to be .../library.zip/root/dll/monty.dll.
I found a way to package the DLL in library.zip but I suspect ctypes is rejecting the path within the .zip file. Windows complains that it cannot load the module.
Getting the above to work would be my first choice but a fallback might be to have everything in library.zip except my package. Any ideas as to how this can be accomplished?
My python program uses plugins (python files) which I import dynamically using __import__.
I bundle my python program into a Windows exe using py2exe.
I've just spend searching 2 hours why my plugin python file couldn't be loaded properly from the .exe version. I got an ImportError: "no module named urllib2"
It appeared my plugin was using urllib2 (through an import urllib2 statement), and that standard library module was apparently not bundled into the exe. Other modules used in the plugin (re, urllib, ...) gave no problem, but perhaps they were already references in python files I statically include in my program.
How can I know which standard Python library modules py2exe bundles by default in the exe? (so I know whether I or somebody else can use them in a plugins). The py2exe documentation doesn't give an hints, except for saying that it includes a lot of modules from the standard library.
To see which modules are included look inside the library.zip (if there is no library.zip file - then try opening the EXE in any ZIP application - or rename it to .ZIP and try and open it).
You will be able to see a list of *.pyc. You can look at the list of files and directories to get an impression of which modules are included or not.
If you require a specific package to be added - add it to the 'packages' list.
As to why it doesn't include everything or how it chooses to include some and not others? My understanding is that py2exe looks in your code to figure out what you are using and includes those (and some that it probably needs itself) but maybe it also has some heuristics to add other modules too (I haven't checked :)
Currently, I'm deploying a full python distribution (the original python 2.7 msi) with my app. Which is an embedded web server made with delphi.
Reading this, I wonder if is possible to embed the necessary python files with my app, to decrease load files and avoid conflict with several python versions.
I have previous experience with python for delphi so I only need to know if only shipping the python dll + zip with the distro + own scripts will work (and if exist any caveats I must know or a sample where I can look)
zipimport should work just fine for you -- I'm not familiar with Python for Delphi, but I doubt it disables that functionality (an embedding application can do that, but it's an unusual choice). Just remember that what you can zip up and import directly are the Python-coded modules (or just their corresponding .pyc or .pyo byte codes) -- DLLs (even if renamed as .pyds;-) need to be on disk to be loaded (so if you have a zipfile with them it will need to be unzipped at the start of the app, e.g. into a temporary directory).
Moreover, you don't even need to zip up all modules, just those you actually need (by transitive closure) -- and you can easily find out exactly which modules those are, with the modulefinder module of the standard Python library. The example on the documentation page I just pointed to should clarify things. Happy zipping!
Yes it is possible.
I'm actually writing automatisation script in Python with the Zipimport library. I actually included every .py files in my zip as well as configuration or xml files needed by those script.
Then, I call a .command file targeting a __main__.py class that redirect towards the desired script according to my sys.argv parameters which is really useful!