My python program uses plugins (python files) which I import dynamically using __import__.
I bundle my python program into a Windows exe using py2exe.
I've just spend searching 2 hours why my plugin python file couldn't be loaded properly from the .exe version. I got an ImportError: "no module named urllib2"
It appeared my plugin was using urllib2 (through an import urllib2 statement), and that standard library module was apparently not bundled into the exe. Other modules used in the plugin (re, urllib, ...) gave no problem, but perhaps they were already references in python files I statically include in my program.
How can I know which standard Python library modules py2exe bundles by default in the exe? (so I know whether I or somebody else can use them in a plugins). The py2exe documentation doesn't give an hints, except for saying that it includes a lot of modules from the standard library.
To see which modules are included look inside the library.zip (if there is no library.zip file - then try opening the EXE in any ZIP application - or rename it to .ZIP and try and open it).
You will be able to see a list of *.pyc. You can look at the list of files and directories to get an impression of which modules are included or not.
If you require a specific package to be added - add it to the 'packages' list.
As to why it doesn't include everything or how it chooses to include some and not others? My understanding is that py2exe looks in your code to figure out what you are using and includes those (and some that it probably needs itself) but maybe it also has some heuristics to add other modules too (I haven't checked :)
Related
I've tried looking online and I'm honestly lost at this point.
I'm trying to find if there's a way to import python scripts and run them AFTER my Python program has been compiled.
For an example, let's say I have a main.py such that:
import modules.NewModule
a = NewModuleClass()
a.showYouWork()
then I compile main.py with pyinstaller so that my directory looks like:
main.exe
modules/NewModule.py
My end goal is to make a program that can dynamically read new Python files in a folder (this will be coded in) and use them (the part I'm struggling with). I know it's possible, since that's how add-ons work in Blender 3D but I've struggled for many hours to figure this out. I think I'm just bad at choosing the correct terms in Google.
Maybe I just need to convert all of the Python files in the modules directory to .pyc files? Then, how would I use them?
Also, if this is a duplicate on here (it probably is), please let me know. I couldn't find this issue on this site either.
You may find no detailed answer simply because there is no problem. PyInstaller does not really compile Python scripts into machine code executables. It just assembles then into a folder along with an embedded Python interpretor, or alternatively creates a compressed single file executable that will automatically uncompress itself at run time into a temporary folder containing that.
From then on, you have an almost standard Python environment, with normal .pyc file which can contain normal Python instructions like calls to importlib to dynamically load other Python modules. You have just to append the directory containing the modules to sys.path before importing them. An other possible caveat, is that pyinstaller only gets required modules and not a full Python installation, so you must either make sure that the dynamic modules do not rely on missing standard modules, or be prepared to face an ImportError.
I am using PyInstaller to create a single-file executable. Is it possible for my script to perform an import such that i) the imported module is imported from the same directory as the exe (i.e. it's not packaged into the exe) and ii) that imported module can import other modules that were packaged into the exe?
The background here is that the imported module contains configuration that the user should be able to modify. This may include creation of custom derived classes and use of enums from the packaged modules.
I haven't found any advice on this, though it's a difficult search because there are so many similar topics that use basically the same keywords.
The following steps allow a Python module (named external_module here) outside of an executable created by PyInstaller to be imported and for that module to import modules that were bundled into the executable.
Add excludes=['external_module'] to the Analysis object used in the PyInstaller spec. This prevents external_module.py being bundled into the executable.
Add sys.path.append(os.path.dirname(sys.executable)) where external_module is imported in your application. This allows it to be imported from the directory the executable is in, which is different to the directory that the application will run in (due to being decompressed to a temporary folder). See below for my recommended method of achieving this.
Make sure any imports performed by external_module.py are also performed by one of the bundled modules before external_module.py is imported. The interpreter will not resolve the external module's imports against bundled modules, but will use ones that already exist in sys.modules.
In order to set up the paths correctly you can use the following:
if getattr(sys, 'frozen', False):
app_path = os.path.dirname(sys.executable)
sys.path.append(app_path)
else:
app_path = os.path.dirname(os.path.abspath(__file__))
frozen is only available in generated executables, not when running as a script directly. This snippet will add the executable's location to sys.path if required as well as giving you easy access to the executable or script's location for use in code.
As an example of the final bullet point, consider the following.
# bundled_module1.py
import external_module
# bundled_module2.py
# module content
# external_module.py
import bundled_module2
This will fail in external_module.py because bundled_module2 can't be found. However, the following modification to bundled_module1.py will work:
# bundled_module1.py
import bundled_module2
import external_module
This will be fine if there are a limited set of bundled modules that the external one should be able to import. It may get unwieldy for larger sets.
Given that the documentation states that the interpreter will resolve imports against modules bundled into the executable, this feels like a possible bug. Interoperating with modules outside of the executable isn't explicitly called out though.
Type in Pyinstaller -h. It will give you info about pyinstaller and tell you about --runtime-hook. I presume adding this to the executable should work. There's actually a whole page of documentation for this. Surprised you could not find that.
Anyway,
The docs say put in:
pyinstaller --additional-hooks-dir=. myscript.py.
I presume then something like pyinstaller --additional-hooks-dir=C:\pathtofolder myscript.py should work in theory. Yet to test it. Tell us how it goes and what made kinks made it work for you.
Lastly, if you want to be hipster try integrating cython for speed and obfuscation. Fair warning, cython is not as user friendly as pyinstaller appears to be. I have yet to use it successfully.
I am developing a plugin for a multi-platform Python program (Deluge). My plugin needs to import some modules which aren't available by default (in my case, the requests module).
On Linux, everything works flawlessly assuming the required modules are installed beforehand (e.g. via pip).
On Windows, the program makes use of python27.dll which comes as part of the installer, so importing modules - even those available on the local Python installation (verified via interpreter) - yields an import error.
I've seen the answers to this question, but I'd like to know if there is a proper way of adding module search paths for Python on Windows specifically. Is it safe to assume C:\Python27\Lib\site-packages will point me to the local Python installation's modules?
EDIT: Is there a different method I could incorporate for using "external" modules? Could I perhaps package other modules into my final .egg file? Not just plain Python, but more sophisticated modules like requests which need to be properly built and may even rely on other modules.
Currently, I'm deploying a full python distribution (the original python 2.7 msi) with my app. Which is an embedded web server made with delphi.
Reading this, I wonder if is possible to embed the necessary python files with my app, to decrease load files and avoid conflict with several python versions.
I have previous experience with python for delphi so I only need to know if only shipping the python dll + zip with the distro + own scripts will work (and if exist any caveats I must know or a sample where I can look)
zipimport should work just fine for you -- I'm not familiar with Python for Delphi, but I doubt it disables that functionality (an embedding application can do that, but it's an unusual choice). Just remember that what you can zip up and import directly are the Python-coded modules (or just their corresponding .pyc or .pyo byte codes) -- DLLs (even if renamed as .pyds;-) need to be on disk to be loaded (so if you have a zipfile with them it will need to be unzipped at the start of the app, e.g. into a temporary directory).
Moreover, you don't even need to zip up all modules, just those you actually need (by transitive closure) -- and you can easily find out exactly which modules those are, with the modulefinder module of the standard Python library. The example on the documentation page I just pointed to should clarify things. Happy zipping!
Yes it is possible.
I'm actually writing automatisation script in Python with the Zipimport library. I actually included every .py files in my zip as well as configuration or xml files needed by those script.
Then, I call a .command file targeting a __main__.py class that redirect towards the desired script according to my sys.argv parameters which is really useful!
I am distributing an app that uses the Python/C API. I have all standard python modules in python31.zip which is basically an archive of the Lib folder in the python install directory. Here is the problem - most common modules like sys and io work fine. BUT tkinter does not. I get an error "cannot find module _tkinter". I really need tkinter in my project. I'm using Windows if that helps.
I don't the best way to bundle tkinter with your app, but I do know why you're getting the error you are. The relevant section of the zipimport module documentation:
Any files may be present in the ZIP archive, but only files .py and .py[co] are available for import. ZIP import of dynamic modules (.pyd, .so) is disallowed. Note that if an archive only contains .py files, Python will not attempt to modify the archive by adding the corresponding .pyc or .pyo file, meaning that if a ZIP archive doesn’t contain .pyc files, importing may be rather slow.
The module _tkinter is a c-extension / shared library. It can't be imported from a zip file.
why don't you use py2exe to bundle your application as an executable? It should take care of all the dependencies, and will include whatever you need.