I use ctypes on Windows to access a DLL. Generally, I wrap the DLL with a .py of the same name and add additional support for the DLL such as ctypes structs that are used with some of the functions. I load the DLL use the following technique.
# Determine the absolute path to the DLL that resides in the same directory as this module.
absolute_path = os.path.join(os.path.dirname(__file__), 'monty.dll')
interface = ctypes.CDLL(absolute_path)
The module is part of a package that I maintain and I'd like to use Py2exe with bundle_files set to 1 to create fewer files. I am happy to have library.zip sitting alongside my .exe.
So the obvious problem is that monty.py will be trying to access monty.dll and it will expect that path to be .../library.zip/root/dll/monty.dll.
I found a way to package the DLL in library.zip but I suspect ctypes is rejecting the path within the .zip file. Windows complains that it cannot load the module.
Getting the above to work would be my first choice but a fallback might be to have everything in library.zip except my package. Any ideas as to how this can be accomplished?
Related
Is it possible to change the assembly name of a DLL after it has been compiled and linked? If so what tool should I use to do this? I've failed in my searches for the answer, likely because I don't know enough about the subject.
I have been provided DLLs that I am linking to a Python extension on Windows generated with numpy.distutils and F2PY wrappers. I am using the MinGW-w64 tools and gfortran as the compiler. The DLLs were also compiled with gfortran. The extension compiles and links just fine with, however at runtime I get the error ImportError: DLL load failed: The specified module could not be found. Using dependency walker I was able to discover that the library name is in fact different from the file name. I can change the file name to match the library name dependency walker reports, but name contains metadata the individual used ot track their projects, e.g. 'mylib-v4-fix'. I would like to modify the library name to be consistent with the file name of the DLL, e.g 'mylib.dll'. I was not provided .lib or .def files, but presumably I could create these if necessary.
I am using PyInstaller to create a single-file executable. Is it possible for my script to perform an import such that i) the imported module is imported from the same directory as the exe (i.e. it's not packaged into the exe) and ii) that imported module can import other modules that were packaged into the exe?
The background here is that the imported module contains configuration that the user should be able to modify. This may include creation of custom derived classes and use of enums from the packaged modules.
I haven't found any advice on this, though it's a difficult search because there are so many similar topics that use basically the same keywords.
The following steps allow a Python module (named external_module here) outside of an executable created by PyInstaller to be imported and for that module to import modules that were bundled into the executable.
Add excludes=['external_module'] to the Analysis object used in the PyInstaller spec. This prevents external_module.py being bundled into the executable.
Add sys.path.append(os.path.dirname(sys.executable)) where external_module is imported in your application. This allows it to be imported from the directory the executable is in, which is different to the directory that the application will run in (due to being decompressed to a temporary folder). See below for my recommended method of achieving this.
Make sure any imports performed by external_module.py are also performed by one of the bundled modules before external_module.py is imported. The interpreter will not resolve the external module's imports against bundled modules, but will use ones that already exist in sys.modules.
In order to set up the paths correctly you can use the following:
if getattr(sys, 'frozen', False):
app_path = os.path.dirname(sys.executable)
sys.path.append(app_path)
else:
app_path = os.path.dirname(os.path.abspath(__file__))
frozen is only available in generated executables, not when running as a script directly. This snippet will add the executable's location to sys.path if required as well as giving you easy access to the executable or script's location for use in code.
As an example of the final bullet point, consider the following.
# bundled_module1.py
import external_module
# bundled_module2.py
# module content
# external_module.py
import bundled_module2
This will fail in external_module.py because bundled_module2 can't be found. However, the following modification to bundled_module1.py will work:
# bundled_module1.py
import bundled_module2
import external_module
This will be fine if there are a limited set of bundled modules that the external one should be able to import. It may get unwieldy for larger sets.
Given that the documentation states that the interpreter will resolve imports against modules bundled into the executable, this feels like a possible bug. Interoperating with modules outside of the executable isn't explicitly called out though.
Type in Pyinstaller -h. It will give you info about pyinstaller and tell you about --runtime-hook. I presume adding this to the executable should work. There's actually a whole page of documentation for this. Surprised you could not find that.
Anyway,
The docs say put in:
pyinstaller --additional-hooks-dir=. myscript.py.
I presume then something like pyinstaller --additional-hooks-dir=C:\pathtofolder myscript.py should work in theory. Yet to test it. Tell us how it goes and what made kinks made it work for you.
Lastly, if you want to be hipster try integrating cython for speed and obfuscation. Fair warning, cython is not as user friendly as pyinstaller appears to be. I have yet to use it successfully.
How can I get my own dll to use a dll in a specific location?
I have build an 32bit Excel dll (a .xll). I am using the Python C-Api in the dll. Problem is in general there is only a 64bit Python installation. The 64 python26.dll is located \System32 .
If I put the 32bit python26.dll in the Excel exe Folder or in \SysWOW64 everything is fine, but usually I have no access to those folders. Placing the 32 bit python dll in the xll folders does not help, it seems \System32 comes first.
I there any solution without linking explicitly using "LoadLibrary" and many times "GetProcAddress".?
Any option in VS2010?
Many thanks for your help.
You are right, the search order has the system directories first (and for a reason). This is a traditional situation with plugins. The applications themselves can store their dependent libraries in their own program folder and be fine, because that has the highest priority. Plugins commonly don't have that luxury.
The solution might be to use delayed loading for the DLL. If you have a program like this:
int main()
{
Py_Initialize();
}
...and you link it with python26.lib, it will create a static entry in the import table, pointing to python26.dll!Py_Initialize. Because the import table is processed before your code runs, you can't say which python26.dll you want to use, where to find it etc.
But if you set linker flag /DELAYLOAD:python26.dll and add MSVC's delayimp.lib to the build, it will postpone loading the Python DLL until some function from it is called. So now you can make sure you load the correct one: the import table has just the base name of the dependency (python26.dll) and if some library with the same name is already loaded, it will just reuse it. So you can preload the one you want from any location of your choice, then let the delayed loader do its job, like so:
int main()
{
LoadLibrary(TEXT("C:\\Program Files (x86)\\...\\...\\python26.dll"));
Py_Initialize();
}
But even with that, you might face another problem. The fact that the first loaded module with the correct base name is used by default means that even if you managed to do everything else right, there could still be some other Excel plugin or some silly Shell extension loaded (e.g. when the user goes to the "Open file" dialog), which would load their own python26.dll with possibly incompatible API, built with another compiler or anything else, which could ruin your day.
So I would also recommend you bundle your own custom version renamed to something like productname-python26.dll and link to that instead. There surely would never be a similarly named DLL by any other product, so you would be safe. In order to create a .lib file from a renamed DLL, try e.g. this guide.
My python program uses plugins (python files) which I import dynamically using __import__.
I bundle my python program into a Windows exe using py2exe.
I've just spend searching 2 hours why my plugin python file couldn't be loaded properly from the .exe version. I got an ImportError: "no module named urllib2"
It appeared my plugin was using urllib2 (through an import urllib2 statement), and that standard library module was apparently not bundled into the exe. Other modules used in the plugin (re, urllib, ...) gave no problem, but perhaps they were already references in python files I statically include in my program.
How can I know which standard Python library modules py2exe bundles by default in the exe? (so I know whether I or somebody else can use them in a plugins). The py2exe documentation doesn't give an hints, except for saying that it includes a lot of modules from the standard library.
To see which modules are included look inside the library.zip (if there is no library.zip file - then try opening the EXE in any ZIP application - or rename it to .ZIP and try and open it).
You will be able to see a list of *.pyc. You can look at the list of files and directories to get an impression of which modules are included or not.
If you require a specific package to be added - add it to the 'packages' list.
As to why it doesn't include everything or how it chooses to include some and not others? My understanding is that py2exe looks in your code to figure out what you are using and includes those (and some that it probably needs itself) but maybe it also has some heuristics to add other modules too (I haven't checked :)
I am distributing an app that uses the Python/C API. I have all standard python modules in python31.zip which is basically an archive of the Lib folder in the python install directory. Here is the problem - most common modules like sys and io work fine. BUT tkinter does not. I get an error "cannot find module _tkinter". I really need tkinter in my project. I'm using Windows if that helps.
I don't the best way to bundle tkinter with your app, but I do know why you're getting the error you are. The relevant section of the zipimport module documentation:
Any files may be present in the ZIP archive, but only files .py and .py[co] are available for import. ZIP import of dynamic modules (.pyd, .so) is disallowed. Note that if an archive only contains .py files, Python will not attempt to modify the archive by adding the corresponding .pyc or .pyo file, meaning that if a ZIP archive doesn’t contain .pyc files, importing may be rather slow.
The module _tkinter is a c-extension / shared library. It can't be imported from a zip file.
why don't you use py2exe to bundle your application as an executable? It should take care of all the dependencies, and will include whatever you need.