Develop Reference

How do I know which .dll is missing in python with cx_freeze?

I have a project which I am packaging with cx_freeze. When I try to run the resulting exe I get an ImportError: DLL load failed: The specified module could not be found.
Here is my setup.py:
from cx_Freeze import setup, Executable
# Dependencies are automatically detected, but it might need
# fine tuning.
buildOptions = dict(build_exe='liquidation', packages=['scipy', 'numpy'], excludes=['scipy.spatial.cKDTree'],
includes=['scipy', 'numpy', 'numpy.core._methods', 'scipy.sparse.csgraph._validation',
'numpy.lib.format', 'numpy.linalg._umath_linalg', 'scipy.sparse._csparsetools',
'scipy.sparse.linalg.isolve._iterative', 'scipy.sparse.linalg.eigen.arpack._arpack',
'scipy.special._ufuncs_cxx', 'scipy.special.specfun', 'scipy.integrate._odepack',
'scipy.integrate._quadpack', 'scipy.integrate.vode', 'scipy.integrate._dop',
'scipy.integrate.lsoda', 'scipy.optimize._minpack', 'scipy.optimize._zeros',
'scipy.spatial', 'scipy.spatial.ckdtree', 'scipy.spatial.kdtree',
'scipy._distributor_init', 'numpy.core._multiarray_umath'])
base = 'Console'
executables = [
Executable('liquidation.py', base=base, targetName='liquidation.exe')
]
setup(name='liquidation',
version='2.0',
description='Program to run and test Liquidation algorithms',
options=dict(build_exe=buildOptions),
executables=executables)
When I run python setup.py build_exe it seems to complete successfully, but when I run the exe I get the following output:
Traceback (most recent call last):
File "C:\Users\tamar\PycharmProjects\liquidation\venv\lib\site-packages\numpy\core\__init__.py", line 24, in <module>
from . import multiarray
File "C:\Users\tamar\PycharmProjects\liquidation\venv\lib\site-packages\numpy\core\multiarray.py", line 14, in <module>
from . import overrides
File "C:\Users\tamar\PycharmProjects\liquidation\venv\lib\site-packages\numpy\core\overrides.py", line 7, in <module>
from numpy.core._multiarray_umath import (
ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\tamar\PycharmProjects\liquidation\venv\lib\site-packages\cx_Freeze\initscripts\__startup__.py", line 40, in run
module.run()
File "C:\Users\tamar\PycharmProjects\liquidation\venv\lib\site-packages\cx_Freeze\initscripts\Console.py", line 37, in run
exec(code, {'__name__': '__main__'})
File "liquidation.py", line 64, in <module>
from io_tools.stock_wrapper import StockWrapper
File "C:\Users\tamar\PycharmProjects\liquidation\io_tools\stock_wrapper.py", line 4, in <module>
from io_tools.io_helper import *
File "C:\Users\tamar\PycharmProjects\liquidation\io_tools\io_helper.py", line 7, in <module>
from numpy import float_
File "C:\Users\tamar\PycharmProjects\liquidation\venv\lib\site-packages\numpy\__init__.py", line 142, in <module>
from . import core
File "C:\Users\tamar\PycharmProjects\liquidation\venv\lib\site-packages\numpy\core\__init__.py", line 54, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
1. Check that you expected to use Python3.7 from "C:\Users\tamar\PycharmProjects\liquidation\liquidation\liquidation.exe",
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy version "1.18.3" you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: DLL load failed: The specified module could not be found.
I understand that I need to point cx_freeze to the missing DLL's (either by adding them to buildOptions.include_files or by actually copying the DLL's to the build folder) but I am at a loss as to how to know which DLL's I am missing. numpy.core._multiarray_umath is in the build folder so I don't know what it could be.
I believe I am using the latest versions of numpy (1.18.3), scipy (1.4.1) and cx-freeze (6.1). Any help would be much appreciated!
Update: I used Dependency Walker to try and see what I am missing and it listed a file by the name of LIBOPENBLAS.SVHFG5YE3RK3Z27NVFUDAPL2O3W6IMXW.GFORTRAN-WIN32.DLL which cannot be found. All of the other dependencies have been copied into the build so it makes sense that this would be the missing one. Does anyone know where I can find this file?

cx_freeze: ModuleNotFoundError and program stopped working

I have a console mode program that I would like to distribute with cx_Freeze. I have just downloaded python 3.6.3 (both amd64 and win32 version) on a newly installed win 10 machine. I have installed cx_Freeze 5.1 from wheels (not from pypi). Then I have built both amd64 and win32 versions of the program. If I try to execute that exe file on the same machine where I have built it, I get this error:
Fatal Python error: Py_Initialize: unable to load the file system codec
Traceback (most recent call last):
File "C:\Program Files\Python36\lib\encodings\__init__.py", line 31, in <module>
ModuleNotFoundError: No module named 'codecs'
I have the same error on both architectures. The setup file looks like this:
#!/usr/bin/env python3
from cx_Freeze import setup, Executable
# Dependencies are automatically detected, but it might need fine tuning.
build_exe_options = {
"packages": [
"os", "io", "copy", "struct", "hashlib", "random",
"urllib", "pycurl", "json", "multiprocessing",
"cryptography",
"tornado", "watchdog", "pathtools"
],
"includes": [
"cryptography", "urllib.parse",
],
"include_files": ["tmp", "server.ini", "client.ini"],
"excludes": ["tkinter"],
}
setup(
name="BlindBackup",
version="1.0",
description="BlindBackup",
options={"build_exe": build_exe_options},
executables=[
Executable("backup.py", base=None),
Executable("bsync.py", base=None),
Executable("server.py", base=None),
]
)
In other words, I have specified the "codecs" package explicitly.
What is wrong?
UPDATE There is a file called library.zip in the distribution dir. It contains codecs.pyc but the exe file does not see it. If I extract all files from that zip, then I get a different error message:
C:\Python\Projects\blindbackup\build\exe.win-amd64-3.6>backup.exe
Traceback (most recent call last):
File "C:\Program Files\Python36\lib\site-packages\cx_Freeze\initscripts\__startup__.py", line 14, in run
module.run()
File "C:\Program Files\Python36\lib\site-packages\cx_Freeze\initscripts\Console.py", line 22, in run
importer = zipimport.zipimporter(os.path.dirname(os.__file__))
zipimport.ZipImportError: not a Zip file
I guess that the problem is with the code that tries to import modules from the library.zip file? But not sure how to fix it.

The solution was this:
install cx_freeze 6, because version 5 does not support python 3.6 (apparently)
install pywin32 (although it was not required for my program)
add all missing pyd files to "includes" and all missing packages to "packages" section it the setup script.

How to package psutil with py2exe?

The app is working fine on my development win8 environment, but when it is packaged with py2exe and run on the production machine it throw the exception:
"The procedure entry point RtlIdnToAscii could not be located in the dynamic link library ntdll.dll"
The detail content of the log file is
Traceback (most recent call last):
File "DataviewerBackupRestorer.py", line 6, in <module>
File "RestorController.pyc", line 7, in <module>
File "psutil\__init__.pyc", line 136, in <module>
File "psutil\_psmswindows.pyc", line 14, in <module>
File "_psutil_mswindows.pyc", line 12, in <module>
File "_psutil_mswindows.pyc", line 10, in __load
ImportError: DLL load failed: The specified procedure could not be found.
It seems that a dll required by psutil is missing during the package process. I have tried to add the py2exe options with
py2exe_options = {"py2exe":{"includes":['decimal', '_mssql', 'uuid', 'psutil', '_psutil_mswindows']}}
But it is not working. Any ideas? Thanks in advance!

The solution is to remove System DLLs from the project directory. When I added psutil to my application py2exe added a lot of system DLLs to my project. It worked correctly on my and some other computers, but failed on another one. Removing from the project the .dll files that were available in C:\Windows\System32 solved the issue.
Finally in my case the solution was to add:
'dll_excludes': [ "IPHLPAPI.DLL", "NSI.dll", "WINNSI.DLL", "WTSAPI32.dll"]
into the py2exe options in setup.py file.

entry_points path changed in development mode install

My console_scripts/entry_points are not working in "develop" mode install and produce a traceback saying ImportError cannot load the module containing the entry points. Need help understanding how entry_points figure out which module to load and what paths to setup and stuff. Help fixing my error.
I have a setup.py script that has some entry points as follows
entry_points = {'console_scripts':[
'pyjampiler=pyjs.pyjampiler:Builder',
'pyjscompile=pyjs.translator:main',
'pyjsbuild=pyjs.browser:build_script',
]}
my code is organized as
workspace/
setup.py
pyjs/
src/
pyjs/
browser.py
My setup.py makes use of packages and package_dir arguments to setup() in setup.py to make sure the pyjs package gets picked up from pyjs/src/pyjs and so a regular install produces the following console scripts containing the following and it runs fine. it is able to load the module and call the entry point fine.
sys.exit(
load_entry_point('pyjs==0.8.1', 'console_scripts', 'pyjsbuild')()
)
But when I install and run it in development as "python setup.py develop", the install goes fine and I see the egg.lnk files getting created. but executing the console script causes the following errors
localhost:pyjs sarvi$ lpython/bin/pyjsbuild
Traceback (most recent call last):
File "lpython/bin/pyjsbuild", line 8, in <module>
load_entry_point('pyjs==0.8.1', 'console_scripts', 'pyjsbuild')()
File "/Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 318, in load_entry_point
File "/Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 2221, in load_entry_point
File "/Users/sarvi/Workspace/pyjs/lpython/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 1954, in load
ImportError: No module named browser
I suspect it has something to do with sys.path and the directory structure in the development sources. How can I get this to work in "develop" mode install?

Create standalone MoinMoin wiki executable

I'm trying to create a standalone, desktop version of a MoinMoin wiki so I can distribute it on a CDROM to people who may or may not have Python installed. I've tried both py2exe and bbfreeze with no luck. They both create an executable, but when that executable is run I get the same error from both:
C:\python_class\cdrom\bb-binary>wikiserver.exe
2011-08-22 15:06:21,312 WARNING MoinMoin.log:138 load_config for "C:\python_class\cdrom\bb-binary\wikiserverlogging.conf
" failed with "No section: 'formatters'".
2011-08-22 15:06:21,312 WARNING MoinMoin.log:139 using logging configuration read from built-in fallback in MoinMoin.log
module!
Traceback (most recent call last):
File "<string>", line 6, in <module>
File "__main__.py", line 128, in <module>
File "__main__wikiserver__.py", line 35, in <module>
File "MoinMoin/script/__init__.py", line 138, in run
File "MoinMoin/script/__init__.py", line 248, in mainloop
File "MoinMoin/wikiutil.py", line 1078, in importBuiltinPlugin
File "MoinMoin/wikiutil.py", line 1117, in builtinPlugins
File "MoinMoin/util/pysupport.py", line 81, in importName
ImportError: No module named server
Here is the setup.py script I used for py2exe:
from distutils.core import setup
import py2exe
includes = ["MoinMoin"]
excludes = []
packages = []
setup(options = {
"py2exe" : {
"includes" : includes,
"excludes" : excludes,
"packages" : packages,
"dist_dir" : "dist"
}
},
console=["wikiserver.py"])
And here is the setup.py script I used for bbfreeze:
from bbfreeze import Freezer
includes = ["MoinMoin.*"]
excludes = []
f = Freezer(distdir="bb-binary", includes=includes, excludes=excludes)
f.addScript("wikiserver.py")
f.use_compression = 0
f.include_py = True
f()
If anyone has any help or suggestions, I'd greatly appreciate it!
Thanks,
Doug

py2exe has limitations in identifying which modules to include, especially if they are imported conditionally. For example,
import module
on its own line will work, however,
if someCondition:
import module
won't. As is often the case with many large frameworks, MoinMoin only imports the modules it needs to use when it needs them. Unfortunately, you will need to tell py2exe to include these missing modules manually, and this is going to take some trial-and-error until you find all the ones you need.
See here for how to include modules manually.