Develop Reference

Create an executable from Python code with gmsh import

I'm trying to package up my Python package into an executable using pyinstaller. The script name is called "run-jointbuilder.py" The package has a number of dependancies (such as numpy), but importantly gmsh.
When using pyinstaller to compile my code, it appears to be successful, but then when I try to run the executable I get the following errors:
import gmsh # PyInstaller PYZ\
import ctypes.util # PyInstaller PYZ\
import 'ctypes.util' # <pyimod03_importers.FrozenImporter object at 0x000001BD783FC910>\
Traceback (most recent call last):\
File "PyInstaller\loader\pyiboot01_bootstrap.py", line 144, in __init__
File "ctypes\__init__.py", line 381, in __init__\
FileNotFoundError: Could not find module 'C:\Users\willber\Anaconda3\Scripts\gmsh' (or one of its dependencies). Try using the full path with constructor syntax.
I then get this error:
__main__.PyInstallerImportError: Failed to load dynlib/dll
'C:\\Users\\willber\\Anaconda3\\Scripts\\gmsh'. Most probably this dynlib/dll was not found when the application was frozen.
[18612] Failed to execute script run-jointbuilder
Has anybody tried to compile some Python code that imports the gmsh package? I'd really appreciate an example .spec file, for use with pyinstaller if so!

The gmsh python package wraps a bunch of compiled libraries which contain the implementations of the methods you call from python. When you import gmsh.py into your script then gmsh loads these libraries in the background giving you access to their functionality through python methods. So it's essential that these libraries are embedded in the pyinstaller output for your code to function as it does when you run it directly through the python interpreter.
It's difficult for pyinstaller to consistently find these libraries since they're not accessed in the same way as normal python imports, they are loaded using the cytpes package. There's some description of how pyinstaller does this in the docs. Since you're seeing a dynlib/dll loading error when you run the compiled python script, this suggests that pyinstaller isn't finding the gmsh library during compile and hence it is missing from the executable.
If you look in the gmsh.py source, you can see that gmsh.py loads up a .dll library called gmsh-4.9.dll for Windows OS. You can use the binaries input of the pyinstaller .spec file to point the compiler to the gmsh-4.9.dll.
Here's an example .spec file which dynamically locates the gmsh-4.9.dll at the time of compile so it picks up the right .dll for your active environment. You could make this more generic by filtering all *.dll in the gmsh directory, but I've hardcoded for clarity:
# -*- mode: python ; coding: utf-8 -*-
from pathlib import Path
import gmsh
# get the location of the gmsh dll which sits next to the gmsh.py file
libname = 'gmsh-4.9.dll'
libpath = Path(gmsh.__file__).parent / libname
print('Adding {} to binaries'.format(libpath))
block_cipher = None
a = Analysis(['gmsh-test.py'],
pathex=['C:\\Users\\user\\dev\\gmsh'],
# tell pyinstaller to add the binary to the compiled path
binaries=[(str(libpath), '.')],
datas=[],
hiddenimports=[],
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='gmsh-test',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
console=True,
runtime_tmpdir=None,
)

How do I resolve a missing utility module preventing my executable from running when I already installed the parent module?

I've written a few Python scripts to create a tkinter GUI for a machine learning algorithm process. I originally coded everything in PyCharm, but I'd really like to put everything together into a stand-alone executable. I've moved my main script and its .py dependencies into their own directory and tested it out using the Command Prompt, and it works great. However, when I run pyinstaller, the executable is created but fails on startup.
The program is made up of three files, with GUI.py being the main script. As mentioned above, I moved the dependent files into a new directory and tested GUI.py in the Command Prompt, and it worked great. Executable is created (albeit with a lot of warnings about missing 'api-ms-win-crt' files) but can't be run.
I created the executable using the command:
pyinstaller --onefile GUI.py
When the executable is run from the command line after creation, I get a big long traceback ending in the following:
File "site-packages\sklearn\metrics\pairwise.py", line 32, in <module>
File "sklearn\metrics\pairwise_fast.pyx", line 1, in init
sklearn.metrics.pairwise_fast
ModuleNotFoundError: No module named 'sklearn.utils._cython_blas'
[3372] Failed to execute script GUI
I know I've already explicitly imported sklearn through the command prompt, but from the traceback, it seems I'm missing a utility module somewhere. I tried to import the missing module specifically, but I got an error that no distributed module was available.
I don't have much experience with pyinstaller, and I have no idea where to go from here. I'm using Windows 10 and Python 3.7.3.

It seems that Pyinstaller can't resolve sklearn import. So one easy way is to just bring the whole module directory which located in <path_to_python>/Lib/site-packages/sklearn/ with executable output. So use below spec file to generate your executable:
# -*- mode: python -*-
block_cipher = None
a = Analysis(['test.py'],
pathex=['<path to root of your project>'],
binaries=[],
datas=[],
hiddenimports=[],
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False)
a.datas += Tree('<path_to_sklearn_in_python_dir>', prefix='sklearn')
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
[],
name='test',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=False,
runtime_tmpdir=None,
console=True )
Finally generate your executable with
pyinstaller test.spec
This should resolve import errors for sklearn but if you face other NotFound imports add them like above to spec file.

Building up on M.R.'s answer, you can directly include the path to sklearn in your original pyinstaller command:
pyinstaller --onefile GUI.py --add-data "<path-to-python>\Lib\site-packages\sklearn;sklearn"
which results in the following line of code being added inside a = Analysis() in the automatically-generated GUI.spec file:
datas=[('<path-to-python>\\Lib\\site-packages\\sklearn', 'sklearn')]
Note that the --onefile option will result in an executable that is slower to start up than the default one-folder bundle (based on both the pyinstaller documentation and my own experience bundling up sklearn):
pyinstaller GUI.py --add-data "<path-to-python>\Lib\site-packages\sklearn;sklearn"

PyPandoc in combination with PyInstaller

I installed PyInstaller to create executables for my python scripts, and that works fine. I used PyPandoc to create .docx reports, which also run fine when the normal python files are run, but not from the PyInstaller generated executable. It gives the error:
Traceback (most recent call last):
File "src\flexmodel_postcalc.py", line 295, in postcalculate_everything
File "src\flexmodel_postcalc.py", line 281, in generate_report_docx
File "src\flexmodel_report_docx.py", line 118, in generate_text_useages_docx
File "pypandoc\__init__.py", line 50, in convert
File "pypandoc\__init__.py", line 70, in _convert
File "pypandoc\__init__.py", line 197, in get_pandoc_formats
File "pypandoc\__init__.py", line 336, in _ensure_pandoc_path
OSError: No pandoc was found: either install pandoc and add it
to your PATH or install pypandoc wheels with included pandoc.
During the executable creation I see no strange issues about PyPandoc. How can I include Pandoc into my executable so others (without Python and/or Pandoc installation) can use the executable and create .docx reports?
edit: a working process included the following steps:
Create a file including the following code:
import pypandoc
pypandoc.convert(sou‌​rce='# Sample title\nPlaceholder', to='docx', format='md', outputfile='test.doc‌​x')
Save this file as pythonfile.py
create an executable with PyInstaller:
pyinstaller --onefile --clean pythonfile.py
Now the executable should run on a computer without Pandoc (or PyPandoc) installed.

There are two issues here. The first one is that pypandoc needs pandoc.exe to work. This is not picked up by pyinstaller automatically, but you can specify it manually.
To do this you you have to create a .spec file. The one I generated and used looks like this:
block_cipher = None
a = Analysis(['pythonfile.py'],
pathex=['CodeDIR'],
binaries=[],
datas=[],
hiddenimports=[],
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='EXEName',
debug=False,
strip=False,
upx=True,
console=True ,
resources=['YourPandocLocationHere\\\\pandoc.exe'])
You can build the executable by using pyinstaller myspec.spec. Don't forget to change the paths and the name parameter.
If you were building that in a directory mode this should be enough. However, for the one-file mode, things are a bit more complicated due to the way the pyinstaller bootloader process works. The pandoc.exe file is unzipped during execution in a temporary folder, but the execution happens in your original .exe folder. According to this question, you have to add the following lines to your code before calling pypandoc to change your current folder, if you run the frozen code.
if hasattr(sys, '_MEIPASS'):
os.chdir(sys._MEIPASS)

pyinstaller doesn't bundle pyd and dll files with --onefile

I'm trying to bundle a cefpython1 application into a single exe with pyinstaller.
I have a working spec file which creates a distrubiton for the cefpython1 example, cefsimple:
# -*- mode: python -*-
def get_cefpython_path():
import cefpython1 as cefpython
path = os.path.dirname(cefpython.__file__)
return "%s%s" % (path, os.sep)
cefp = get_cefpython_path()
a = Analysis(['cefsimple.py'],
hiddenimports=["json.decoder", "json.scanner", "json.encoder"])
pyz = PYZ(a.pure)
exe = EXE(pyz,
a.scripts,
exclude_binaries=True,
name='cefsimple.exe',
debug=False,
strip=None,
upx=False,
console=False )
coll = COLLECT(exe,
a.binaries + [('icudt.dll', '%s/icudt.dll' % cefp, 'BINARY')],
a.zipfiles,
a.datas + [('locales/en-US.pak', '%s/locales/en-US.pak' % cefp, 'DATA'), ('cefsimple.html', 'cefsimple.html', 'DATA'), ('icon.ico', 'icon.ico', 'DATA')],
strip=None,
upx=False,
name='cefsimple')
The project files can be found on my Google Drive. Don't care about setup.py, it contains a py2exe build which i was playing with next to pyinstaller. You need Python 2.7, Win32gui, cefpython1 and of course pyinstaller packages to run this, I've tested this with the Win32 version only! I've even tried installing the development pyinstaller if it makes any change.
If i try to execute pyinstaller with the --onefile attribute seems like nothing changes, pyinstaller just creates the distribution directory under dist. The command I'm using is: pyinstaller --onefile cefsimple.spec
Tested --onefile with a simple Hello World python file and it does work by that. What's causing pyinstaller to not creating a single exe? The build log doesn't show anything interesting, but there are some things i don't understand in the warning file. For eg. it says there's no module named cefpython1.cefpython but the correct pyd is copied to the dist directory and the application is working anyway.
Here is the list of files created under dist/: cefsimple.lst Maybe this helps finding the problem.

The command I'm using is: pyinstaller --onefile cefsimple.spec
Tested --onefile with a simple Hello World python file and it does
work by that. What's causing pyinstaller to not creating a single exe?
The option --onefile is ignored when you type pyinstaller --onefile cefsimple.spec because the .spec already defines if you will get a directory or a standalone file. A .spec file with a COLLECT function will make a whole dist directory.
I would suggest remaking a new .spec file by typing pyi-makespec --onefile cefsimple.py and adding back your various modifications (data, binaries, hiddenimports...), then trying pyinstaller cefsimple.spec without option. That works for me with pyinstaller 3.3.1.

How to compile python script to binary executable

I need to convert a Python script to a Windows executable.
I have Python 2.6 installed to python26.
I have created one script and kept it in C:\pythonscript. Inside this folder there are two files
Setup.py and oldlogs.py (this file need coversion)
setup.py code is
from distutils.core import setup
import py2exe
setup(console=['oldlogs.py'])
How can I convert oldlogs.py to an exe file?

Or use PyInstaller as an alternative to py2exe. Here is a good starting point. PyInstaller also lets you create executables for linux and mac...
Here is how one could fairly easily use PyInstaller to solve the issue at hand:
pyinstaller oldlogs.py
From the tool's documentation:
PyInstaller analyzes myscript.py and:
Writes myscript.spec in the same folder as the script.
Creates a folder build in the same folder as the script if it does not exist.
Writes some log files and working files in the build folder.
Creates a folder dist in the same folder as the script if it does not exist.
Writes the myscript executable folder in the dist folder.
In the dist folder you find the bundled app you distribute to your users.

I recommend PyInstaller, a simple python script can be converted to an exe with the following commands:
utils/Makespec.py [--onefile] oldlogs.py
which creates a yourprogram.spec file which is a configuration for building the final exe. Next command builds the exe from the configuration file:
utils/Build.py oldlogs.spec
More can be found here

Since other SO answers link to this question it's worth noting that there is another option now in PyOxidizer.
It's a rust utility which works in some of the same ways as pyinstaller, however has some additional features detailed here, to summarize the key ones:
Single binary of all packages by default with the ability to do a zero-copy load of modules into memory, vs pyinstaller extracting them to a temporary directory when using onefile mode
Ability to produce a static linked binary
(One other advantage of pyoxidizer is that it does not seem to suffer from the GLIBC_X.XX not found problem that can crop up with pyinstaller if you've created your binary on a system that has a glibc version newer than the target system).
Overall pyinstaller is much simpler to use than PyOxidizer, which often requires some complexity in the configuration file, and it's less Pythony since it's written in Rust and uses a configuration file format not very familiar in the Python world, but PyOxidizer does some more advanced stuff, especially if you are looking to produce single binaries (which is not pyinstaller's default).

# -*- mode: python -*-
block_cipher = None
a = Analysis(['SCRIPT.py'],
pathex=[
'folder path',
'C:\\Windows\\WinSxS\\x86_microsoft-windows-m..namespace-downlevel_31bf3856ad364e35_10.0.17134.1_none_50c6cb8431e7428f',
'C:\\Windows\\WinSxS\\x86_microsoft-windows-m..namespace-downlevel_31bf3856ad364e35_10.0.17134.1_none_c4f50889467f081d'
],
binaries=[(''C:\\Users\\chromedriver.exe'')],
datas=[],
hiddenimports=[],
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
name='NAME OF YOUR EXE',
debug=False,
strip=False,
upx=True,
runtime_tmpdir=None,
console=True )