Develop Reference

Minimal set of files required to distribute an embed-Cython-compiled code and make it work on any machine

TL;DR: how to use Cython as a distribution method instead of Py2exe, cx_freeze, pyinstaller, etc.
Following Making an executable in Cython, I'd like to see how it could be possible to distribute a Python program to any Windows user (who doesn't have Python already installed on his machine) by compiling it first with Cython --embed.
Let's use a test.py:
import json
print(json.dumps({'key': 'hello world'}))
and compile it:
cython test.py --embed
call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" x64
cl test.c /I C:\Python37\include /link C:\Python37\libs\python37.lib
It works and produces a 140KB test.exe executable.
Running test.exe on another machine doesn't work out-of-the-box, it requires:
python37.dll in the same folder
to install the usual vc_redist.x64.exe file
Even with this, it still does not work (screenshot below instead of copy/paste because I didn't manage the copy/paste in the VM - off topic here):
ModuleNotFoundError: No module named 'encodings'
Question: what is the minimal set of files required to distribute an --embed-Cython-compiled code and make it work on any machine (without Python previously installed on it)?

After further research (I tried in an empty Win 7 x64 bit VM, without any VCredist previously installed), it seems that these files are enough:
the program itself, test.exe (produced by cython --embed and compilation with cl.exe)
python37.dll
python37.zip coming from packages named "Windows x86-64 embeddable zip file" in https://www.python.org/downloads/windows/
vcruntime140.dll, as mentioned in Can I bundle the Visual Studio 2015 C++ Redistributable DLL's with my application? or ask the user to install vc_redist.x64.exe before
ucrtbase.dll
more than 30 files api-ms-win-*.dll were required too; if not you will have the following error:
... api-ms-win-crt-runtime-l1-1-0.dll is missing ...
Notes:
if you require another library, like pygame, just copy/paste the folder from C:\Python37\Lib\site-packages\pygame seems to work
for me, concrt140.dll, msvcp140.dll, vccorlib140.dll did not seem necessary
Useful to test all this: Prevent a Python-embedded to look in my default path C:\Python38 for modules.

How to build PJSUA2 (swig) with Visual Studio 2015

I try to compile PJSUA2 for Python. I could manage to compile the source with Visual Studio 2015 Community edition.
My question is, how can I install the Python module now?
The guide suggests to use make and make install. I tried to install also minwg, but it is not working as the project was compiled with VS.
I have also tried to compile with minwg, but I never could succeed due to undefined requirements. Also official PJSUA guide recommend to use VS for Windows.
Following settings have been applied to build from source:
Swig location has been added to path
JAVA_home system variable has been added
Java location has been added to path
Empty pjlib/include/pj/config_site.h has been created
Opened source in VS 2015 Community (all suggested VS package and module has been installed)
Set following project to do not compile:
pjsua_cli_uwp_comp
pjsua_cli_wp8
pjsua_cli_wp8_comp
Set swig_java_pjsua2 to build.
Add following folders to swig_java_pjsua2 VC++ include directories
c:\Program Files\Java\jdk-11.0.2\include
c:\Program Files\Java\jdk-11.0.2\include\win32
Build solution

I could make a workaround to compile pjsua2 for Python 3.7. If you have a better solution or just suggestion to this workaround, let me know
Preconditions:
List item
Download and extract swig (swigwin 4.0.0)
Download and install JDK
Download and install Python (Python 3.7.3 x64) and set at custom installation:
tick "Add Python to environment variables"
tick "Precompile standard library"
tick "Download debugging symbols"
tick "Download debug binaries (requires VS 2015 or later)"
set custom path if you want
add swigwin location to system path
add Python location to system path
add JAVA_HOME system variable
add %JAVA_HOME%\bin to system path
Install Visual Studio 2015 Community edition
Steps:
Download and extract pjsip 2.8
Create empty config_site.h under pjlib/inlclude/pj/ folder
Open pjproject-vs14.sln in VS
There will be an unsupported warning window, press OK
At the "Install Missing Features" windows, press install
VS installer will be started, and VS has to be closed to complete installation.
I have also added "Windows 10 SDK (10.0.10586)" to the installation.
I have also enabled developer mode on my Windows 10, but this could be optional
Open project again in VS
Set project to Release and x64 on the top dropdown
Right click on the solution, and go to Configuration Properties> Configuration
Remove all uwp and wp8 related projects from selection, like pjsua_cli_uwp_comp, pjsua_cli_wp8_comp
press OK, and build solution
I have had 34 succeeded and 0 failed solution after build.
Add following lines to pjsua2.i in pjsip-apps/src/swig folder
%inline %{
pj_ssize_t new_pj_ssize_t(int s) {
return (pj_ssize_t) s;
}
%}
this extra inline swig function provides workaround for create recorder issue
Open a command line (powershell did not work for this)
go to pjsip-apps/src/swig/python
Execute following command:
swig -I../../../../pjlib/include -I../../../../pjlib-util/include -I../../../../pjmedia/include -I../../../../pjsip/include -I../../../../pjnath/include -py3 -c++ -python -threads ../pjsua2.i
Add new "Empty Project" (Visual C++) to solution with swig_python_pjsua2 name
Add libpjproject and pjsua2_lib as reference to this new project
Right click on the Header Files>Add>Existing Item...
Add pjsip-apps\src\swig\pjsua2_wrapp.h
Add a "new filter" to the project with name "Generated Code"
Right click on "Generated code" and add new existing item.
Add pjsip-apps\src\swig\pjsua2_wrapp.cxx
Right click on the project and Linker>Input
Add "Ws2_32.Lib" to "Additional Dependencies"
Go to "VC++ Directories"
Add following folder to include path(I have used full path!):
c:\python37\include
pjnath\include
pjsip\include
pjmedia\include
pjlib-util\include
pjlib\include
Add following folder to Library Directories:
C:\Python36\libs
Set at General:
Target Name: _$(ProjectName)
Target Extension: .pyd
Configuration Type: Dynamic Library (.dll)
Go to properties of the solution, and select swig_python_pjsua2 to build
Right click on swig_python_pjsua2 and build
Build should complete successfully
You will need to file to use pjsua2 library in python.
pjsua.py located under pjsip-apps/src/swig/python
x64/Release/_swig_python_pjsua2.pyd (rename this to _pjsua2.pyd)
To test, copy those to file into a folder, start python from same folder and type:
import pjsua2

Thank You very much Krisz for this awesome work!
I just got it working for Python 2.7 / 32bit and some things are slightly different, so I will add them here:
Environment:
Windows 10
SWIG 4.0.1
PJSIP 2.9
Python 2.7.14 (32bit)
Visual Studio Express 2015 (V 14.0.25431.01 Update 3)
Whaddado:
Important: Check if your Python-interpreter is really x64 or win32 like the following:
start a python session and enter:
import platform
platform.architecture()
If you have 32bit, consider this in your choice for target platform in VS.
when you add this %inline%-Snippet to pjsip-apps/src/swig/pjsua2.i, add it at the end of the file!
when starting the swig-command, omit the "-py3"-parameter, if you use Python 2.x
there is a little typo in Krisz' description, at "pjsua.py located under pjsip-apps/src/swig/python" -> of course pjsua2.py is meant. Do not use the pjsua.py.
finally copy _pjsua2.lib and pjsua2.py into \Lib\site-packages\ and the _pjsua2.pyd into \DLLs\ to make them available everywhere.

Dependency errors when loading custom dll in python, built with Visual Studio 2017 and boost

I am trying to build the BGSLibrary python module from C++ and boost source. The library compiles without a problem on linux. For windows I am using Visual Studio 2017 and Cmake 3.9. Here were my steps.
Visual Studio Install options
Download Boost, Install Script from within Visual Studio Command Line Terminal
bootstrap.bat
b2 -j%cores% toolset=msvc-14.1 address-model=32 architecture=x86 link=static threading=multi runtime-link=static --build-type=complete stage --with-python
Rename boost_python-vc141-mt-s-1_64.lib to python_boost.lib to satisfy cmake requirements.
CMake Config options
Visual Studio External Include Directories
Visual Studio External Lib Directories
Move a copy of the originally named boost_python-vc141-mt-s-1_64.lib to the bgs build folder.
Execute Visual Studio Build. It successful creates libbgs python dll
According to this question I rename libbgs.dll to .pyd and have the boost lib in the folder. I added bgslibrary/build to path. Unfortunately not quite there yet.
Some dependency issue. A quick look at DependencyWalker is very intimidating, a huge number of problems.
Before I go and try to individual add all those .dll manually to the bgslibrary/build folder, is there a higher level error that I've done. Should I have selected "static library" in the general properties of visual studio, would that make a difference?

Solved. Dependency walker is old enough where it just shows alot of errors. Really it was jut the 4 opencv dlls. Adding opencv/build/release/bin to my path did it.

pyusb 2.7 windows install error

I know similar things have been asked, but none of the solutions work for me/I get different errors. I am trying to install PyUSB on a 64-bit Windows 7 VM, with Python 2.7.
When I run the install from command line, i get error: Unable to find vcvarsall.bat. When I try to run the compiled executable to install, I get an error saying Python 2.7 was not found in the registry.
Any help with this would be greatly appreciated.

If you haven't installed VS 2008, don't.
If you have installed VS 2008 and can't get extensions to compile, I would uninstall VS 2008 and remove the directory for it under program files. The problem is likely to be that the express version doesn't ship with the 64-bit compilers.
To get compiling extension files, install the Python 2.7 compilers for Windows from
http://www.microsoft.com/en-gb/download/details.aspx?id=44266
The instructions say that setuptools 6.0 is needed but I couldn't get it to work even after updating.
Once the compilers are installed, locate the directory containing vcvarsall.bat. For me it was
C:\Users\MY_USER_NAME\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0
Open powershell there and run .\vcvarsall.bat amd64
If you see this:
PS C:\Users\MY_USER_NAME\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0> .\vcvarsall.bat amd64
Setting environment for using Microsoft Visual Studio 2008 x64 tools.
All is well.
The next step is bizarre but essential and is explained at the end. Rename the directory 9.0 to VC.
Add the environment variable VS90COMNTOOLS and set the value to be the path of your bin directory, which should now be:
C:\Users\MY_USER_NAME\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\VC\VC\bin
Now it should all work. Go and test it on an extension file:
python setup.py build_ext --inplace
Why the renaming of 9.0 to VC? Go to your python installation directory and find the distutils directory: python_base_dir\Lib\distutils
Open up msvc9compiler.py in your text editor of choice and find line 247.
productdir = os.path.join(toolsdir, os.pardir, os.pardir, "VC")
This means that distutils will simply navigate to the environment var you added, go up two levels and then down into the VC file, hoping to find vcvarsall.bat.
You can edit this file as an alternative to renaming 9.0 -> VC but I thought that was "dirtier".
Also note the reason there needs to be no VS 2008 directory is line 242:
if not productdir or not os.path.isdir(productdir):
which ignores old registry entries for VS 2008 if the directory is removed.

How do I point easy_install to vcvarsall.bat?

I already have MSVC++ 2010 Express installed, and my vcvarsall.bat file is at C:\Program Files\Microsoft Visual Studio 10.0\VC, which is in my system PATH. When I run easy_install, it can't find vcvarsall.bat.
Is there something I need to set in my distutils.cfg file to point it to my MSVC++ installation?
G:\>easy_install hg-git
install_dir C:\Python26\Lib\site-packages\
Searching for hg-git
Best match: hg-git 0.2.6
Processing hg_git-0.2.6-py2.6.egg
hg-git 0.2.6 is already the active version in easy-install.pth
Using c:\python26\lib\site-packages\hg_git-0.2.6-py2.6.egg
Processing dependencies for hg-git
Searching for dulwich>=0.6.0
Reading http://pypi.python.org/simple/dulwich/
Reading http://samba.org/~jelmer/dulwich
Reading http://launchpad.net/dulwich
Best match: dulwich 0.7.1
Downloading http://www.samba.org/~jelmer/dulwich/dulwich-0.7.1.tar.gz
Processing dulwich-0.7.1.tar.gz
Running dulwich-0.7.1\setup.py -q bdist_egg --dist-dir c:\docume~1\mlin\locals~1
\temp\easy_install-fhraep\dulwich-0.7.1\egg-dist-tmp-qozily
error: Setup script exited with error: Unable to find vcvarsall.bat

I'd still like to know where to set that reference to vsvarsall.bat...
Well, as martineau wrote you have to have either Visual Studio 2008 or Visual C++ Express installed. Having said that I understand you would like to know where Python looks for this batch file. You can see this by looking at definition of find_vcvarsall function in distutils/msvc9compiler.py standard module. Python checks in turn if any of folders saved in the registry under keys
HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\VisualStudio\9.0\Setup\VC\ProductDir
HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\VCExpress\9.0\Setup\VC\ProductDir
(for 64bit Windows) or
HKEY_LOCAL_MACHINE\Software\Microsoft\VisualStudio\9.0\Setup\VC\ProductDir
HKEY_LOCAL_MACHINE\Software\Microsoft\VCExpress\9.0\Setup\VC\ProductDir
(for 32bit Windows) exists and if so it treats such folder as the one containing vcvarsall.bat file. If none of these folders exists Python checks if there's environment variable VS90COMNTOOLS. If this variable exits Python treats folder two levels above value of this variable as the folder containing vcvarsall.bat file.
See also my other answer which explains why you can't use MSVC++ 2010 to build extensions for Python 2.6
EDIT:
The VC++ 2008 files are now packaged in an installer from MS which can be downloaded here. Once installed vcvarsall.bat will be in C:\Users\username\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0

Met same problem on python 3.9.12 (anaconda),I install Visual C++ for Python at
C:\Users\UserName\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0
and find the msvc9compiler.py at
C:\Users\UserName\anaconda3\Lib\distutils
showed
VS_BASE = r"Software\Microsoft\VisualStudio%0.1f"
how to point the vcvarsall.bat to python ?
(I added the system PATH but not working at all.)
C:\Users\UserName\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0
May you please point out how to add registry for the value?
I did't find HKEY_LOCAL_MACHINE\Software\Microsoft\VisualStudio\9.0\Setup\VC\ProductDir
in my PC.