Say, I have the following cython module renamed_module.pyx. This module contains all my cython code which include C and Python functions. Normally in development below is how I compile and run renamed_module.pyx.
A python file called setup.py that calls cython to convert my pyx modules into C code.
Setup.py code:
from distutils.core import setup
from Cython.Build import cythonize
setup(
name="appname",
ext_modules=cythonize(['renamed_module.pyx', 'other1.pyx', 'other2.pyx', 'other3.pyx']),
language_level=3)
I have another python file called run_renamed_module.py with the following code:
run_renamed_module.py:
import renamed_module
import os
import sys
sys.path.insert(0, os.path.abspath("."))
renamed_module.startingfunction()
Finally I compile it as following which works perfectly: python Setup.py build_ext --inplace && python run_renamed_module.py
Question
Now, I would like to convert my renamed_module.pyx into a standalone executable or a *.exe file that would open my cython GUI App.
After doing some research, I was able to first convert my renamed_module.pyx code into C code using cython renamed_module.pyx --embed -o renamed_module_comp.c
and then compile it to a binary file using gcc -c -DMS_WIN64 -shared -pthread -Wall -IC:/Users/[username]/AppData/Local/Programs/Python/Python39/Include -LC:/Users/[username]/AppData/Local/Programs/Python\Python39\libs -o app.exe renamed_module_comp.c.
With these two steps, I fall into no errors and they compile just fine. But now when I attempt to execute app.exe, I get the following error:
This app can't run on your PC. To find a version for your PC, check with the software publisher.
As reported/commented by other developers on the web, app.exe seem to be a DLL file. So, I tried to copy app.exe into an external folder, open python terminal from that directory, and call import app. With that I get:
ImportError: DLL load failed while importing app: %1 is not a valid Win32 application.
Unfortunately I don't know where to go from here. Any direction is really appreciated.
OS: Windows 10 x64
Python Version: Python 3.9.1
Cython Version: Cython version 0.29.23
GCC Version: gcc.exe (GCC) 9.2.0
GUI Libs: PyQT5 - Tkinter and pysimplegui
Summary of the question: I basically, want to compile my cython GUI app into a standalone executable program.
Related
I have read How to enable `--embed` with cythonize? and Cython --embed flag in setup.py but this method does not work on Windows. Let's say we have:
# app.py
print("hello")
# build.py
import setuptools # allows us to avoid calling vcvarsall.bat, see https://stackoverflow.com/a/53172602/
from distutils.core import setup
from Cython.Build import cythonize
from Cython.Compiler import Options
Options.embed = "main"
setup(ext_modules=cythonize(r'app.py', build_dir="release"), script_args=['build'], options={'build': {'build_lib': 'release'}})
Running this build.py script on Python 3.8 for Windows does not produce an app.exe file like it would with the command line command:
cython app.py --embed
Instead, it produces a .pyd file.
How to use cythonize + embed from a Python script, producing a .exe, on Windows?
Solved: in fact the problem did not come from cythonize itself, but from the fact distutils.core.setup(...) is configured to compile+link into a .pyd instead of a .exe.
Here is the solution:
from distutils._msvccompiler import MSVCCompiler # "from distutils.msvccompiler" did not work for me!
from Cython.Compiler import Options
Options.embed = "main"
cythonize(r'src\app.py', build_dir="build")
compiler = MSVCCompiler()
compiler.compile([r"build\src\app.c"], include_dirs=["C:/Python38/include"])
compiler.link_executable([r"build\src\app.obj"], "app", libraries=["python38"], library_dirs=["C:/Python38/libs"], output_dir="release", extra_preargs=["/NOIMPLIB", "/NOEXP"])
The .exe will be in the release folder.
(Note: I also upgraded Cython to the latest version 0.29.30, it might have helped as well)
I'm trying cython for the first time. Here is what I have.
setup.py
import os
from distutils.core import setup
from Cython.Build import cythonize
os.chdir(os.path.dirname(__file__))
setup(ext_modules = cythonize('example123.pyx'))
example123.pyx
def say_hello(name):
print(f"Hello {name}")
After setup.py build_ext --inplace I only see example123.c in the folder, no other files created. Searching through all the files didn't provide any results. As far as I know, a .so or .pyd file must have appeared to import into a python script. What may be the problem?
My specs: Windows 7 x64, gcc mingw64, python 3.8, cython 0.29.24
I am trying to setup a CMake project that creates python bindings for its c++ functions using pybind11 on Ubuntu.
The directory structure is:
pybind_test
arithmetic.cpp
arithmetic.h
bindings.h
CMakeLists.txt
main.cpp
pybind11 (github repo clone)
Repo contents (https://github.com/pybind/pybind11)
The CMakeLists.txt file:
cmake_minimum_required(VERSION 3.10)
project(pybind_test)
set(CMAKE_CXX_STANDARD 17)
find_package(PythonLibs REQUIRED)
include_directories(${PYTHON_INCLUDE_DIRS})
include_directories(pybind11/include/pybind11)
add_executable(pybind_test main.cpp arithmetic.cpp)
add_subdirectory(pybind11)
pybind11_add_module(arithmetic arithmetic.cpp)
target_link_libraries(pybind_test ${PYTHON_LIBRARIES})
The repository builds successfully and the file arithmetic.cpython-36m-x86_64-linux-gnu.so is produced. How do I import this shared object file into python?
The documentation in the pybind11 docs has this line
$ c++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix`
but I want to build using CMake and I also don't want to have to specify extra include directories every time I run python to use this module.
How would I import this shared object file into python like a normal python module?
I am using Ubuntu 16.04.
If you open a terminal, go to the directory where arithmetic.cpython-36m-x86_64-linux-gnu.so is located and run python followed by import arithmetic the module will get imported just like any other module.
Another options is to use the method of
import sys
sys.path.insert(0, 'path/to/directory/where/so-file/is')
import arithmetic
With this method you can use both relative and absolute path.
Besides the solution of setting the path in the Python script that is presented by #super, you have two more generic solutions.
Setting PYTHONPATH
There is an environment variable in Linux (and macOS) called PYTHONPATH. If you add the path that contains your *.so to the PYTHONPATH before you call Python, Python will be able to find your library.
To do this:
export PYTHONPATH="/path/that/contains/your/so":"${PYTHONPATH}"
To apply this 'automatically' for every session you can add this line to ~/.bash_profile or ~/.bashrc (see the same reference). In that case, Python will always be able to find your library.
Copying your to a path already in Python's path
You can also 'install' the library. The usual way to do this is to create a setup.py file. If set up correctly you can build and install your library using
python setup.py build
python setup.py install
(Python will know where to put your library. You can 'customize' a bit with an option like --user to use your home-folder, but this doesn't seems to be of particular interest to you.)
The question remains: How to write setup.py? For your case you can actually call CMake. In fact there exists an example that does exactly that: pybind/cmake_example. You can basically copy-paste from there.
I have a C++ file called VBB.cpp which contains implementations of a few classes and I wrote Python bindings for those classes using the pybind11 library, these are located in bindings.cpp. I can successfully compile the code with:
g++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` bindings.cpp VBB.cpp -o VBB`python3-config --extension-suffix
And then use the C++ code from Python with import library.
I want to turn this into a Python package via setuptools. I just used the example setup.py file available at https://github.com/pybind/python_example and modified the Extension call with
Extension(
'VBB',
['src/bindings.cpp', 'src/VBB.cpp'],
include_dirs=[
# Path to pybind11 headers
get_pybind_include(),
get_pybind_include(user=True)
],
language='c++'
),
If I run the install script it compiles but if I try to run import VBB in Python, I get the following error:
ImportError: dynamic module does not define module export function (PyInit_VBB)
I'm new to using setuptools so I'm not sure if I'm doing something wrong. The example package from GitHub works without any issues.
I'm trying to compile a Python wrapper to a small C++ library I've written. I've written the following setup.py script to try to use setuptools to compile the wrapper:
from setuptools import setup, Extension
import numpy as np
import os
atmcmodule = Extension(
'atmc',
include_dirs=[np.get_include(), '/usr/local/include'],
libraries=['mcopt', 'c++'], # my C++ library is at ./build/libmcopt.a
library_dirs=[os.path.abspath('./build')],
sources=['atmcmodule.cpp'],
language='c++',
extra_compile_args=['-std=c++11', '-v'],
)
setup(name='tracking',
version='0.1',
description='Particle tracking and MC optimizer module',
ext_modules=[atmcmodule],
)
However, when I run python setup.py build on OS X El Capitan, clang complains about not finding some C++ standard library headers:
In file included from atmcmodule.cpp:7:
In file included from ./mcopt.h:11:
In file included from ./arma_include.h:4:
/usr/local/include/armadillo:54:12: fatal error: 'initializer_list' file not found
#include <initializer_list>
^
1 error generated.
error: command 'gcc' failed with exit status 1
Passing the -v flag to the compiler shows that it is searching the following include paths:
#include <...> search starts here:
/Users/[username]/miniconda3/include
/Users/[username]/miniconda3/lib/python3.4/site-packages/numpy/core/include
/usr/local/include
/Users/[username]/miniconda3/include/python3.4m
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/c++/4.2.1
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/c++/4.2.1/backward
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/7.0.0/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/System/Library/Frameworks (framework directory)
End of search list.
This apparently doesn't include the path to the C++ standard library headers. If I compile a small test C++ source with the -v option, I can see that clang++ normally also searches the path /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1, and if I include this path in the include_dirs option for Extension in my setup.py script, then the extension module compiles correctly and works. However, hard-coding this path into the script doesn't seem like a good solution since this module also needs to work on Linux.
So, my question is how do I properly make setuptools include the required headers?
Update (11/22/2015)
As setuptools tries to compile the extension, it prints the first command it's running:
gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/[username]/miniconda3/include -arch x86_64 -I/Users/[username]/miniconda3/lib/python3.4/site-packages/numpy/core/include -I/Users/[username]/Documents/Code/ar40-aug15/monte_carlo/mcopt -I/usr/local/include -I/Users/[username]/miniconda3/include/python3.4m -c /Users/[username]/Documents/Code/ar40-aug15/monte_carlo/atmc/atmcmodule.cpp -o build/temp.macosx-10.5-x86_64-3.4/Users/[username]/Documents/Code/ar40-aug15/monte_carlo/atmc/atmcmodule.o -std=c++11 -fopenmp -v
If I paste this command into a terminal and run it myself, the extension compiles successfully. So I suspect either setuptools is modifying some environment variables I'm not aware of, or it's lying a little about the commands it's actually running.
Setuptools tries to compile C/C++ extension modules with the same flags used to compile the Python interpreter. After checking the flags used to compile my Python install (from Anaconda), I found it was compiling for a minimum Mac OS X version of 10.5. This seems to make it use the GCC libstdc++ instead of clang's libc++ (which supports C++11).
This can be fixed by either setting the environment variable MACOSX_DEPLOYMENT_TARGET to 10.9 (or later), or adding '-mmacosx-version-min=10.9' to extra_compile_args.