I have read How to enable `--embed` with cythonize? and Cython --embed flag in setup.py but this method does not work on Windows. Let's say we have:
# app.py
print("hello")
# build.py
import setuptools # allows us to avoid calling vcvarsall.bat, see https://stackoverflow.com/a/53172602/
from distutils.core import setup
from Cython.Build import cythonize
from Cython.Compiler import Options
Options.embed = "main"
setup(ext_modules=cythonize(r'app.py', build_dir="release"), script_args=['build'], options={'build': {'build_lib': 'release'}})
Running this build.py script on Python 3.8 for Windows does not produce an app.exe file like it would with the command line command:
cython app.py --embed
Instead, it produces a .pyd file.
How to use cythonize + embed from a Python script, producing a .exe, on Windows?
Solved: in fact the problem did not come from cythonize itself, but from the fact distutils.core.setup(...) is configured to compile+link into a .pyd instead of a .exe.
Here is the solution:
from distutils._msvccompiler import MSVCCompiler # "from distutils.msvccompiler" did not work for me!
from Cython.Compiler import Options
Options.embed = "main"
cythonize(r'src\app.py', build_dir="build")
compiler = MSVCCompiler()
compiler.compile([r"build\src\app.c"], include_dirs=["C:/Python38/include"])
compiler.link_executable([r"build\src\app.obj"], "app", libraries=["python38"], library_dirs=["C:/Python38/libs"], output_dir="release", extra_preargs=["/NOIMPLIB", "/NOEXP"])
The .exe will be in the release folder.
(Note: I also upgraded Cython to the latest version 0.29.30, it might have helped as well)
Related
Say, I have the following cython module renamed_module.pyx. This module contains all my cython code which include C and Python functions. Normally in development below is how I compile and run renamed_module.pyx.
A python file called setup.py that calls cython to convert my pyx modules into C code.
Setup.py code:
from distutils.core import setup
from Cython.Build import cythonize
setup(
name="appname",
ext_modules=cythonize(['renamed_module.pyx', 'other1.pyx', 'other2.pyx', 'other3.pyx']),
language_level=3)
I have another python file called run_renamed_module.py with the following code:
run_renamed_module.py:
import renamed_module
import os
import sys
sys.path.insert(0, os.path.abspath("."))
renamed_module.startingfunction()
Finally I compile it as following which works perfectly: python Setup.py build_ext --inplace && python run_renamed_module.py
Question
Now, I would like to convert my renamed_module.pyx into a standalone executable or a *.exe file that would open my cython GUI App.
After doing some research, I was able to first convert my renamed_module.pyx code into C code using cython renamed_module.pyx --embed -o renamed_module_comp.c
and then compile it to a binary file using gcc -c -DMS_WIN64 -shared -pthread -Wall -IC:/Users/[username]/AppData/Local/Programs/Python/Python39/Include -LC:/Users/[username]/AppData/Local/Programs/Python\Python39\libs -o app.exe renamed_module_comp.c.
With these two steps, I fall into no errors and they compile just fine. But now when I attempt to execute app.exe, I get the following error:
This app can't run on your PC. To find a version for your PC, check with the software publisher.
As reported/commented by other developers on the web, app.exe seem to be a DLL file. So, I tried to copy app.exe into an external folder, open python terminal from that directory, and call import app. With that I get:
ImportError: DLL load failed while importing app: %1 is not a valid Win32 application.
Unfortunately I don't know where to go from here. Any direction is really appreciated.
OS: Windows 10 x64
Python Version: Python 3.9.1
Cython Version: Cython version 0.29.23
GCC Version: gcc.exe (GCC) 9.2.0
GUI Libs: PyQT5 - Tkinter and pysimplegui
Summary of the question: I basically, want to compile my cython GUI app into a standalone executable program.
I'm trying cython for the first time. Here is what I have.
setup.py
import os
from distutils.core import setup
from Cython.Build import cythonize
os.chdir(os.path.dirname(__file__))
setup(ext_modules = cythonize('example123.pyx'))
example123.pyx
def say_hello(name):
print(f"Hello {name}")
After setup.py build_ext --inplace I only see example123.c in the folder, no other files created. Searching through all the files didn't provide any results. As far as I know, a .so or .pyd file must have appeared to import into a python script. What may be the problem?
My specs: Windows 7 x64, gcc mingw64, python 3.8, cython 0.29.24
I am trying to link to my own C library from Cython, following the directions I've found on the web, including this answer:
Using Cython To Link Python To A Shared Library
I am running IPython through Spyder.
My setup.py looks like this:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
import numpy as np
setup(
ext_modules = cythonize(
[Extension("*",["*.pyx"],
libraries =["MyLib"],
extra_compile_args = ["-fopenmp","-O3"],
extra_link_args=["-L/path/to/lib"])
]),
include_dirs = [np.get_include()],
)
The file libMyLib.so is in /path/to/lib and it compiles fine.
I have a Python script in my IPython profile startup folder that does this
try:
os.environ["LD_LIBRARY_PATH"] += ":/path/to/lib"
except KeyError:
os.environ["LD_LIBRARY_PATH"] = "/path/to/lib"
I can confirm that this is running, because if I type os.environ["LD_LIBRARY_PATH"] into the IPython interpreter, it returns /path/to/lib
But when I try to load the Cython module (i.e. import mycythonmodule) I get:
ImportError: libMyLib.so: cannot open shared object file: No such file or directory
I've also tried putting libMyLib.so in other places to see if cython would find it:
In the directory where Python is running
On the Python path
In the same folder as the cython module
But it still doesn't find the shared library. The only way I can get it to find the library is by dropping it in /usr/lib, but I don't want it there, I want to be able to set the library path.
Am I missing something?
I'm self-answering, in case anyone else runs into the same problem. Looks like the answers are here:
Set LD_LIBRARY_PATH before importing in python
Changing LD_LIBRARY_PATH at runtime for ctypes
According to these answers (and my experience), the linker reads LD_LIBRARY_PATH when python is launched, so changing it from within python doesn't have any useful effect, at least not the effect I was hoping for. The only solution is to either wrap python in a shell script that sets LD_LIBRARY_PATH, or else drop the shared object somewhere on the linker search path.
Kind of a pain, but it is what it is.
I have fixed it by change setup.py.
I have a C++ dynamic shared library called "libtmsmdreader.so". and a header file named "TmsMdReader.hpp"
I wrapper C++ shared library to cython library called "tmsmdreader-pythonxxxxxx.so"
from setuptools import setup
from distutils.extension import Extension
from Cython.Build import cythonize
setup(
name="tmsmdreader",
ext_modules=cythonize([
Extension(
name="tmsmdreader",
language="c++",
sources=["TmsMdReaderApi.pyx"],
libraries=["tmsmdreader"],
library_dirs=["."],
include_dirs=["."],
extra_compile_args=["-std=c++14"],
compiler_directives={'language_level': 3},
runtime_library_dirs=["."])
]))
library_dirs=["."] and runtime_library_dirs=["."] can fixed LD_LIBRARY_PATH if libtmsmdreader.so in python scripy directory
I'm trying to cythonize some python modules (on windows) and want the resulting .c files stored in a different directory.
I tried something like this:
from Cython.Build import cythonize
module_list = cythonize(r'C:\myproject\module.py',
force=True,
build_dir=r'C:\myproject\compiled_modules')
The compilation is done but the file module.c is created in C:\myproject and not in C:\myproject\compiled_modules as expected. What am I doing wrong?
I'm not sure the cythonize function can be coerced into doing that. I would just use the command line:
/path/to/cython yourmod.py -o compiled_modules/yourmod.c
Edit:
This command can be called from a python script using the subprocess module:
import subprocess
subprocess.check_call(["/path/to/cython", "yourmod.py", "-o", "compiled_modules/yourmod.c"])
Edit 2:
According to this Sage developer, relocating the c/cpp files is not supported by cython, so it looks like either compiling from the command line or using a normal configuration and then moving the c/cpp files are your only options.
I am trying to create an exe on Windows from a Python 3 package with a C extension module. In distutils, you can create an extension like this:
from distutils.core import setup, Extension
module1 = Extension('demo',
sources = ['demo.c'])
setup (name = 'PackageName',
version = '1.0',
description = 'This is a demo package',
ext_modules = [module1])
Than, the extension will be compiled with the appropriate compiler and placed alongside your other modules with the command:
python setup.py build_ext --inplace
cx_Freeze is a module that can package your code into an exe file along with a Python interpreter and the relevant packages. Then, an end user can use your program without having a Python installation. Unfortunately, cx_Freeze doesn't have an Extension class, and I cannot find a way to handle compilation with cx_Freeze.
One solution I am unsure about is to first build the extensions in place with distutils/setuptools, and then use cx_Freeze to create the executable. I don't want to reinvent the wheel though, so I wonder if someone else with more experience in this area has a solution.
I found a working solution. I can import Extension from distutils, and pass it into the setup from cx_Freeze:
from cx_Freeze import setup
from distutils.core import Extension
...
setup=(...
ext_modules=Extension(...))
This makes sense, since cx_Freeze is built on top of distutils. Originally, I was trying to use setuptools.setup, but that doesn't work.