Develop Reference

Cython embed on Windows

I have read How to enable `--embed` with cythonize? and Cython --embed flag in setup.py but this method does not work on Windows. Let's say we have:
# app.py
print("hello")
# build.py
import setuptools # allows us to avoid calling vcvarsall.bat, see https://stackoverflow.com/a/53172602/
from distutils.core import setup
from Cython.Build import cythonize
from Cython.Compiler import Options
Options.embed = "main"
setup(ext_modules=cythonize(r'app.py', build_dir="release"), script_args=['build'], options={'build': {'build_lib': 'release'}})
Running this build.py script on Python 3.8 for Windows does not produce an app.exe file like it would with the command line command:
cython app.py --embed
Instead, it produces a .pyd file.
How to use cythonize + embed from a Python script, producing a .exe, on Windows?

Solved: in fact the problem did not come from cythonize itself, but from the fact distutils.core.setup(...) is configured to compile+link into a .pyd instead of a .exe.
Here is the solution:
from distutils._msvccompiler import MSVCCompiler # "from distutils.msvccompiler" did not work for me!
from Cython.Compiler import Options
Options.embed = "main"
cythonize(r'src\app.py', build_dir="build")
compiler = MSVCCompiler()
compiler.compile([r"build\src\app.c"], include_dirs=["C:/Python38/include"])
compiler.link_executable([r"build\src\app.obj"], "app", libraries=["python38"], library_dirs=["C:/Python38/libs"], output_dir="release", extra_preargs=["/NOIMPLIB", "/NOEXP"])
The .exe will be in the release folder.
(Note: I also upgraded Cython to the latest version 0.29.30, it might have helped as well)

how to use pythran add a myfunction from other py files?

toolsTep.py
def HelloWord():
print('hello word')
testpythran.py
from calaTools.toolsTep import *
#pythran export callOtherPyFiles()
def callOtherPyFiles():
HelloWord()
complie pythran testpythran.py
CRITICAL :
I am in trouble. Your input file does not seem to match Pythran's constraints...
testpythran.py:
None:None error: Module 'calaTools.toolsTep' not found.
when two function in save file and it can find ,in diffrent file it occured those errors

When distributing a Python application with Pythran modules, you can either:
declare the module as a regular Python module. After all, they are 100% Python compatible.
declare them as a PythranExtension and Pythran will compile them:
from distutils.core import setup
These two lines are required to be able to use pythran in the setup.py
import setuptools
setuptools.dist.Distribution(dict(setup_requires='pythran'))
from pythran.dist import PythranExtension, PythranBuildExt
setup(...,
ext_modules=[PythranExtension("mymodule", ["mymodule.py"])],
cmdclass={"build_ext": PythranBuildExt})
PythranBuildExt is optional, but necessary to build extensions with different C++ compilers. It derives from distuil’s build_ext by default, but you can change its base class by using PythranBuildExt[base_cls] instead.
all configuration options supported in .pythranrc can also be passed through the optional config argument, in the form of a list, e.g. config=['compiler.blas=openblas']
from pythran doc.https://pythran.readthedocs.io/en/latest/MANUAL.html

Use Setuptools with C Extension as part of package

I wrote a C Extension to access an error message for a camera from a proprietary library. The structure is
setup.py
dicamsdk\
|---__init__.py
|---control.py
|---pcoError.c
with setup.py
from setuptools import setup, Extension, find_packages
from dicamsdk.control import __version__
pcoErrorModule = Extension("dicamsdk.pcoError",
sources=["dicamsdk/pcoError.c"],
include_dirs=['C:\Program Files (x86)'
'\Digital Camera Toolbox'
'\Sensicam SDK\include'],
define_macros=[("PCO_ERRT_H_CREATE_OBJECT", None)],
)
setup(
name="pydicamsdk",
platforms=["win-amd64", 'win32'],
license="GNU GPLv3",
ext_modules=[pcoErrorModule],
packages=find_packages(),
version=__version__
)
and the control.py intends to import the compiled C Extension with
from . import pcoError
When I try to build (or install) the package I always receive the error ImportError: cannot import name 'pcoError'.
The only way it seems to work is to comment out the import in control.py and build the C Extension with setup.py build_ext --inplace. Just with the compiled present I can build/install my library.
Is there a solution to be implemented in my setup.py to compile my extension in first place to enable a simple installation?

A more proper solution based on your own answer:
Your version is presumably for the entire project, not just the control module. It is standard to define __version__ in __init__.py. In that case, your import in setup.py would look like from dicamsdk import __version__. This will cause no conflicts, unless you do something foolish like import all your modules automatically from the package root.

The problem was with the import at the beginning of setup.py. The import in line 2 (from dicamsdk.control import __version__) forced the code check of the module.
When removed, the install or build of the packages runs correctly.

Cython unable to find shared object file

I am trying to link to my own C library from Cython, following the directions I've found on the web, including this answer:
Using Cython To Link Python To A Shared Library
I am running IPython through Spyder.
My setup.py looks like this:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
import numpy as np
setup(
ext_modules = cythonize(
[Extension("*",["*.pyx"],
libraries =["MyLib"],
extra_compile_args = ["-fopenmp","-O3"],
extra_link_args=["-L/path/to/lib"])
]),
include_dirs = [np.get_include()],
)
The file libMyLib.so is in /path/to/lib and it compiles fine.
I have a Python script in my IPython profile startup folder that does this
try:
os.environ["LD_LIBRARY_PATH"] += ":/path/to/lib"
except KeyError:
os.environ["LD_LIBRARY_PATH"] = "/path/to/lib"
I can confirm that this is running, because if I type os.environ["LD_LIBRARY_PATH"] into the IPython interpreter, it returns /path/to/lib
But when I try to load the Cython module (i.e. import mycythonmodule) I get:
ImportError: libMyLib.so: cannot open shared object file: No such file or directory
I've also tried putting libMyLib.so in other places to see if cython would find it:
In the directory where Python is running
On the Python path
In the same folder as the cython module
But it still doesn't find the shared library. The only way I can get it to find the library is by dropping it in /usr/lib, but I don't want it there, I want to be able to set the library path.
Am I missing something?

I'm self-answering, in case anyone else runs into the same problem. Looks like the answers are here:
Set LD_LIBRARY_PATH before importing in python
Changing LD_LIBRARY_PATH at runtime for ctypes
According to these answers (and my experience), the linker reads LD_LIBRARY_PATH when python is launched, so changing it from within python doesn't have any useful effect, at least not the effect I was hoping for. The only solution is to either wrap python in a shell script that sets LD_LIBRARY_PATH, or else drop the shared object somewhere on the linker search path.
Kind of a pain, but it is what it is.

I have fixed it by change setup.py.
I have a C++ dynamic shared library called "libtmsmdreader.so". and a header file named "TmsMdReader.hpp"
I wrapper C++ shared library to cython library called "tmsmdreader-pythonxxxxxx.so"
from setuptools import setup
from distutils.extension import Extension
from Cython.Build import cythonize
setup(
name="tmsmdreader",
ext_modules=cythonize([
Extension(
name="tmsmdreader",
language="c++",
sources=["TmsMdReaderApi.pyx"],
libraries=["tmsmdreader"],
library_dirs=["."],
include_dirs=["."],
extra_compile_args=["-std=c++14"],
compiler_directives={'language_level': 3},
runtime_library_dirs=["."])
]))
library_dirs=["."] and runtime_library_dirs=["."] can fixed LD_LIBRARY_PATH if libtmsmdreader.so in python scripy directory

Cythonize python modules to a different target directory

I'm trying to cythonize some python modules (on windows) and want the resulting .c files stored in a different directory.
I tried something like this:
from Cython.Build import cythonize
module_list = cythonize(r'C:\myproject\module.py',
force=True,
build_dir=r'C:\myproject\compiled_modules')
The compilation is done but the file module.c is created in C:\myproject and not in C:\myproject\compiled_modules as expected. What am I doing wrong?

I'm not sure the cythonize function can be coerced into doing that. I would just use the command line:
/path/to/cython yourmod.py -o compiled_modules/yourmod.c
Edit:
This command can be called from a python script using the subprocess module:
import subprocess
subprocess.check_call(["/path/to/cython", "yourmod.py", "-o", "compiled_modules/yourmod.c"])
Edit 2:
According to this Sage developer, relocating the c/cpp files is not supported by cython, so it looks like either compiling from the command line or using a normal configuration and then moving the c/cpp files are your only options.