Develop Reference

Cython embed on Windows

I have read How to enable `--embed` with cythonize? and Cython --embed flag in setup.py but this method does not work on Windows. Let's say we have:
# app.py
print("hello")
# build.py
import setuptools # allows us to avoid calling vcvarsall.bat, see https://stackoverflow.com/a/53172602/
from distutils.core import setup
from Cython.Build import cythonize
from Cython.Compiler import Options
Options.embed = "main"
setup(ext_modules=cythonize(r'app.py', build_dir="release"), script_args=['build'], options={'build': {'build_lib': 'release'}})
Running this build.py script on Python 3.8 for Windows does not produce an app.exe file like it would with the command line command:
cython app.py --embed
Instead, it produces a .pyd file.
How to use cythonize + embed from a Python script, producing a .exe, on Windows?

Solved: in fact the problem did not come from cythonize itself, but from the fact distutils.core.setup(...) is configured to compile+link into a .pyd instead of a .exe.
Here is the solution:
from distutils._msvccompiler import MSVCCompiler # "from distutils.msvccompiler" did not work for me!
from Cython.Compiler import Options
Options.embed = "main"
cythonize(r'src\app.py', build_dir="build")
compiler = MSVCCompiler()
compiler.compile([r"build\src\app.c"], include_dirs=["C:/Python38/include"])
compiler.link_executable([r"build\src\app.obj"], "app", libraries=["python38"], library_dirs=["C:/Python38/libs"], output_dir="release", extra_preargs=["/NOIMPLIB", "/NOEXP"])
The .exe will be in the release folder.
(Note: I also upgraded Cython to the latest version 0.29.30, it might have helped as well)

How to compile a cython program to a standalone executable?

Say, I have the following cython module renamed_module.pyx. This module contains all my cython code which include C and Python functions. Normally in development below is how I compile and run renamed_module.pyx.
A python file called setup.py that calls cython to convert my pyx modules into C code.
Setup.py code:
from distutils.core import setup
from Cython.Build import cythonize
setup(
name="appname",
ext_modules=cythonize(['renamed_module.pyx', 'other1.pyx', 'other2.pyx', 'other3.pyx']),
language_level=3)
I have another python file called run_renamed_module.py with the following code:
run_renamed_module.py:
import renamed_module
import os
import sys
sys.path.insert(0, os.path.abspath("."))
renamed_module.startingfunction()
Finally I compile it as following which works perfectly: python Setup.py build_ext --inplace && python run_renamed_module.py
Question
Now, I would like to convert my renamed_module.pyx into a standalone executable or a *.exe file that would open my cython GUI App.
After doing some research, I was able to first convert my renamed_module.pyx code into C code using cython renamed_module.pyx --embed -o renamed_module_comp.c
and then compile it to a binary file using gcc -c -DMS_WIN64 -shared -pthread -Wall -IC:/Users/[username]/AppData/Local/Programs/Python/Python39/Include -LC:/Users/[username]/AppData/Local/Programs/Python\Python39\libs -o app.exe renamed_module_comp.c.
With these two steps, I fall into no errors and they compile just fine. But now when I attempt to execute app.exe, I get the following error:
This app can't run on your PC. To find a version for your PC, check with the software publisher.
As reported/commented by other developers on the web, app.exe seem to be a DLL file. So, I tried to copy app.exe into an external folder, open python terminal from that directory, and call import app. With that I get:
ImportError: DLL load failed while importing app: %1 is not a valid Win32 application.
Unfortunately I don't know where to go from here. Any direction is really appreciated.
OS: Windows 10 x64
Python Version: Python 3.9.1
Cython Version: Cython version 0.29.23
GCC Version: gcc.exe (GCC) 9.2.0
GUI Libs: PyQT5 - Tkinter and pysimplegui
Summary of the question: I basically, want to compile my cython GUI app into a standalone executable program.

Compile cython code with "python compile.py" and no "build" command-line parameter

I have a compile.py script:
from distutils.core import setup
from Cython.Build import cythonize
setup(ext_modules = cythonize("module1.pyx"))
that compiles my Cython code. The drawback is that I have to call it with a command-line parameter build:
python compile.py build
Instead, I would like to be able to call this compile.py directly from Sublime Text, as usual, with CTRL+B. To do that, it should work from:
python compile.py
Question: how to modify the above script so that it can be run with python compile.py?

Method #1:
Use script_args like this:
setup(ext_modules=cythonize("module1.pyx", build_dir="build"), script_args=['build'])
or
setup(ext_modules=cythonize("module1.pyx", build_dir="build"), script_args=['build_ext'])
(both work).
If you want the output files to be in the same directory, you can use:
setup(ext_modules=cythonize("module1.pyx", build_dir="build"), script_args=['build'],
options={'build':{'build_lib':'.'}})
or
setup(ext_modules=cythonize("module1.pyx", build_dir="build"), script_args=['build_ext'],
options={'build_ext':{'inplace':True}})
Method #2:
Add this on top:
import sys; sys.argv = ["", "build"]
It's a bit hack-ish but it works fine, and avoids to have to create a new build-system, like with Build and run with arguments in Sublime Text 2 (link kindly provided by #Melvin).

python cmd error: invalid command 'y'

Python - When parsing the following command-line arguments...
sys.argv = ('%s %s build_ext --inplace' % ('python', sys.argv[0]))
...for Cython inside my buildscript "myscript_pyd_setup.py" for "myscript.pyd" that uses the "myscript.py" file I get the following error (e1):
python error 1: invalid command 'y'
The python command-line option python -- help revealed no corresponding option 'y'. This makes sense otherwise the error would not exist. Searching SO, Python docs and the web resulted in unrelated articles about Tkinter. As I do absolutely nothing with Tkinter, as far as I know it, I'm wondering if this error is from the python command-line interpreter at all?
My effort:
What I tried to accomplish is parsing arguments sys.argv.append('build_ext --inplace') to python command-line interpreter when running the "buildscript" from within the editor (Komodo edit 11.x) but it returns also with an error (e2) similar to the version from command-line or another editor.
python error 2: invalid command name 'build_ext --inplace'
Running "myscript_pyd_setup.py" with sys.argv.append('build_ext') builds the required *.pyd file just fine. What I don't understand is why its choking on --infile. This works for py2exe and pyinstaller.
The objective:
To run the setup-script of "myscript.py" from within the editor and not having to flip back-and-forth to the command-line editor for compiling *.py > *.pyd when I changed code inside "myscript.py" and want to see the result quickly.
Note: parsing command-line python myscript_pyd_setup.py build_ext --inplace works fine when sys.argv... is commented-out in the "buildscript"!
A third option was to use cythonize in combo with the "myscript.py" and "myscript.pyd" files but that showed a copyfile error for "myscript.pyd". Relevant but not for the above asked "error = y" question.
Any thoughts and help on how to automate this part to prevent RSI are more than welcome! Thx.
My "myscript.py" example code:
import sys, time, os
#...snippet...
def print_me():
text = "bar(man), yes, Hello, how do you do Mr. foo?"
return text
if __name__ == '__main__':
#...snippet...
print_me()
The myscript_pyd_setup.py:
# myscript.py
try:
from setuptools import setup
from setuptools import Extension
except ImportError:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
#from Cython.Build import cythonize
import sys
print 'sys.argv[0] : %s' % sys.argv[0]
#sys.argv = ('%s %s build_ext --inplace' % ('python', sys.argv[0]))
sys.argv.append('build_ext --inplace')
ext_modules = [Extension("myscript",['myscript.py'])]
#ext_modules = cythonize("myscript.py")
#setup: "name" and "cmdclass" are commented-out when using cythonize.
setup(
name= 'XYZ model class',
cmdclass = {'build_ext': build_ext},
include_dirs = [],
ext_modules = ext_modules)

sys.argv is a list of arguments. When you append 'build_ext --inplace', you literally append that as a single argument (as if you've passed it enclosed in quotes from the shell command line). That is what happened in the second case.
In the first one, you've assigned string back to sys.argv. But string is also a sequence, so your commands ran as if called with (argv[1:]): ['y', 't', `h`, ...]
In either case you ended up with an option/sub-command unknown the the argument parser.

Cythonize python modules to a different target directory

I'm trying to cythonize some python modules (on windows) and want the resulting .c files stored in a different directory.
I tried something like this:
from Cython.Build import cythonize
module_list = cythonize(r'C:\myproject\module.py',
force=True,
build_dir=r'C:\myproject\compiled_modules')
The compilation is done but the file module.c is created in C:\myproject and not in C:\myproject\compiled_modules as expected. What am I doing wrong?

I'm not sure the cythonize function can be coerced into doing that. I would just use the command line:
/path/to/cython yourmod.py -o compiled_modules/yourmod.c
Edit:
This command can be called from a python script using the subprocess module:
import subprocess
subprocess.check_call(["/path/to/cython", "yourmod.py", "-o", "compiled_modules/yourmod.c"])
Edit 2:
According to this Sage developer, relocating the c/cpp files is not supported by cython, so it looks like either compiling from the command line or using a normal configuration and then moving the c/cpp files are your only options.