I'm a Matlab and C++ user, and have recently discovered python (spyder) as a possible replacement for both. One of the main benefits I thought python had was the ability to work in interpreter mode, and then seamlessly translate it into fast compiled code once I'm satisfied with the result. The interpreted environment is great for prototyping, analyzing data while stopped at a breakpoint, throwing plots and images all around, etc.
I started looking into Cython, and I don't fully understand the programming flow. Lets say you have a .py code you'd like to speed up - Do you have to write a .pyx file from scratch? Can you run a .pyx file in interpreted mode as if it were a regular .py file (before compiling)? How do you debug the code in a .pyx file?
I don't have too much experience with Cython, but judging from this entry in their documentation, the recommended workflow is to have a setup.py file with the following lines:
from distutils.core import setup
from Cython.Build import cythonize
setup(name='Hello world app', ext_modules=cythonize("hello.pyx"))
Here hello.pyx is just an example file, you will have to replace the string to reference your Python script.
Afterwards you will be able to call
python setup.py build_ext --inplace
which will compile your code and leave you with a new file. Now, as long as that file is in the same directory, you can easily import what you defined in your file, just like with any other module. E.g., suppose you compiled a file hello.pyx with the function f, you could write:
from hello import f
and then proceed to use f.
Now, regarding your other questions. .pyx seems to just indicate, that this should be Cython code, there is no real difference. Using the method with a setup.py script as described above, you could also reference a file with the ending .py. However, Python won't allow you to import from .pyx files, only from the files created after compiling.
As to how you would debug code in a .pyx file, I don't have enough information on that, though you could probably just debug the non-compiled file like a .py file.
Related
I am developing a project that I want to release as closed source, but its written in python, and you can open any file with a text editor to see the code, so not ideal. I use pyinstaller to compile the project, but that only "hides" the main file, and the rest of them are still accesible, which is not ideal at all. I know that python compiles the imported files with cpython, and those are the .pyc files in the pycache folder, but I am also aware that these files can be decompiled easily, so that isn't a good solution. Is there any way I can compile my python packages and to make them non-readable by the user but still be importable by python?
You might want to look into Cython
Cython can compile your python code into native C while still being available to be imported from python.
My goal is to have the ability to call functions in C++ with meaningful arguments.
I can't do that with just subprocess.call because then I go into main(int argc,char** argv) and I have a bunch of strings to deal with. I do not want to have to parse matrices out of strings.
I'm trying to use Cython because that seems like the reasonable thing to do. But although there are a good amount of guides for getting Cython running most of them are for 2.7, and it's rare to see two advise the same thing.
My question basically is does anybody here know how to get Cython running on Py3.5? Or know of a guide or something? I'm lost.
Okay so I had a pretty silly mistake, was compiling with msvs, spent so much time trying to get mingw to work but forget that, 'msvc' does the trick. For any passersby if you're on 3.5+ you should be using Visual Studio 2015. After installing cython with 'pip3 install cython', create a setup.py file where you put this
from distutils.core import setup
from Cython.Build import cythonize
setup(ext_modules = cythonize(
"TestCython.pyx", # our Cython source
#sources=["Rectangle.cpp"], # additional source file(s)
language="c++", # generate C++ code
))
Create a .pyx file (let's say 'TestCython.pyx') where you write whatever you want for example (let's say 'print("Hello World")'. The sources argument for cythonize is optional.
Then cd into where your .pyx and .py file is and run
'python setup.py build_ext --inplace --compiler=msvc'
This should compile to a .cpp and then .pyd file (the latter is the one you will use). If you just had the hello world that will get printed out as soon as you import TestCython.
Refer to the docs and google for anything else. ;)
I wrote a python/c extension file lda_model.c
and I added setup.py:
from setuptools import setup, Extension
modules = [Extension('c_lda_model', sources=["lda_model.c"])]
setup(ext_modules=modules)
Now I have to compile the C code by
python setup.py build
before running python code where call the C code.
Is there any way to automatically compile the invoked C extension,
while running the python code?
Not with the standard way of writing extensions, which is what you just do.
However, there are a couple other approaches of writing native code extensions to Python which do compilation in execution time.
One such example is Weave that comes with Scipy - there are others ways if you look for.
I downloaded a pyx and c file from the internet and I am just wondering how can i incorporate it into python? The documentation on Cython is fairly vague and mainly focus on generating pyx/c from py file. It would be great if you could give me some solid examples on how to do this properly. Many thanks
The Cython executable turns a .pyx file into a .c file. You then use your favorite C build tool to compile it into a shared library (e.g. an .so file on Linux). Then poof, you have an extension module. Note that you'll need to figure out all the extra arguments to your C compiler like the header paths for Python and numpy. These all depend very heavily on not only your OS but also the particulars of how you've installed Python and SciPy on it.
If this all sounds a bit scary, see if the .pyx file is simple enough that you can use pyximport to handle all the messy compilation for you. If external libraries are needed, you'll probably need to construct a .pyxbld file.
Cython is a compiler: it converts a .pyx into a .c, which you then build to a .so or .pyd. Take a look at the cython docs on compilation. You will probably want to use pyximport module technique if you want to modify the code and experiment, and then use a setup.py when done and you need a final version of your Cython .pyx module.
I want to make "common" script which I will use in all my sconscripts
This script must use some scons functions like Object() or SharedObject()
Is there any scons file that i can import or maybe another useful hack.
Im new to python and scons.
I have done exactly what you are explaining. If your SConstruct and/or SConscript scripts simply import your common Python code, then there is nothing special you have to do, except import the appropriate SCons modules in your Python code.
If, on the other hand, you have a Python script, from which you want to invoke SCons (as opposed to launching scons from the command line) then much more effort will be needed. I originally looked into doing this, but later decided it wasnt worth the effort.