I'm trying to compile some C++ code into a dll to import into Python for the first time. I want to be able to return a Numpy array from one of the functions, with an example line that looks like lNumpyArray = PyArray_SimpleNewFromData( 2, lDimensions, NPY_UINT8, (void*)lImage->GetDataPointer() );
At the start of the code I have included Python.h and arrayobject.h.
Using GCC (working on Windows) I have been able to compile the code to a .O file without errors. However, when trying to go to a dll, I'm getting a lot of errors like undefined reference to '__imp__Py_Dealloc'. From my limited understanding, it might be because I'm missing a library somewhere in the linker. Is there some other library I need to include for using Numpy arrays in C, or should I be looking elsewhere? The gcc command I've been using is included below.
gcc -Wall -shared Pipeline.cpp -I"C:/Python38/include" -I"C:/Program Files/Pleora Technologies Inc/eBUS SDK/Includes" -I "C:/Python38/Lib/site-packages/numpy/core/include" -L"C:/Python38/libs" -L"C:/Program Files/Pleora Technologies Inc/eBUS SDK/Libraries" -o lib.dll -lPvBuffer64 -lPvDevice64 -lPvStream64 -lPvAppUtils64 -lPvSystem64 -lSimpleImagingLib64 -lPvGenICam64 -lPvSerial64 -lPvBase64 -lPtUtilsLib64
I would avoid using the C Python interface to numpy and use a library link xtensor (found here).
I'm using some example code I found on the web to try to set up libboost so I can call into cpp routines with Python code. (I intend to use python to write my UI and cpp for my backend for this application) Boost seems simple enough to use, but it's currently not exposing any functionality.
#include <boost/python.hpp>
char const* greet()
{
return "hello, world";
}
BOOST_PYTHON_MODULE(hello_ext)
{
using namespace boost::python;
def("greet", greet);
}
I compile this using the line g++ -c hello.cpp -I/usr/include/python3.6/
(that last include is necessary because I'm on ubuntu, where g++ doesn't locate python correctly, and I'm too lazy to add it to my path)
import hello_ext
print(hello_ext.greet())
I run this using python3, and I get the following output
File "hello.py", line 1, in <module>
import hello_ext
ModuleNotFoundError: No module named 'hello_ext'
This implies to me that Boost is not properly exposing the C++ functionality I created a module for. What am I missing here? I've already tried exposing the functionality to python in a header file instead of in the cpp file, and that has the same result.
Also, if anyone looking at this post is having issues accessing functionality within their module, but it seems like the module is being exposed, make sure python doesn't already have a default module with the same name which would take precedence over your module.
According to boost::python docs your code should be compiled as a shared library to be used in python:
g++ hello.cpp -I /usr/include/python3.6 -lboost_python-py36 -shared -fPIC -o hello_ext.so
Note that the name of the shared library must be the same as the name of your python module. You also forgot to link your code with boost lib.
I'm trying to compile very simple file to python on windows and I'm having a bad time.
The .i file is testfile.i:
%module testfile
%include "stl.i"
%{
int get_num() {
return 3;
}
%}
int get_num() {
return 3;
}
The swig function:
{swig path}\swig.exe -c++ -python testfile.i
This works perfectly, I now got the testfile.py file and testfile_wrap.cxx file.
Now I understood that I need to compile this to library (.pyd on windows). I tried:
{gcc path}\gcc.exe -fPIC -shared testfile_wrap.cxx -o testfile_wrap.pyd -L. -LC:\Python27\libs\ -lpython27 -IC:\python27\include.
Here is the problem, I get alot of errors like this ones:
C:\Users\itay\AppData\Local\Temp\ccANsNeU.o:testfile_wrap.cxx:(.text+0xc00): undefined reference to `__imp_PyExc_MemoryError'
C:\Users\itay\AppData\Local\Temp\ccANsNeU.o:testfile_wrap.cxx:(.text+0xc13): undefined reference to `__imp_PyExc_IOError'
C:\Users\itay\AppData\Local\Temp\ccANsNeU.o:testfile_wrap.cxx:(.text+0xc26): undefined reference to `__imp_PyExc_RuntimeError'
C:\Users\itay\AppData\Local\Temp\ccANsNeU.o:testfile_wrap.cxx:(.text+0xc39): undefined reference to `__imp_PyExc_IndexError'
And it continues on and on.
What am I doing wrong?
Thank you for your help
Update:
I managed to call swig and compile/link using visual studio 2013 but I get the same error. I followed tutorials and it still does not work.
The solution: my python was 64 bit and it didn't work. Changed to python 32 bit and now it is working (python 2.7.10)
I'm trying to use the C library libnfc for NFC Devices on python (http://nfc-tools.org/). With C I can run the example program and it works fine. I downloaded the package (https://code.google.com/p/pynfc/), which should allow to use the library on python and run the command "sudo python setup.py build_ext", like it is told in the README, but I get the following error:
running build_ext
building '_nfc' extension
swigging nfc.i to nfc_wrap.c
swig -python -I/usr/include -module nfc -interface _nfc -O -o nfc_wrap.c nfc.i
nfc/nfc.h:1489: Error: Syntax error in input(3).
error: command 'swig' failed with exit status 1
This is the content of nfc.h:
typedef struct {
PyObject_HEAD
void *ptr; // <- line 1489
swig_type_info *ty;
int own;
PyObject *next;
#ifdef SWIGPYTHON_BUILTIN
PyObject *dict;
#endif
} SwigPyObject;
I'm using Linux Mint 15.
Thanks to everyone who takes the time to read this!
SWIG is a software development tool that connects programs written in C and C++ with a variety of high-level programming languages.
So, just try:
apt-get install swig
So I have a few Python C extensions I have previously built for and used in 32 bit Python running in Win7. I have now however switched to 64 bit Python, and I am having issues building the C extension with MinGW-w64.
I made the changes to distutils as per this post, but I am getting some weird errors suggesting something is wrong:
$ python setup.py build
running build
running build_ext
building 'MyLib' extension
c:\MinGW64\bin\x86_64-w64-mingw32-gcc.exe -mdll -O -Wall -Ic:\Python27\lib\site-packages\numpy\core\include -Ic:\Python27\include -Ic:\Python27\PC -c MyLib.c -o build\temp.win-amd64-2.7\Release\mylib.o
MyLib.c: In function 'initMyLib':
MyLib.c:631:5: warning: implicit declaration of function 'Py_InitModule4_64' [-Wimplicit-function-declaration]
writing build\temp.win-amd64-2.7\Release\MyLib.def
c:\MinGW64\bin\x86_64-w64-mingw32-gcc.exe -shared -s build\temp.win-amd64-2.7\Release\mylib.o build\temp.win-amd64-2.7\Release\MyLib.def -Lc:\Python27\libs -Lc:\Python27\PCbuild\amd64 -lpython27 -o build\lib.win-amd64-2.7\MyLib.pyd
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x13d): undefined reference to `__imp_PyExc_ValueError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1275): undefined reference to `__imp_PyExc_ValueError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1eef): undefined reference to `__imp_PyExc_ImportError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1f38): undefined reference to `__imp_PyExc_AttributeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1f4d): undefined reference to `__imp_PyCObject_Type'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1f61): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1fc7): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1ffe): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x2042): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x206c): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x208a): more undefined references to `__imp_PyExc_RuntimeError' follow
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x20a7): undefined reference to `__imp_PyExc_ImportError'
collect2.exe: error: ld returned 1 exit status
error: command 'x86_64-w64-mingw32-gcc' failed with exit status 1
I have googled around quite a bit to find information, but it's not easy to find a definite answer. Could someone shed some light on this? What further changes should I do to be able to successfully build C extensions for 64 bit Python in Win7?
EDIT:
After some helpful pointers in cgohlke's comments below I managed to generate libpython27.a. However after following the advice on this post (2nd to last) I still had a the __imp_Py_InitModule4_64 error. After some serious Google-fu I managed to trip over this post telling me to rename the Py_InitModule4 line to Py_InitModule4_64. After that everything worked swimmingly.
This worked for me with Python 3.3 :
create static python lib from dll
python dll is usually in C:/Windows/System32; in msys shell:
gendef.exe python33.dll
dlltool.exe --dllname python33.dll --def python33.def --output-lib libpython33.a
mv libpython33.a C:/Python33/libs
use swig to generate wrappers
e.g., swig -c++ -python myExtension.i
wrapper MUST be compiled with MS_WIN64, or your computer will crash when you import the class in Python
g++ -c myExtension.cpp -I/other/includes
g++ -DMS_WIN64 -c myExtension_wrap.cxx -IC:/Python33/include
shared library
g++ -shared -o _myExtension.pyd myExtension.o myExtension_wrap.o -lPython33 -lOtherSharedLibs -LC:/Python33/libs -LC:/path/to/other/shared/libs
make sure all shared libs (gdal, OtherSharedLibs) are in your PATH
(windows does not use LD_LIBRARY_PATH or PYTHONPATH)
in Python, just: import myExtension
voila!
I realize this is an old question, but it is still the top search result. Today, in 2019, I was able to do this:
https://github.com/PetterS/quickjs/commit/67bc2428b8c0716538b4583f4f2b0a2a5a49106c
In short:
Make sure a 64-bit version of mingw-w64 is in the PATH.
Monkey-patch distutils:
import distutils.cygwinccompiler
distutils.cygwinccompiler.get_msvcr = lambda: []
Some differences in the shell w.r.t. escaping.
extra_link_args = ["-Wl,-Bstatic", "-lpthread"] in order to link statically and not have extra runtime deps.
pipenv run python setup.py build -c mingw32 now works.
Here is a example code for VC++ Build Tools
https://github.com/starnight/python-c-extension/tree/master/00-HelloWorld
You could try:
python setup.py -c mingw32
However this is not work for me.
My Solution is:
install Anaconda 64bit python 3.6
install mingw64
add mingw64/bin to PATH
compile dll from c file by
gcc -c libmypy.c -IC:\Users\{user_name}\Anaconda3\pkgs\python-3.6.4-h6538335_1\include
gcc -shared -o libmypy.dll libmypy.o -LC:\Users\{user_name}\Anaconda3\pkgs\python-3.6.4-h6538335_1\libs -lPython36
load dll file in .py script
from ctypes import *
m = cdll.LoadLibrary(r"C:\{path_to_dll}\libmypy.dll")
print(m.hello())
I created a monkey-patch for setuptools to let you to build_ext with mingw64 on Windows easily. See https://github.com/imba-tjd/mingw64ccompiler
I used this thread to wade through learning how to make a C extension, and since most of what I learned is in it, I thought I'd put the final discovery here too, so that someone else can find it if they are looking.
I wasn't trying to compile something big, just the example in Hetland's Beginning Python. Here is what I did (the example C pgm is called palindrome.c). I'm using Anaconda with python 3.7 in it, and the TDM-GCC version of MinGW64. I put all of the tools used into my Path, and all of the paths needed in PYTHONPATH, and the ..\Anaconda3 directory into PYTHON_HOME. I still ended up using explicit paths on some things.
I created the libpython37.a library with gendef.exe and dlltool.exe as Mark said above, and put it in ..\Anaconda3\libs.
I followed the prescription in Hetland:
gcc -c palindrome.c
gcc -I$PYTHON_HOME -I$PYTHON_HOME/Include -c palindrome_wrap.c
The second failed, the compiler couldn't find Python.h, the following worked:
gcc -I[somedirectories]\Anaconda3\Include -c palindrome_wrap.c
I then did, as many have said, including Hetland 3rd ed.,
gcc -shared palindrome.o palindrome_wrap.o [somedirectories]/Anaconda3/libs/libpython37.a -o _palindrome.dll
This did not work. Even with the Load Library cswu used (which I found elsewhere, too).
So I gendef'd _palindrome.dll and couldn't find the function in it, "is_palindrome" in the exports. I went through some of the SWIG documentation, and declared the function both in the %{ %} section and below it, both extern, that finally got the function extern'd in palindrome_wrap.c as it should have been. But no export, so I went back into palindrome.c and redeclared the function as:
declspec(dllexport) extern int __stdcall is_palindrome(char* text)
and redeclared it in palindrome.i in both places as above with this signature.
Partial success! It got listed in the Export section when I gendef'd _palindrome.dll and I could do cswu's call using Load Library. But still not do what Hetland says and do
import _palindrome
in Python.
Going back to all the sources again, I could not figure this out. I finally started reading the SWIG documentation from the beginning leaving no stone unturned -- Searching through the manual doesn't produce the place found.
At the end of Introduction Sec. 2.7 Incorporating Into a Build System, under the sample Make process, it says:
"The above example will generate native build files such as makefiles, nmake files and Visual Studio projects which will invoke SWIG and compile the generated C++ files into _example.so (UNIX) or _example.pyd (Windows). For other target languages on Windows a dll, instead of a .pyd file, is usually generated."
And that's the answer to the last problem:
The compile step for the dll should read:
gcc -shared palindrome.o palindrome_wrap.o [somedirectories]/Anaconda3/libs/libpython37.a -o _palindrome.pyd
(I didn't go back and change out my declspec declarations so I don't know whether they were necessary, so they were still there too).
I got a file, _palindrome.pyd
Which if in the PYTHONPATH (mine was local) works, and one can then do
import _palindrome
from _palindrome import is_palindrome
and use the exported, properly wrapped and packaged C function, compiled with TDM-GCC, in python as promised. gcc, which is MinGW64 in a different installation, knows how to do the .pyd file. I diffed the dll and pyd since they were the same byte length. They are not the same at hundreds of points.
Hope this helps someone else.