I'm starting the study of Python/C API and I make the first code to test some functions, I write this:
file: test.c
#include "Python.h"
int main() {
PyObject* none = Py_BuildValue("");
}
I compile with command:
gcc -I/usr/include/python2.7 test.c
I've the error undefined reference to `Py_BuildValue'
After I run:
gcc -I/usr/include/python2.7 --shared -fPIC hashmem.c
this compile without errors, but when I run the compiled file I've a
Segmentation fault (core dumped)
How do I set the gcc parameters?
I've ubuntu 12.04, python 2.7.3, gcc 4.6.3 and I installed python-dev.
Thanks.
In the comments #Pablo has provided the solution
gcc -I/usr/include/python2.7 test.c -lpython2.7
I forgot to link the python library with the "-l" parameter.
-llibrary
-l library
Search the library named library when linking. (The second alternative with the library as a separate argument is only for POSIX) compliance and is not recommended.)It makes a difference where in the command you write this option; the linker searches and processes libraries and object files in the order they are specified. Thus, foo.o -lz bar.o' searches libraryz' after file foo.o but before bar.o. If bar.o refers to functions in z', those functions may not be loaded.The linker searches a standard list of directories for the library, which is actually a file named liblibrary.a. The linker then uses this file as if it had been specified precisely by name.The directories searched include several standard system directories plus any that you specify with -L.Normally the files found this way are library files—archive files whose members are object files. The linker handles an archive file by scanning through it for members which define symbols that have so far been referenced but not defined. But if the file that is found is an ordinary object file, it is linked in the usual fashion. The only difference between using an -l option and specifying a file name is that - l surrounds library withlib' and `.a' and searches several directories.
Parameter description source
Related
I have a pure Python script that I would like to distribute to systems with unkown Python configuration. Therefore, I would like to compile the Python code to a stand-alone executable.
I run cython --embed ./foo.py without problems giving foo.c. Then, I run
gcc $(python3-config --cflags) $(python3-config --ldflags) ./foo.c
where python3-config --cflags gives
-I/usr/include/python3.5m -I/usr/include/python3.5m -Wno-unused-result -Wsign-compare -g -fdebug-prefix-map=/build/python3.5-MLq5fN/python3.5-3.5.3=. -fstack-protector-strong -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
and python3-config --ldflags gives
-L/usr/lib/python3.5/config-3.5m-x86_64-linux-gnu -L/usr/lib -lpython3.5m -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions
This way I obtain a dynamically linked executable that runs without a problem. ldd a.out yields
linux-vdso.so.1 (0x00007ffcd57fd000)
libpython3.5m.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0 (0x00007fda76823000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fda76603000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fda763fb000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fda761f3000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fda75eeb000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fda75b4b000)
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fda7591b000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fda756fb000)
/lib64/ld-linux-x86-64.so.2 (0x00007fda77103000)
Now, I try to add the option -static to gcc, but this results in an error:
/usr/bin/ld: dynamic STT_GNU_IFUNC symbol `strcmp' with pointer equality in `/usr/lib/gcc/x86_64-linux-gnu/6/../../../x86_64-linux-gnu/libc.a(strcmp.o)' can not be used when making an executable; recompile with -fPIE and relink with -pie
collect2: error: ld returned 1 exit status
I checked that all shared libraries given by ldd are also installed as static libraries.
So, is this some incompatibility with the options given by python3-config?
The experienced problems are obviously from the linker (gcc started a linker under the hood, to see it - just start gcc with -v - in verbose mode). So let's start with a short reminder how the linkage process works:
The linker keeps the names of all symbols it needs to resolve. In the beginning it is only the symbol main. What happens, when linker inspects a library?
If it is a static library, the linker looks at every object file in this library, and if this object files defines some looked for symbols, the whole object file is included (which means some symbols becomes resolved, but some further new unresolved symbols can be added). Linker might need to pass multiple times over a static library.
If it is a shared library, it is viewed by the linker as a library consisting out of a single huge object file (after all, we have to load this library at the run time and don't have to pass multiple times over and over to prune unused symbols): If there is at least one needed symbol the whole library is "linked" (not really the linkage happens at the run-time, this is a kind of a dry-run), if not - the whole library is discarded and never looked again at.
For example if you link with:
gcc -L/path -lpython3.x <other libs> foo.o
you will get a problem, no matter whether python3.x is a shared or a static lib: when the linker sees it, it looks only for the symbol main, but this symbol is not defined in the python-lib, so it the python-lib is discarded and never looked again at. Only when the linker sees the object-file foo.o, it realizes, that the whole Python-Symbols are needed, but now it is already too late.
There is a simple rule to handle this problem: put the object files first! That means:
gcc -L/path foo.o -lpython3.x <other libs>
Now the linker knows what it needs from the python-lib, when it first sees it.
There are other ways to achieve a similar result.
A) Let the linker to reiterate a group of archives as long as at least one new symbol definition was added per sweep:
gcc -L/path --Wl,-start-group -lpython3.x <other libs> foo.o -Wl,-end-group
Linker-options -Wl,-start-group and -Wl,-end-group says to linker iterate more than once over this group of archives, so the linker has a second chance (or more) to include symbols. This option can lead to longer linkage time.
B) Switching on the option --no-as-needed will lead to a shared library (and only shared library) being linked in, no matter whether in this library defined symbols are needed or not.
gcc -L/path -Wl,-no-as-needed -lpython3.x -Wl,-as-needed <other libs> foo.o
Actually, the default ld-behavior is --no-as-needed, but the gcc-frontend calls ld with option --as-needed, so we can restore the behavior by adding -no-as-needed prior to the python-library and then switch it off again.
Now to your problem of statical linking. I don't think it is advisable to use static versions of all standard libraries (all above glibc), what you should probably do is to link only the python-library statically.
The rules of the linkage are simple: per default the linker tries to open a shared version of the library first and than the static version. I.e. for the library libmylib and paths A and B, i.e.
-L/A -L/B lmylib
it tries to open libraries in the following order:
A/libmylib.so
A/libmylib.a
B/libmylib.so
B/libmylib.a
Thus if the folder A has only a static version, so this static version is used (no matter whether there is a shared version in folder B).
Because it is quite opaque which library is really used - it depends on the setup of your system, usually one would switch on the logging of the linker via -Wl,-verbose to trouble-shoot.
By using the option -Bstatic one can enforce the usage of the static version of a library:
gcc foo.o -L/path -Wl,-Bstatic -lpython3.x -Wl,-Bdynamic <other libs> -Wl,-verbose -o foo
Notable thing:
foo.o is linked before the libraries.
switch the static-mode off, directly after the python-library, so other libraries are linked dynamically.
And now:
gcc <cflags> L/paths foo.c -Wl,-Bstatic -lpython3.X -Wl,-Bdynamic <other libs> -o foo -Wl,-verbose
...
attempt to open path/libpython3.6m.a succeeded
...
ldd foo shows no dependency on python-lib
./foo
It works!
And yes, if you link against static glibc (I don't recommend), you will need to delete -Xlinker -export-dynamic from the command line.
The executable compiled without -Xlinker -export-dynamic will not be able to load some of c-extension which depend on this property of the executable to which they are loaded with ldopen.
Possible issues due to implicit -pie option.
Recent versions of gcc build with pie-option per default. Often/sometimes, older python versions where build with an older gcc-version, thus python-config --cflags would miss the now necessary -no-pie, as it was not needed back then. In this case the linker will produce an error message like:
relocation R_X86_64_32S against symbol `XXXXX' can not be used when
making a PIE object; recompile with -fPIC
In this case, -no-pie option should be added to <cflags>.
Is it possible (and how) to use MinGW-w64 for building of C-extensions for Python or embeding Python on Windows?
Let's take as example the following cython-extension foo.pyx:
print("foo loaded")
from which the C-code can be generated either via cython -3 foo.pyx or cython -3 --embed foo.pyx if interpreter should be embedded.
While mingw-w64-compiler is not really supported (the only supported windows compiler is MSVC), it can be used to create C-extensions or to embed Python. There are however no guarantee, this won't break in the future versions.
distutils does not support mingw-w64, so there is no gain in setting up a setup.py-file - the steps must be performed manually.
First we need some information usually provided by distutils:
Headers: We need the path to the Python includes. For a way to find them see this SO-post.
DLL: mingw-w64's linker works differently than MSVC's: python-dll and not python-lib is needed. So we need the path to the pythonXY.dll which is usually next the the python.exe.
Once the C-code is created/generated, the extension can be build via
x86_64-w64-mingw32-gcc -shared foo.c -DMS_WIN64 -O2 <other_options> -I <path_to_python_include> -L <path_to_python_dll> -lpython37 -o foo.pyd
The important details are:
it is probably Ok to use only use -O2 for optimization and leave <other_options> empty-
It is important to define MS_WIN64-macro (e.g. via -DMS_WIN64). In order to build for x64 on windows it must be set, but it works out of the box only for MSVC (defining _WIN64 could have slightly different outcomes):
#ifdef _WIN64
#define MS_WIN64
#endif
if it is not done, at least for files generated by Cython the following error message will be generated by the compiler:
error: enumerator value for ‘__pyx_check_sizeof_voidp’ is not an integer constant
201 | enum { __pyx_check_sizeof_voidp = 1 / (int)(SIZEOF_VOID_P == sizeof(void*)) };
pyd is just a dll in disguise, thus we need the -shared option, which means a dynamic library (i.e. shared-object in Linux-world) will be created.
It is important, that the python-library (pythonXY) should be the dll itself and not the lib (see this SO-post). Thua we use the path to pythonXY.dll (in my case python37) and not pythonXY.lib, as it would be the case for MSVC.
One probably should add the proper suffix to the resulting pyd-file, I use the old convention for simplicity here.
Embeded Python:
In this case an executable should be build (e.g. the C-file is generated by Cython with --embed option: cython -3 --embed foo.pyx) and thus the command line looks as follows:
x86_64-w64-mingw32-gcc foo.c -DMS_WIN64 -O2 <other_options> -I <path_to_python_include> -L <path_to_python_dll> -lpython37 -o foo.exe -municode
There are two important differences:
-shared should no longer be used, as the result is no longer a dynamic library (that is what *.pyd-file is after all) but an executable.
-municode is needed, because for Windows, Cython defines int wmain(int argc, wchar_t **argv) instead of int main(int argc, char** argv). Without this option, an error message like
/build/mingw-w64-_1w3Xm/mingw-w64-4.0.4/mingw-w64-crt/crt/crt0_c.c:18: undefined reference to 'WinMain'
collect2: error: ld returned 1 exit status
would appear (see this SO-post for more information).
Note: for the resulting executable to run, a whole python-distribution (and not only the dll) is needed (see also this SO-post), otherwise the resulting executable will abort with error (either the python dll wasn't found or the python installation or the site packages - depending on the configuration of the machine on which the exe has to run).
mingw-w64 can also be used on Linux for cross-compilation for Windows, see this SO-post.
Is it possible (and how) to use MinGW-w64 for building of C-extensions for Python or embeding Python on Windows?
Let's take as example the following cython-extension foo.pyx:
print("foo loaded")
from which the C-code can be generated either via cython -3 foo.pyx or cython -3 --embed foo.pyx if interpreter should be embedded.
While mingw-w64-compiler is not really supported (the only supported windows compiler is MSVC), it can be used to create C-extensions or to embed Python. There are however no guarantee, this won't break in the future versions.
distutils does not support mingw-w64, so there is no gain in setting up a setup.py-file - the steps must be performed manually.
First we need some information usually provided by distutils:
Headers: We need the path to the Python includes. For a way to find them see this SO-post.
DLL: mingw-w64's linker works differently than MSVC's: python-dll and not python-lib is needed. So we need the path to the pythonXY.dll which is usually next the the python.exe.
Once the C-code is created/generated, the extension can be build via
x86_64-w64-mingw32-gcc -shared foo.c -DMS_WIN64 -O2 <other_options> -I <path_to_python_include> -L <path_to_python_dll> -lpython37 -o foo.pyd
The important details are:
it is probably Ok to use only use -O2 for optimization and leave <other_options> empty-
It is important to define MS_WIN64-macro (e.g. via -DMS_WIN64). In order to build for x64 on windows it must be set, but it works out of the box only for MSVC (defining _WIN64 could have slightly different outcomes):
#ifdef _WIN64
#define MS_WIN64
#endif
if it is not done, at least for files generated by Cython the following error message will be generated by the compiler:
error: enumerator value for ‘__pyx_check_sizeof_voidp’ is not an integer constant
201 | enum { __pyx_check_sizeof_voidp = 1 / (int)(SIZEOF_VOID_P == sizeof(void*)) };
pyd is just a dll in disguise, thus we need the -shared option, which means a dynamic library (i.e. shared-object in Linux-world) will be created.
It is important, that the python-library (pythonXY) should be the dll itself and not the lib (see this SO-post). Thua we use the path to pythonXY.dll (in my case python37) and not pythonXY.lib, as it would be the case for MSVC.
One probably should add the proper suffix to the resulting pyd-file, I use the old convention for simplicity here.
Embeded Python:
In this case an executable should be build (e.g. the C-file is generated by Cython with --embed option: cython -3 --embed foo.pyx) and thus the command line looks as follows:
x86_64-w64-mingw32-gcc foo.c -DMS_WIN64 -O2 <other_options> -I <path_to_python_include> -L <path_to_python_dll> -lpython37 -o foo.exe -municode
There are two important differences:
-shared should no longer be used, as the result is no longer a dynamic library (that is what *.pyd-file is after all) but an executable.
-municode is needed, because for Windows, Cython defines int wmain(int argc, wchar_t **argv) instead of int main(int argc, char** argv). Without this option, an error message like
/build/mingw-w64-_1w3Xm/mingw-w64-4.0.4/mingw-w64-crt/crt/crt0_c.c:18: undefined reference to 'WinMain'
collect2: error: ld returned 1 exit status
would appear (see this SO-post for more information).
Note: for the resulting executable to run, a whole python-distribution (and not only the dll) is needed (see also this SO-post), otherwise the resulting executable will abort with error (either the python dll wasn't found or the python installation or the site packages - depending on the configuration of the machine on which the exe has to run).
mingw-w64 can also be used on Linux for cross-compilation for Windows, see this SO-post.
The scikit-build distribution provides usage examples of FindF2PY and UseF2PY, but they are incomplete, only providing a partial CMakeLists.txt file without the other required files. Based on the documentation I have not been able to make something that builds.
Following the examples in the scikit-build documentation, I created the following files:
CMakeLists.txt:
cmake_minimum_required(VERSION 3.10.2)
project(skbuild_test)
enable_language(Fortran)
find_package(F2PY REQUIRED)
add_f2py_target(f2py_test f2py_test.f90)
add_library(f2py_test MODULE f2py_test.f90)
install(TARGETS f2py_test LIBRARY DESTINATION f2py_test)
setup.py:
import setuptools
from skbuild import setup
requires=['numpy']
setup(
name="skbuild-test",
version='0.0.1',
description='Performs line integrals through SAMI3 grids',
author='John Haiducek',
requires=requires,
packages=['f2py_test']
)
f2py_test.f90:
module mod_f2py_test
implicit none
contains
subroutine f2py_test(a,b,c)
real(kind=8), intent(in)::a,b
real(kind=8), intent(out)::c
end subroutine f2py_test
end module mod_f2py_test
In addition, I created a directory f2py_test containing an empty init.py.
The output from python setup.py develop shows that scikit-build invokes CMake and compiles my Fortran code. However, it fails to find Python.h while compiling the f2py wrapper code:
[2/7] Building C object CMakeFiles/_f2...kages/numpy/f2py/src/fortranobject.c.o
FAILED: CMakeFiles/_f2py_runtime_library.dir/venv/lib/python3.8/site-packages/numpy/f2py/src/fortranobject.c.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -O3 -DNDEBUG -arch x86_64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -mmacosx-version-min=10.14 -MD -MT CMakeFiles/_f2py_runtime_library.dir/venv/lib/python3.8/site-packages/numpy/f2py/src/fortranobject.c.o -MF CMakeFiles/_f2py_runtime_library.dir/venv/lib/python3.8/site-packages/numpy/f2py/src/fortranobject.c.o.d -o CMakeFiles/_f2py_runtime_library.dir/venv/lib/python3.8/site-packages/numpy/f2py/src/fortranobject.c.o -c ../../../venv/lib/python3.8/site-packages/numpy/f2py/src/fortranobject.c
In file included from ../../../venv/lib/python3.8/site-packages/numpy/f2py/src/fortranobject.c:2:
../../../venv/lib/python3.8/site-packages/numpy/f2py/src/fortranobject.h:7:10: fatal error: 'Python.h' file not found
#include "Python.h"
^~~~~~~~~~
1 error generated.
First caveat, there may be a better way to do this, as I just figured out how to make scikit-build work by stumbling on your post and looking at documentation. Second caveat, I'm also learning cmake. So, there may be a better way.
There are a couple of things that you'll need to make your example here work. The big one is the second argument of add_f2py_target() isn't the source file. It is either the name of a pre-generated *.pyf, or to let f2py generate one, provide it an argument without the *.pyf extension. The other is adding the include directories for various components.
I made your example work with the following CMakeLists.txt:
cmake_minimum_required(VERSION 3.10.2)
project(skbuild_test)
enable_language(Fortran)
find_package(F2PY REQUIRED)
find_package(PythonLibs REQUIRED)
find_package(Python3 REQUIRED COMPONENTS NumPy)
#the following command either generates or points to an existing .pyf
#if provided an argument with .pyf extension, otherwise f2py generates one (not source code).
add_f2py_target(f2py_test f2py_test)
add_library(f2py_test MODULE f2py_test.f90)
include_directories(${PYTHON_INCLUDE_DIRS})
include_directories(${_Python3_NumPy_INCLUDE_DIR})
target_link_libraries(f2py_test ${PYTHON_LIBRARIES})
install(TARGETS f2py_test LIBRARY DESTINATION f2py_test)
So I have a few Python C extensions I have previously built for and used in 32 bit Python running in Win7. I have now however switched to 64 bit Python, and I am having issues building the C extension with MinGW-w64.
I made the changes to distutils as per this post, but I am getting some weird errors suggesting something is wrong:
$ python setup.py build
running build
running build_ext
building 'MyLib' extension
c:\MinGW64\bin\x86_64-w64-mingw32-gcc.exe -mdll -O -Wall -Ic:\Python27\lib\site-packages\numpy\core\include -Ic:\Python27\include -Ic:\Python27\PC -c MyLib.c -o build\temp.win-amd64-2.7\Release\mylib.o
MyLib.c: In function 'initMyLib':
MyLib.c:631:5: warning: implicit declaration of function 'Py_InitModule4_64' [-Wimplicit-function-declaration]
writing build\temp.win-amd64-2.7\Release\MyLib.def
c:\MinGW64\bin\x86_64-w64-mingw32-gcc.exe -shared -s build\temp.win-amd64-2.7\Release\mylib.o build\temp.win-amd64-2.7\Release\MyLib.def -Lc:\Python27\libs -Lc:\Python27\PCbuild\amd64 -lpython27 -o build\lib.win-amd64-2.7\MyLib.pyd
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x13d): undefined reference to `__imp_PyExc_ValueError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1275): undefined reference to `__imp_PyExc_ValueError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1eef): undefined reference to `__imp_PyExc_ImportError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1f38): undefined reference to `__imp_PyExc_AttributeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1f4d): undefined reference to `__imp_PyCObject_Type'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1f61): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1fc7): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1ffe): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x2042): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x206c): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x208a): more undefined references to `__imp_PyExc_RuntimeError' follow
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x20a7): undefined reference to `__imp_PyExc_ImportError'
collect2.exe: error: ld returned 1 exit status
error: command 'x86_64-w64-mingw32-gcc' failed with exit status 1
I have googled around quite a bit to find information, but it's not easy to find a definite answer. Could someone shed some light on this? What further changes should I do to be able to successfully build C extensions for 64 bit Python in Win7?
EDIT:
After some helpful pointers in cgohlke's comments below I managed to generate libpython27.a. However after following the advice on this post (2nd to last) I still had a the __imp_Py_InitModule4_64 error. After some serious Google-fu I managed to trip over this post telling me to rename the Py_InitModule4 line to Py_InitModule4_64. After that everything worked swimmingly.
This worked for me with Python 3.3 :
create static python lib from dll
python dll is usually in C:/Windows/System32; in msys shell:
gendef.exe python33.dll
dlltool.exe --dllname python33.dll --def python33.def --output-lib libpython33.a
mv libpython33.a C:/Python33/libs
use swig to generate wrappers
e.g., swig -c++ -python myExtension.i
wrapper MUST be compiled with MS_WIN64, or your computer will crash when you import the class in Python
g++ -c myExtension.cpp -I/other/includes
g++ -DMS_WIN64 -c myExtension_wrap.cxx -IC:/Python33/include
shared library
g++ -shared -o _myExtension.pyd myExtension.o myExtension_wrap.o -lPython33 -lOtherSharedLibs -LC:/Python33/libs -LC:/path/to/other/shared/libs
make sure all shared libs (gdal, OtherSharedLibs) are in your PATH
(windows does not use LD_LIBRARY_PATH or PYTHONPATH)
in Python, just: import myExtension
voila!
I realize this is an old question, but it is still the top search result. Today, in 2019, I was able to do this:
https://github.com/PetterS/quickjs/commit/67bc2428b8c0716538b4583f4f2b0a2a5a49106c
In short:
Make sure a 64-bit version of mingw-w64 is in the PATH.
Monkey-patch distutils:
import distutils.cygwinccompiler
distutils.cygwinccompiler.get_msvcr = lambda: []
Some differences in the shell w.r.t. escaping.
extra_link_args = ["-Wl,-Bstatic", "-lpthread"] in order to link statically and not have extra runtime deps.
pipenv run python setup.py build -c mingw32 now works.
Here is a example code for VC++ Build Tools
https://github.com/starnight/python-c-extension/tree/master/00-HelloWorld
You could try:
python setup.py -c mingw32
However this is not work for me.
My Solution is:
install Anaconda 64bit python 3.6
install mingw64
add mingw64/bin to PATH
compile dll from c file by
gcc -c libmypy.c -IC:\Users\{user_name}\Anaconda3\pkgs\python-3.6.4-h6538335_1\include
gcc -shared -o libmypy.dll libmypy.o -LC:\Users\{user_name}\Anaconda3\pkgs\python-3.6.4-h6538335_1\libs -lPython36
load dll file in .py script
from ctypes import *
m = cdll.LoadLibrary(r"C:\{path_to_dll}\libmypy.dll")
print(m.hello())
I created a monkey-patch for setuptools to let you to build_ext with mingw64 on Windows easily. See https://github.com/imba-tjd/mingw64ccompiler
I used this thread to wade through learning how to make a C extension, and since most of what I learned is in it, I thought I'd put the final discovery here too, so that someone else can find it if they are looking.
I wasn't trying to compile something big, just the example in Hetland's Beginning Python. Here is what I did (the example C pgm is called palindrome.c). I'm using Anaconda with python 3.7 in it, and the TDM-GCC version of MinGW64. I put all of the tools used into my Path, and all of the paths needed in PYTHONPATH, and the ..\Anaconda3 directory into PYTHON_HOME. I still ended up using explicit paths on some things.
I created the libpython37.a library with gendef.exe and dlltool.exe as Mark said above, and put it in ..\Anaconda3\libs.
I followed the prescription in Hetland:
gcc -c palindrome.c
gcc -I$PYTHON_HOME -I$PYTHON_HOME/Include -c palindrome_wrap.c
The second failed, the compiler couldn't find Python.h, the following worked:
gcc -I[somedirectories]\Anaconda3\Include -c palindrome_wrap.c
I then did, as many have said, including Hetland 3rd ed.,
gcc -shared palindrome.o palindrome_wrap.o [somedirectories]/Anaconda3/libs/libpython37.a -o _palindrome.dll
This did not work. Even with the Load Library cswu used (which I found elsewhere, too).
So I gendef'd _palindrome.dll and couldn't find the function in it, "is_palindrome" in the exports. I went through some of the SWIG documentation, and declared the function both in the %{ %} section and below it, both extern, that finally got the function extern'd in palindrome_wrap.c as it should have been. But no export, so I went back into palindrome.c and redeclared the function as:
declspec(dllexport) extern int __stdcall is_palindrome(char* text)
and redeclared it in palindrome.i in both places as above with this signature.
Partial success! It got listed in the Export section when I gendef'd _palindrome.dll and I could do cswu's call using Load Library. But still not do what Hetland says and do
import _palindrome
in Python.
Going back to all the sources again, I could not figure this out. I finally started reading the SWIG documentation from the beginning leaving no stone unturned -- Searching through the manual doesn't produce the place found.
At the end of Introduction Sec. 2.7 Incorporating Into a Build System, under the sample Make process, it says:
"The above example will generate native build files such as makefiles, nmake files and Visual Studio projects which will invoke SWIG and compile the generated C++ files into _example.so (UNIX) or _example.pyd (Windows). For other target languages on Windows a dll, instead of a .pyd file, is usually generated."
And that's the answer to the last problem:
The compile step for the dll should read:
gcc -shared palindrome.o palindrome_wrap.o [somedirectories]/Anaconda3/libs/libpython37.a -o _palindrome.pyd
(I didn't go back and change out my declspec declarations so I don't know whether they were necessary, so they were still there too).
I got a file, _palindrome.pyd
Which if in the PYTHONPATH (mine was local) works, and one can then do
import _palindrome
from _palindrome import is_palindrome
and use the exported, properly wrapped and packaged C function, compiled with TDM-GCC, in python as promised. gcc, which is MinGW64 in a different installation, knows how to do the .pyd file. I diffed the dll and pyd since they were the same byte length. They are not the same at hundreds of points.
Hope this helps someone else.