problem with soaplib (lxml) with apache2 + mod_wsgi - python

when I launch my application with apache2+modwsgi
I catch
Exception Type: ImportError
Exception Value: DLL load failed: The specified module could not be found.
in line
from lxml import etree
with Django dev server all works fine
Visual C++ Redistributable 2008 installed
Dependency walker told that msvcrt90.dll is missed
but there is same situation with cx_Oracle, but cx_Oracle's dll loads correct
any ideas?
windows 2003 server 64bit and windows XP sp3 32bit
python 2.7 32 bit
cx_Oracle 5.0.4 32bit
UPD:
download libxml2-2.7.7 and libxslt-1.1.26
tried to build with setup.py build --compiler mingw32
Building lxml version 2.3.
Building with Cython 0.14.1.
ERROR: 'xslt-config' is not recognized as an internal or external command,
operable program or batch file.
** make sure the development packages of libxml2 and libxslt are installed **
Using build configuration of libxslt
running build
running build_py
running build_ext
skipping 'src/lxml\lxml.etree.c' Cython extension (up-to-date)
building 'lxml.etree' extension
C:\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -IC:\Python27\include -IC:\Python27\PC -c src/lxml\lxml.etree.c -o build\temp.win32-2.7\Release\src\lxml\lxml.et
ree.o -w
writing build\temp.win32-2.7\Release\src\lxml\etree.def
C:\MinGW\bin\gcc.exe -mno-cygwin -shared -s build\temp.win32-2.7\Release\src\lxml\lxml.etree.o build\temp.win32-2.7\Release\src\lxml\etree.def -LC:\Python27\lib
s -LC:\Python27\PCbuild -llibxslt -llibexslt -llibxml2 -liconv -lzlib -lWS2_32 -lpython27 -lmsvcr90 -o build\lib.win32-2.7\lxml\etree.pyd
build\temp.win32-2.7\Release\src\lxml\lxml.etree.o:lxml.etree.c:(.text+0xd11): undefined reference to `_imp__xmlFree'
build\temp.win32-2.7\Release\src\lxml\lxml.etree.o:lxml.etree.c:(.text+0xd24): undefined reference to `_imp__xmlFree'
build\temp.win32-2.7\Release\src\lxml\lxml.etree.o:lxml.etree.c:(.text+0x1ee92): undefined reference to `_imp__xmlFree'
build\temp.win32-2.7\Release\src\lxml\lxml.etree.o:lxml.etree.c:(.text+0x1eed6): undefined reference to `_imp__xmlFree'
build\temp.win32-2.7\Release\src\lxml\lxml.etree.o:lxml.etree.c:(.text+0x2159e): undefined reference to `_imp__xmlMalloc'
build\temp.win32-2.7\Release\src\lxml\lxml.etree.o:lxml.etree.c:(.text+0x2e741): undefined reference to `_imp__xmlFree'
build\temp.win32-2.7\Release\src\lxml\lxml.etree.o:lxml.etree.c:(.text+0x2e784): undefined reference to `_imp__xmlFree'
build\temp.win32-2.7\Release\src\lxml\lxml.etree.o:lxml.etree.c:(.text+0x3f157): undefined reference to `_imp__xmlFree'
build\temp.win32-2.7\Release\src\lxml\lxml.etree.o:lxml.etree.c:(.text+0x3f19a): undefined reference to `_imp__xmlFree'
build\temp.win32-2.7\Release\src\lxml\lxml.etree.o:lxml.etree.c:(.text+0x3f4ac): undefined reference to `_imp__xmlFree'
build\temp.win32-2.7\Release\src\lxml\lxml.etree.o:lxml.etree.c:(.text+0x3f4ef): more undefined references to `_imp__xmlFree' follow
build\temp.win32-2.7\Release\src\lxml\lxml.etree.o:lxml.etree.c:(.text+0xb1ad5): undefined reference to `xsltLibxsltVersion'
build\temp.win32-2.7\Release\src\lxml\lxml.etree.o:lxml.etree.c:(.text+0xb1b9a): undefined reference to `xsltDocDefaultLoader'
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
UPD2:
I understand why import cx_Oracle works fine: cx_Oracle.pyd contains "MSVCRT.dll" dependence etree.pyd doesn't have it

It is indeed because of 'msvcrt90.dll'. From somewhere in micro patch revisions of Python 2.6 they stopped building in automatic dependencies on the DLL for extension modules and relied on Python executable doing it. When embedded in other systems however you are then dependent on that executable linking to DLL and in the case of Apache it doesn't. The change in Python has therefore broken many systems which embed Python on Windows and the only solution is for every extension module to have their own dependencies on required DLLs which many don't. The psycopg2 extension was badly affected by this and they have change their builds to add the dependency back in themselves now. You might go searching about the problem as it occurred for psycopg2. One of the solutions was to rebuild extensions with MinGW compiler on Windows instead.

Related

Embeded python3.6 with MinGW in C++ fail on linking

Try to embed python3.6 in C++ on Win10 64bit system.
Python is installed by python-3.6.1-amd64.exe.
The CMakeList.txt is showing below
cmake_minimum_required(VERSION 3.8)
project(EmbedPython)
set(CMAKE_CXX_STANDARD 11)
set(SOURCE_FILES main.cpp)
find_package(PythonLibs REQUIRED)
include_directories(${PYTHON_INCLUDE_DIRS})
add_executable(EmbedPython ${SOURCE_FILES} ${PYTHON_INCLUDE_DIRS})
target_link_libraries(EmbedPython ${PYTHON_LIBRARIES})
And CMake found the Python package.
PYTHON_INCLUDE_DIR C:/Program Files (x86)/Python36/include
PYTHON_LIBRARY C:/Program Files (x86)/Python36/libs/python36.lib
Find a comment on github it's talking about the API problem.
# used to embed python script
find_package(PythonLibs 2.7 REQUIRED) # this has to be 2.7 because the 3.0 API requires argv to be wchar_t** rather than the default char**
include_directories(${PYTHON_INCLUDE_DIRS})
target_link_libraries( assignment ${PYTHON_LIBRARIES})
The errors below is what I'm facing.
[ 50%] Building CXX object CMakeFiles/EmbedPython.dir/main.cpp.obj
[100%] Linking CXX executable EmbedPython.exe
CMakeFiles\EmbedPython.dir/objects.a(main.cpp.obj): In function `main':
D:/Projects/EmbedPython/main.cpp:16: undefined reference to `_imp__Py_Initialize'
D:/Projects/EmbedPython/main.cpp:17: undefined reference to `_imp__PyUnicode_DecodeFSDefault'
D:/Projects/EmbedPython/main.cpp:20: undefined reference to `_imp__PyImport_Import'
D:/Projects/EmbedPython/main.cpp:24: undefined reference to `_imp__PyObject_GetAttrString'
D:/Projects/EmbedPython/main.cpp:27: undefined reference to `_imp__PyCallable_Check'
D:/Projects/EmbedPython/main.cpp:28: undefined reference to `_imp__PyTuple_New'
D:/Projects/EmbedPython/main.cpp:30: undefined reference to `_imp__PyLong_FromLong'
D:/Projects/EmbedPython/main.cpp:38: undefined reference to `_imp__PyTuple_SetItem'
D:/Projects/EmbedPython/main.cpp:40: undefined reference to `_imp__PyObject_CallObject'
D:/Projects/EmbedPython/main.cpp:43: undefined reference to `_imp__PyLong_AsLong'
D:/Projects/EmbedPython/main.cpp:49: undefined reference to `_imp__PyErr_Print'
D:/Projects/EmbedPython/main.cpp:55: undefined reference to `_imp__PyErr_Occurred'
D:/Projects/EmbedPython/main.cpp:56: undefined reference to `_imp__PyErr_Print'
D:/Projects/EmbedPython/main.cpp:63: undefined reference to `_imp__PyErr_Print'
D:/Projects/EmbedPython/main.cpp:67: undefined reference to `_imp__Py_FinalizeEx'
collect2.exe: error: ld returned 1 exit status
mingw32-make.exe[3]: *** [EmbedPython.exe] Error 1
mingw32-make.exe[2]: *** [CMakeFiles/EmbedPython.dir/all] Error 2
mingw32-make.exe[1]: *** [CMakeFiles/EmbedPython.dir/rule] Error 2
mingw32-make.exe: *** [EmbedPython] Error 2
CMakeFiles\EmbedPython.dir\build.make:96: recipe for target 'EmbedPython.exe' failed
CMakeFiles\Makefile2:66: recipe for target 'CMakeFiles/EmbedPython.dir/all' failed
CMakeFiles\Makefile2:78: recipe for target 'CMakeFiles/EmbedPython.dir/rule' failed
Makefile:117: recipe for target 'EmbedPython' failed
Not sure what to do to make the project run.Any help would be appreciated.
EDIT:
the result from make VERBOSE=1
D:\Projects\EmbedPython\cmake-build-debug>make VERBOSE=1
"C:\Program Files\JetBrains\CLion 172.2273.4\bin\cmake\bin\cmake.exe" -HD:\Projects\EmbedPython -BD:\Projects\EmbedPython\cmake-build-debug --check-build-system CMakeFiles\Makefile.cmake 0
"C:\Program Files\JetBrains\CLion 172.2273.4\bin\cmake\bin\cmake.exe" -E cmake_progress_start D:\Projects\EmbedPython\cmake-build-debug\CMakeFiles D:\Projects\EmbedPython\cmake-build-debug\CMakeFiles\progress.marks
make -f CMakeFiles\Makefile2 all
make[1]: Entering directory 'D:/Projects/EmbedPython/cmake-build-debug'
make -f CMakeFiles\EmbedPython.dir\build.make CMakeFiles/EmbedPython.dir/depend
make[2]: Entering directory 'D:/Projects/EmbedPython/cmake-build-debug'
"C:\Program Files\JetBrains\CLion 172.2273.4\bin\cmake\bin\cmake.exe" -E cmake_depends "MinGW Makefiles" D:\Projects\EmbedPython D:\Projects\EmbedPython D:\Projects\EmbedPython\cmake-build-debug D:\Projects\EmbedPython\cmake-build-debug D:\Projects\EmbedPython\cmake-build-debug\CMakeFiles\EmbedPython.dir\DependInfo.cmake --color=
make[2]: Leaving directory 'D:/Projects/EmbedPython/cmake-build-debug'
make -f CMakeFiles\EmbedPython.dir\build.make CMakeFiles/EmbedPython.dir/build
make[2]: Entering directory 'D:/Projects/EmbedPython/cmake-build-debug'
[ 50%] Linking CXX executable EmbedPython.exe
"C:\Program Files\JetBrains\CLion 172.2273.4\bin\cmake\bin\cmake.exe" -E cmake_link_script CMakeFiles\EmbedPython.dir\link.txt --verbose=1
"C:\Program Files\JetBrains\CLion 172.2273.4\bin\cmake\bin\cmake.exe" -E remove -f CMakeFiles\EmbedPython.dir/objects.a
C:\PROGRA~2\MINGW-~1\I686-6~1.0-P\mingw32\bin\ar.exe cr CMakeFiles\EmbedPython.dir/objects.a #CMakeFiles\EmbedPython.dir\objects1.rsp
C:\PROGRA~2\MINGW-~1\I686-6~1.0-P\mingw32\bin\G__~1.EXE -g -Wl,--whole-archive CMakeFiles\EmbedPython.dir/objects.a -Wl,--no-whole-archive -o EmbedPython.exe -Wl,--out-implib,libEmbedPython.dll.a -Wl,--major-image-version,0,--minor-image-version,0 #CMakeFiles\EmbedPython.dir\linklibs.rsp
CMakeFiles\EmbedPython.dir/objects.a(main.cpp.obj): In function `main':
D:/Projects/EmbedPython/main.cpp:16: undefined reference to `_imp__Py_Initialize'
D:/Projects/EmbedPython/main.cpp:17: undefined reference to `_imp__PyUnicode_DecodeFSDefault'
D:/Projects/EmbedPython/main.cpp:20: undefined reference to `_imp__PyImport_Import'
D:/Projects/EmbedPython/main.cpp:24: undefined reference to `_imp__PyObject_GetAttrString'
D:/Projects/EmbedPython/main.cpp:27: undefined reference to `_imp__PyCallable_Check'
D:/Projects/EmbedPython/main.cpp:28: undefined reference to `_imp__PyTuple_New'
D:/Projects/EmbedPython/main.cpp:30: undefined reference to `_imp__PyLong_FromLong'
D:/Projects/EmbedPython/main.cpp:38: undefined reference to `_imp__PyTuple_SetItem'
D:/Projects/EmbedPython/main.cpp:40: undefined reference to `_imp__PyObject_CallObject'
D:/Projects/EmbedPython/main.cpp:43: undefined reference to `_imp__PyLong_AsLong'
D:/Projects/EmbedPython/main.cpp:49: undefined reference to `_imp__PyErr_Print'
D:/Projects/EmbedPython/main.cpp:55: undefined reference to `_imp__PyErr_Occurred'
D:/Projects/EmbedPython/main.cpp:56: undefined reference to `_imp__PyErr_Print'
D:/Projects/EmbedPython/main.cpp:63: undefined reference to `_imp__PyErr_Print'
D:/Projects/EmbedPython/main.cpp:67: undefined reference to `_imp__Py_FinalizeEx'
collect2.exe: error: ld returned 1 exit status
CMakeFiles\EmbedPython.dir\build.make:97: recipe for target 'EmbedPython.exe' failed
make[2]: *** [EmbedPython.exe] Error 1
make[2]: Leaving directory 'D:/Projects/EmbedPython/cmake-build-debug'
CMakeFiles\Makefile2:66: recipe for target 'CMakeFiles/EmbedPython.dir/all' failed
make[1]: *** [CMakeFiles/EmbedPython.dir/all] Error 2
make[1]: Leaving directory 'D:/Projects/EmbedPython/cmake-build-debug'
Makefile:82: recipe for target 'all' failed
make: *** [all] Error 2
I had the exactly same problem, just with Python 3.7 while trying to build the example from the Python manual.
My system is a 64-bit Windows, the installed Python is a 64-bit 3.7 (from Anaconda3), and the compiler is a 32-bit MinGW 5.3.0 (from Qt 5.11).
The simplest solution that worked for me is the following:
Download Windows x86 embeddable zip file from this page.
Put the content of that archive to a folder inside the project you're building.
Do not install anything from that archive and do not put it on the system PATH.
Add the above folder to the gcc library search path, using: -L"path/to/folder/"
Add the python library to the linker, using -l"python37" ( replace 37 with whatever version you download)
As noted here (this was a hard one to find!), add a definition, using -DPy_BUILD_CORE_BUILTIN=1 (It is also possible to write #define DPy_BUILD_CORE_BUILTIN 1 just before #include-ing the Python header, though I would not recommend it)
Provide the path to the Python headers, using -I"path/to/include". I used the one from Anaconda, e.g. -I"c:/Users/user/Anaconda3/include/". Any installed header will do, just make sure its from the same version (headers from an older version might also be used, but this may cause a rupture in the space-time continuum)
Make sure that #include <Python.h> is before any other include, as stated in the manual.
As a side note, the MinGW linker (ld) is perfectly content, for many years now, if you just give it the "dll" file. No "lib", nor "a", is needed if you don't have one and only perform dynamic linking. This is why I just told the linker to use python37.
It it actually smart enough to look for files like python37.lib, python37.dll & libpython37.a.
Also note that the "embedable zip" is just the bare minimum needed for, well, embedding Python.
And as a last comment, contrary to what is stated here, there is no problem whatsoever linking from GCC with a library compiled using MSVS (and vice versa), as long as the library uses the plain C interface, since that interface is pretty much the only thing that is compatible across different compilers for any given platform.
This is pretty much the exact reason why Python is using the C interface (and nothing else, not even C++) for extending & embedding - to allow code written in any language, using any tool set, to interface with Python.
I had the same problem.
probably the problem is because of using 32 bit MinGW and 64 bit Python. you should install 32 bit version of python
Hope it will be useful to you :-)

PythonXY: Error when compiling C Extension on Windows: bad reloc address 0x0 in section `.data' collect2.exe: error: ld returned 1 exit status

I face a problem when compiling pyxrr (http://carichte.github.io/pyxrr/) on a Windows 7 machine (64bit) when using the latest Version of PythonXY with mingw32.
In the past the Compilation worked without problems on similar machines.
The following error output is given:
run setup.py build -c mingw32
running build
running build_py
running build_ext
building 'pyxrr.xrr' extension
C:\MinGW32-xy\bin\gcc.exe -mdll -O -Wall -IC:\Python27\lib\site-packages\numpy\c
ore\include -IC:\Python27\include -IC:\Python27\PC -c pyxrr/xrr.c -o build\temp.
win32-2.7\Release\pyxrr\xrr.o -fopenmp
In file included from C:\Python27\lib\site-packages\numpy\core\include/numpy/nda
rraytypes.h:1761:0,
from C:\Python27\lib\site-packages\numpy\core\include/numpy/nda
rrayobject.h:17,
from C:\Python27\lib\site-packages\numpy\core\include/numpy/arr
ayobject.h:4,
from pyxrr/xrr.c:26:
C:\Python27\lib\site-packages\numpy\core\include/numpy/npy_1_7_deprecated_api.h:
12:9: note: #pragma message: C:\Python27\lib\site-packages\numpy\core\include/nu
mpy/npy_1_7_deprecated_api.h(12) : Warning Msg: Using deprecated NumPy API, disa
ble it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
#pragma message(_WARN___LOC__"Using deprecated NumPy API, disable it by " \
^
writing build\temp.win32-2.7\Release\pyxrr\xrr.def
C:\MinGW32-xy\bin\gcc.exe -shared -s build\temp.win32-2.7\Release\pyxrr\xrr.o bu
ild\temp.win32-2.7\Release\pyxrr\xrr.def -LC:\Python27\libs -LC:\Python27\PCbuil
d -lpython27 -lmsvcr90 -o build\lib.win32-2.7\pyxrr\xrr.pyd -fopenmp
build\temp.win32-2.7\Release\pyxrr\xrr.o:xrr.c:(.text+0x188): undefined referenc
e to `cexp'
build\temp.win32-2.7\Release\pyxrr\xrr.o:xrr.c:(.text+0x32c): undefined referenc
e to `cexp'
build\temp.win32-2.7\Release\pyxrr\xrr.o:xrr.c:(.text+0x42c): undefined referenc
e to `cexp'
build\temp.win32-2.7\Release\pyxrr\xrr.o:xrr.c:(.text+0xb4a): undefined referenc
e to `cexp'
build\temp.win32-2.7\Release\pyxrr\xrr.o:xrr.c:(.text+0xccf): undefined referenc
e to `cexp'
build\temp.win32-2.7\Release\pyxrr\xrr.o:xrr.c:(.text+0xda5): more undefined ref
erences to `cexp' follow
build\temp.win32-2.7\Release\pyxrr\xrr.o:xrr.c:(.text+0x100e): undefined referen
ce to `csqrt'
build\temp.win32-2.7\Release\pyxrr\xrr.o:xrr.c:(.text+0x1898): undefined referen
ce to `csqrt'
c:/mingw32-xy/bin/../lib/gcc/mingw32/4.8.1/../../../../mingw32/bin/ld.exe: build
\temp.win32-2.7\Release\pyxrr\xrr.o: bad reloc address 0x0 in section `.data'
collect2.exe: error: ld returned 1 exit status
An exception has occurred, use %tb to see the full traceback.
SystemExit: error: command 'gcc' failed with exit status 1
The same error occures if I do not use the multi processing option and libraries (--single-core flag). I was searching the web and trying different compiler options like -lm, -DKM_WIN32 etc. without effect. But python as well as mingw run as 32bit versions.
Apparently he doesn't find complex.h library functions which are actually standard.
I tried this on 2 different Win7 machines. Also, compiling using Anaconda works for the single-threaded version.
Since I don't really understand the problems, I hope someone can tell me where it lies.
Any hint is appreciated,
Thank you!

How can I build my C extensions with MinGW-w64 in Python?

So I have a few Python C extensions I have previously built for and used in 32 bit Python running in Win7. I have now however switched to 64 bit Python, and I am having issues building the C extension with MinGW-w64.
I made the changes to distutils as per this post, but I am getting some weird errors suggesting something is wrong:
$ python setup.py build
running build
running build_ext
building 'MyLib' extension
c:\MinGW64\bin\x86_64-w64-mingw32-gcc.exe -mdll -O -Wall -Ic:\Python27\lib\site-packages\numpy\core\include -Ic:\Python27\include -Ic:\Python27\PC -c MyLib.c -o build\temp.win-amd64-2.7\Release\mylib.o
MyLib.c: In function 'initMyLib':
MyLib.c:631:5: warning: implicit declaration of function 'Py_InitModule4_64' [-Wimplicit-function-declaration]
writing build\temp.win-amd64-2.7\Release\MyLib.def
c:\MinGW64\bin\x86_64-w64-mingw32-gcc.exe -shared -s build\temp.win-amd64-2.7\Release\mylib.o build\temp.win-amd64-2.7\Release\MyLib.def -Lc:\Python27\libs -Lc:\Python27\PCbuild\amd64 -lpython27 -o build\lib.win-amd64-2.7\MyLib.pyd
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x13d): undefined reference to `__imp_PyExc_ValueError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1275): undefined reference to `__imp_PyExc_ValueError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1eef): undefined reference to `__imp_PyExc_ImportError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1f38): undefined reference to `__imp_PyExc_AttributeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1f4d): undefined reference to `__imp_PyCObject_Type'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1f61): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1fc7): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x1ffe): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x2042): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x206c): undefined reference to `__imp_PyExc_RuntimeError'
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x208a): more undefined references to `__imp_PyExc_RuntimeError' follow
build\temp.win-amd64-2.7\Release\mylib.o:MyLib.c:(.text+0x20a7): undefined reference to `__imp_PyExc_ImportError'
collect2.exe: error: ld returned 1 exit status
error: command 'x86_64-w64-mingw32-gcc' failed with exit status 1
I have googled around quite a bit to find information, but it's not easy to find a definite answer. Could someone shed some light on this? What further changes should I do to be able to successfully build C extensions for 64 bit Python in Win7?
EDIT:
After some helpful pointers in cgohlke's comments below I managed to generate libpython27.a. However after following the advice on this post (2nd to last) I still had a the __imp_Py_InitModule4_64 error. After some serious Google-fu I managed to trip over this post telling me to rename the Py_InitModule4 line to Py_InitModule4_64. After that everything worked swimmingly.
This worked for me with Python 3.3 :
create static python lib from dll
python dll is usually in C:/Windows/System32; in msys shell:
gendef.exe python33.dll
dlltool.exe --dllname python33.dll --def python33.def --output-lib libpython33.a
mv libpython33.a C:/Python33/libs
use swig to generate wrappers
e.g., swig -c++ -python myExtension.i
wrapper MUST be compiled with MS_WIN64, or your computer will crash when you import the class in Python
g++ -c myExtension.cpp -I/other/includes
g++ -DMS_WIN64 -c myExtension_wrap.cxx -IC:/Python33/include
shared library
g++ -shared -o _myExtension.pyd myExtension.o myExtension_wrap.o -lPython33 -lOtherSharedLibs -LC:/Python33/libs -LC:/path/to/other/shared/libs
make sure all shared libs (gdal, OtherSharedLibs) are in your PATH
(windows does not use LD_LIBRARY_PATH or PYTHONPATH)
in Python, just: import myExtension
voila!
I realize this is an old question, but it is still the top search result. Today, in 2019, I was able to do this:
https://github.com/PetterS/quickjs/commit/67bc2428b8c0716538b4583f4f2b0a2a5a49106c
In short:
Make sure a 64-bit version of mingw-w64 is in the PATH.
Monkey-patch distutils:
import distutils.cygwinccompiler
distutils.cygwinccompiler.get_msvcr = lambda: []
Some differences in the shell w.r.t. escaping.
extra_link_args = ["-Wl,-Bstatic", "-lpthread"] in order to link statically and not have extra runtime deps.
pipenv run python setup.py build -c mingw32 now works.
Here is a example code for VC++ Build Tools
https://github.com/starnight/python-c-extension/tree/master/00-HelloWorld
You could try:
python setup.py -c mingw32
However this is not work for me.
My Solution is:
install Anaconda 64bit python 3.6
install mingw64
add mingw64/bin to PATH
compile dll from c file by
gcc -c libmypy.c -IC:\Users\{user_name}\Anaconda3\pkgs\python-3.6.4-h6538335_1\include
gcc -shared -o libmypy.dll libmypy.o -LC:\Users\{user_name}\Anaconda3\pkgs\python-3.6.4-h6538335_1\libs -lPython36
load dll file in .py script
from ctypes import *
m = cdll.LoadLibrary(r"C:\{path_to_dll}\libmypy.dll")
print(m.hello())
I created a monkey-patch for setuptools to let you to build_ext with mingw64 on Windows easily. See https://github.com/imba-tjd/mingw64ccompiler
I used this thread to wade through learning how to make a C extension, and since most of what I learned is in it, I thought I'd put the final discovery here too, so that someone else can find it if they are looking.
I wasn't trying to compile something big, just the example in Hetland's Beginning Python. Here is what I did (the example C pgm is called palindrome.c). I'm using Anaconda with python 3.7 in it, and the TDM-GCC version of MinGW64. I put all of the tools used into my Path, and all of the paths needed in PYTHONPATH, and the ..\Anaconda3 directory into PYTHON_HOME. I still ended up using explicit paths on some things.
I created the libpython37.a library with gendef.exe and dlltool.exe as Mark said above, and put it in ..\Anaconda3\libs.
I followed the prescription in Hetland:
gcc -c palindrome.c
gcc -I$PYTHON_HOME -I$PYTHON_HOME/Include -c palindrome_wrap.c
The second failed, the compiler couldn't find Python.h, the following worked:
gcc -I[somedirectories]\Anaconda3\Include -c palindrome_wrap.c
I then did, as many have said, including Hetland 3rd ed.,
gcc -shared palindrome.o palindrome_wrap.o [somedirectories]/Anaconda3/libs/libpython37.a -o _palindrome.dll
This did not work. Even with the Load Library cswu used (which I found elsewhere, too).
So I gendef'd _palindrome.dll and couldn't find the function in it, "is_palindrome" in the exports. I went through some of the SWIG documentation, and declared the function both in the %{ %} section and below it, both extern, that finally got the function extern'd in palindrome_wrap.c as it should have been. But no export, so I went back into palindrome.c and redeclared the function as:
declspec(dllexport) extern int __stdcall is_palindrome(char* text)
and redeclared it in palindrome.i in both places as above with this signature.
Partial success! It got listed in the Export section when I gendef'd _palindrome.dll and I could do cswu's call using Load Library. But still not do what Hetland says and do
import _palindrome
in Python.
Going back to all the sources again, I could not figure this out. I finally started reading the SWIG documentation from the beginning leaving no stone unturned -- Searching through the manual doesn't produce the place found.
At the end of Introduction Sec. 2.7 Incorporating Into a Build System, under the sample Make process, it says:
"The above example will generate native build files such as makefiles, nmake files and Visual Studio projects which will invoke SWIG and compile the generated C++ files into _example.so (UNIX) or _example.pyd (Windows). For other target languages on Windows a dll, instead of a .pyd file, is usually generated."
And that's the answer to the last problem:
The compile step for the dll should read:
gcc -shared palindrome.o palindrome_wrap.o [somedirectories]/Anaconda3/libs/libpython37.a -o _palindrome.pyd
(I didn't go back and change out my declspec declarations so I don't know whether they were necessary, so they were still there too).
I got a file, _palindrome.pyd
Which if in the PYTHONPATH (mine was local) works, and one can then do
import _palindrome
from _palindrome import is_palindrome
and use the exported, properly wrapped and packaged C function, compiled with TDM-GCC, in python as promised. gcc, which is MinGW64 in a different installation, knows how to do the .pyd file. I diffed the dll and pyd since they were the same byte length. They are not the same at hundreds of points.
Hope this helps someone else.

Building CPython using clang invokes /usr/bin/ld for final link, and gets "/usr/bin/ld: error: libpython3.2m.a: malformed archive header name at 8"

I'm attempting to build CPython 3.2 using llvm's clang. It builds OK with gcc. I'm on Linux Mint 12.
The compile seems to go fine until the end, where it attempts to produce the ./python binary. The error I get is:
clang -pthread -Xlinker -export-dynamic -o python Modules/python.o libpython3.2m.a -lpthread -ldl -lutil -lm
/usr/bin/ld: error: libpython3.2m.a: malformed archive header name at 8
./Modules/python.c:25: error: undefined reference to 'PyMem_Malloc'
./Modules/python.c:27: error: undefined reference to 'PyMem_Malloc'
./Modules/python.c:51: error: undefined reference to '_Py_char2wchar'
./Modules/python.c:59: error: undefined reference to 'Py_Main'
./Modules/python.c:61: error: undefined reference to 'PyMem_Free'
./Modules/python.c:63: error: undefined reference to 'PyMem_Free'
./Modules/python.c:64: error: undefined reference to 'PyMem_Free'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
I've tried with the llvm 2.9 that's in my trusty Linux Mint 12 .deb repo, and with an SVN trunk of llvm from earlier today, but both give the same errors.
It occurs to me that perhaps things would go better if I could persuade clang to use llvm-link instead of /usr/bin/ld for linking, but I see nothing in the documentation about how to do that. Just setting $LD to llvm-link doesn't appear to help.
Those undefined references are likely red herrings: If I nm -o libpython3.2m.a (or llvm-nm -o libpython3.2m.a - both work), I can see that at least one of these symbols really is defined (according to nm of course - other tools could perhaps, and likely do, have other ideas).
Ultimately I want to generate llvm bitcode rather than an ELF binary, but it'd be a satisfying mini-milestone to get a working python ELF binary with llvm first.
I'm currently using the following for compilation:
export PATH=/usr/local/llvm/bin:$PATH
prefix=/usr/local/p3ib-3.2
export CC='clang'
export AR='llvm-ar'
export LD='llvm-link -emit-llvm-bc'
export RANLIB='llvm-ranlib'
./configure --prefix="$prefix" && make -j 3
Thanks!

A multitude of problems trying to install Python bcrypt on Windows (x64)

This problem has been mentioned elsewhere, but it does not provide a working solution, so I would still like to keep a ticket open.
c:\Users\me\Desktop\temp\py-bcrypt-0.2>setup.py install build --compiler=min
gw32
running install
running build
running build_py
running build_ext
building 'bcrypt._bcrypt' extension
C:\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -IC:\Python27\include -IC:\Python27\PC -c bcrypt/bcrypt_python.c -o build\temp.win32-2.7\Release\bcrypt\bcrypt_python.o
bcrypt/bcrypt_python.c:29:26: error: expected declaration specifiers or '...' before 'u_int8_t'
bcrypt/bcrypt_python.c:29:38: error: expected declaration specifiers or '...' before 'u_int16_t'
bcrypt/bcrypt_python.c:29:49: error: expected declaration specifiers or '...' before 'u_int8_t'
bcrypt/bcrypt_python.c: In function 'bcrypt_encode_salt':
bcrypt/bcrypt_python.c:56:2: error: too many arguments to function 'encode_salt'
bcrypt/bcrypt_python.c:29:6: note: declared here
error: command 'gcc' failed with exit status 1
This is after getting rid of the initial “error: Setup script exited with error: Unable to find vcvarsall.bat”, I am left with what seems like a Gordic knot. pip and easy-install do not work. I have also read a number of other articles.
I don’t need to implement bcrypt at the moment, so it is not an urgent matter, but obviously, implementing a secure password protection is vital.
The simplest way I found was to download and install windows binaries, rather than trying to compile myself. This guy has some on bitbucket
supposing you are using mingw64, you should change _MSC_VER in _WIN32 on ifdefs into bcrypt.c, bcrypt_python.c and pybc_blf.h
(I've also replied the other question)

Categories

Resources