When I compiled Python using PCBuild\build.bat I discovered that several Python external projects like ssl, bz2, ... were not compiled because the compiler did not find them.
I did run the Tools\Buildbot\external.bat and it did download them inside \Tools\ but it looks that the build is not looking for them in this location and the PCBuild\readme.txt does not provide the proper info regarding this.
In case it does matter, I do use VS2008 and VS2010 on this system.
Example:
Build log was saved at "file://C:\dev\os\py3k\PCbuild\Win32-temp-Release\_tkinter\BuildLog.htm"
_tkinter - 2 error(s), 0 warning(s)
Build started: Project: bz2, Configuration: Release|Win32
Compiling...
bz2module.c
..\Modules\bz2module.c(12) : fatal error C1083: Cannot open include file: 'bzlib.h': No such file or
directory
Tools\buildbot\external.bat must be run from py3k root, not from Tools\buildbot\ subdir as you did. Also to build release version of python with Tkinter support you have to edit or copy Tools\buildbot\external.bat to remove DEBUG=1 so it can build tclXY.dll/tkXY.dll (without -g suffix).
Related
I am trying to setup a CMake project that creates python bindings for its c++ functions using pybind11 on Ubuntu.
The directory structure is:
pybind_test
arithmetic.cpp
arithmetic.h
bindings.h
CMakeLists.txt
main.cpp
pybind11 (github repo clone)
Repo contents (https://github.com/pybind/pybind11)
The CMakeLists.txt file:
cmake_minimum_required(VERSION 3.10)
project(pybind_test)
set(CMAKE_CXX_STANDARD 17)
find_package(PythonLibs REQUIRED)
include_directories(${PYTHON_INCLUDE_DIRS})
include_directories(pybind11/include/pybind11)
add_executable(pybind_test main.cpp arithmetic.cpp)
add_subdirectory(pybind11)
pybind11_add_module(arithmetic arithmetic.cpp)
target_link_libraries(pybind_test ${PYTHON_LIBRARIES})
The repository builds successfully and the file arithmetic.cpython-36m-x86_64-linux-gnu.so is produced. How do I import this shared object file into python?
The documentation in the pybind11 docs has this line
$ c++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix`
but I want to build using CMake and I also don't want to have to specify extra include directories every time I run python to use this module.
How would I import this shared object file into python like a normal python module?
I am using Ubuntu 16.04.
If you open a terminal, go to the directory where arithmetic.cpython-36m-x86_64-linux-gnu.so is located and run python followed by import arithmetic the module will get imported just like any other module.
Another options is to use the method of
import sys
sys.path.insert(0, 'path/to/directory/where/so-file/is')
import arithmetic
With this method you can use both relative and absolute path.
Besides the solution of setting the path in the Python script that is presented by #super, you have two more generic solutions.
Setting PYTHONPATH
There is an environment variable in Linux (and macOS) called PYTHONPATH. If you add the path that contains your *.so to the PYTHONPATH before you call Python, Python will be able to find your library.
To do this:
export PYTHONPATH="/path/that/contains/your/so":"${PYTHONPATH}"
To apply this 'automatically' for every session you can add this line to ~/.bash_profile or ~/.bashrc (see the same reference). In that case, Python will always be able to find your library.
Copying your to a path already in Python's path
You can also 'install' the library. The usual way to do this is to create a setup.py file. If set up correctly you can build and install your library using
python setup.py build
python setup.py install
(Python will know where to put your library. You can 'customize' a bit with an option like --user to use your home-folder, but this doesn't seems to be of particular interest to you.)
The question remains: How to write setup.py? For your case you can actually call CMake. In fact there exists an example that does exactly that: pybind/cmake_example. You can basically copy-paste from there.
I need to try python 3.7 with openssl-1.1.1 in Ubuntu 16.04. Both python and openssl versions are pre-release. Following instructions on how to statistically link openssl to python in a previous post, I downloaded the source for opnssl-1.1.1.
Then navigate to the source code for openssl and execute:
./config
sudo make
sudo make install
Then, edit Modules/Setup.dist to uncomment the following lines:
SSL=/usr/local/ssl
_ssl _ssl.c \
-DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
-L$(SSL)/lib -lssl -lcrypto
Then download python 3.7 source code. Then, navigate inside the source code and execute:
./configure
make
make install
After I execute make install I got this error at the end of the terminal output:
./python: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
generate-posix-vars failed
Makefile:596: recipe for target 'pybuilddir.txt' failed
make: *** [pybuilddir.txt] Error 1
I could not figure out what is the problem and what I need to do.
This has (should have) nothing to do with Python or OpenSSL versions.
Python build process, includes some steps when the newly built interpreter is launched, and attempts to load some of the newly built modules - including extension modules (which are written in C and are actually shared objects (.sos)).
When an .so is loaded, the loader must find (recursively) all the .so files that the .so needs (depends on), otherwise it won't be able to load it.
Python has some modules (e.g. _ssl*.so, _hashlib*.so) that depend on OpenSSL libs. Since you built yours against OpenSSL1.1.1 (the lib names differ from what comes by default on the system: typically 1.0.*), the loader won't be able to use the default ones.
What you need to do, is instruct the loader (check [Man7]: LD.SO(8) for more details) where to look for "your" OpenSSL libs (which are located under /usr/local/ssl/lib). One way of doing that is adding their path in ${LD_LIBRARY_PATH} env var (before building Python):
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/ssl/lib
./configure
make
make install
You might also want to take a look at [Python.Docs]: Configure Python - Libraries options (--with-openssl, --with-openssl-rpath).
Check [SO]: How to enable FIPS mode for libcrypto and libssl packaged with Python? (#CristiFati's answer) for details on a wider problem (remotely) related to yours.
What I have done to fix this :
./configure --with-ssl=./libssl --prefix=/subsystem
sed -i 's!^RUNSHARED=!RUNSHARED=LD_LIBRARY_PATH=/path/to/own/libssl/lib!' Makefile
make
make install
Setting LD_LIBRARY_PATH with export was not sufficient
With Python-3.6.5 and openssl-1.1.0h i get stuck in the same problem. I have uncomment _socket socketmodule.c.
I am having a LOT of trouble to start with these are my specs
windows 8
64 bit
python 2.7
Apachehaus 2.4.12
I cannot for the life of me figure out how to compile mod_wsgi to make the necessary mod_wsgi.so file from the source code.
https://github.com/GrahamDumpleton/mod_wsgi/releases
I'm trying to compile the ap24py27-win64-VC9.mk and after opening Visual C++ 2008 64-bit command prompt
c:\mod_wsgi-4.4.13> nmake -f ap24py27-win64-VC9.mk
c:\mod_wsgi-4.4.13\src\server\wsgi_python.h(24) : fatal error 1083: Cannot open include file: 'Python.h': No such file or directory wsgi_restrict.c
c:\mod_wsgi-4.4.13\src\server\wsgi_python.h(24) : fatal error 1083: Cannot open include file: 'Python.h': No such file or directory wsgi_server.c
c:\python27\mod_wsgi-4.4.13\src\server\wsgi_python.h(24) : fatal error C1083: Cannot open include file: 'Python.h': No such file or directory wsgi_stream.c
c:\mod_wsgi-4.4.13\src\server\wsgi_python.h(24) : fatal error C1083: Cannot open include file: 'Python.h': No such file or directory wsgi_validate.c
c:\mod_wsgi-4.4.13\src\server\wsgi_python.h(24) : fatal error C1083: Cannot open include file: 'Python.h': No such file or directory
Generating Code...
c:\mod_wsgi-4.4.13\src\server\wsgi_memory.c(124) : warning C4711: function 'getPeakRSS' selected for automatic inline expansion
c:\mod_wsgi-4.4.13\src\server\wsgi_memory.c(125) : warning C4711: function 'getCurrentRSS' selected for automatic inline expansion
NMAKE : fatal error U1077: '"C:\Users\User\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.EXE"' : return code '0x2'
Stop.
I already set my path to path=%path% Python27/include because that's where the Python.h file is.
I've also tried
python setup install
Traceback (most recent call last):
File "setup.py", line 139, in <module>
'missing Apache httpd server packages.' % APXS)
RuntimeError: The 'apxs' command appears not to be installed or is not executable. Please check the list of prerequisites in the documentation for this package and install any missing Apache httpd server packages.
The compiler doesn't use the PATH environment variable for finding header files. Usually it uses INCLUDE.
I think you can pass this into nmake, like so
nmake INCLUDE=pathToPythonHeader -f ap24py27-win64-VC9.mk
Why are you trying to compile it from source code? Why aren't you using the precompiled binaries?
There is nothing additional in the latter release that you would really need on Windows, so just grab and use the latest binary version which at this time is 4.4.12.
You also can't do a 'python setup.py install' nor a 'pip install' on Windows.
If you really do for some unknown reason need to build from source code, you do at least need to modify the ap24py27-win64-VC9.mk file and override the locations for where Python and Apache is installed. The default locations are based on a non standard layout that allows me to have many different Python and Apache versions installed for different architectures at the same time.
I'm trying to set up Boost.Python 1.54.0 on Windows 7, python 2.7.3. I installed boost_1_54_0 and built it with Visual Studio 2010. I can build and run the quickstart and tutorial examples in the distribution (I had to add some MSVC lib dirs as library-path requirements in Jamroot, but that's OK). But when I copy the tutorial example to a different dir, not inside the boost dist dir, to make it more like a real project, bjam gives build errors. I changed the use-project boost location in Jamroot to /local/boost_1_54_0/ which is where boost wanted to install itself, and added a simple one-line boost-build.jam (based on something I found online):
boost-build C:/local/boost_1_54_0/tools/build/v2 ;
but ultimately bjam gives me this error:
% bjam
notice: no Python configured in user-config.jam
notice: will use default configuration
C:/local/boost_1_54_0/tools/build/v2/build\project.jam:262: in find-jamfile from module project
error: Unable to load Jamfile.
error: Could not find a Jamfile in directory '/local/boost_1_54_0'.
error: Attempted to find it with pattern '[Bb]uild.jam [Jj]amfile.v2 [Jj]amfile [Jj]amfile.jam'.
error: Please consult the documentation at 'http://www.boost.org'.
C:/local/boost_1_54_0/tools/build/v2/build\project.jam:280: in load-jamfile from module project
C:/local/boost_1_54_0/tools/build/v2/build\project.jam:64: in load from module project
C:/local/boost_1_54_0/tools/build/v2/build\project.jam:89: in load-used-projects from module project
C:/local/boost_1_54_0/tools/build/v2/build\project.jam:75: in load from module project
C:/local/boost_1_54_0/tools/build/v2/build\project.jam:145: in project.find from module project
C:/local/boost_1_54_0/tools/build/v2\build-system.jam:535: in load from module build-system
C:\local\boost_1_54_0\tools\build\v2/kernel\modules.jam:289: in import from module modules
C:\local\boost_1_54_0\tools\build\v2/kernel/bootstrap.jam:139: in boost-build from module
C:\tmp\tutorial\boost-build.jam:1: in module scope from module
What else do I need to add, or what am I doing wrong?
I ended up giving up on bjam for this, and just used SCons. A simple SConstruct was enough:
# SConstruct for building boost python tutorial example
import os
boost_python_lib = 'boost_python-vc100-gd-1_54'
boost_top = 'c:/boost'
python_top = 'c:/python27'
env=Environment(TARGET_ARCH='x86',
CCFLAGS=['/MDd', '/DEBUG', '/EHsc'],
CPPPATH=[os.path.join(boost_top,'include/boost-1_54'),
os.path.join(python_top, 'include')],
LIBPATH=[os.path.join(boost_top, 'lib/i386'),
os.path.join(python_top, 'libs')])
dll=env.SharedLibrary('hello_ext', 'hello.cpp',
LIBS=boost_python_lib)
env.InstallAs('hello_ext.pyd', dll[0])
# Copy the boost python lib into this dir so hello_ext will find it at runtime
env.Install('.', os.path.join(boost_top, 'lib/i386', '%s.dll'%boost_python_lib))
Of course you could make a real SCons Tool out of this, but that was enough to get me going. Hope it's useful to others.
I am working on an application written in C. One part of the application should embed python and there is my current problem. I try to link my source to the Python library but it does not work.
As I use MinGW I have created the python26.a file from python26.lib with dlltool and put the *.a file in C:/Program Files (x86)/python/2.6/libs.
Therefore, I compile the file with this command:
gcc -shared -o mod_python.dll mod_python.o "-LC:\Program Files (x86)\python\2.6\libs" -lpython26 -Wl,--out-implib,libmod_python.a -Wl,--output-def,mod_python.def
and I get those errors:
Creating library file: libmod_python.a
mod_python.o: In function `module_init':
mod_python.c:34: undefined reference to `__imp__Py_Initialize'
mod_python.c:35: undefined reference to `__imp__PyEval_InitThreads'
... and so on ...
My Python "root" folder is C:\Program Files (x86)\python\2.6
The Devsystem is a Windows Server 2008
GCC Information: Reading specs from C:/Program Files (x86)/MinGW/bin/../lib/gcc/mingw32/3.4.5/specs
Configured with: ../gcc-3.4.5-20060117-3/configure --with-gcc --with-gnu-ld --with-gnu-as --host=mingw32 --target=mingw32 --prefix=/mingw --enable-threads --disable-nls --enable-languages=c,c++,f77,ada,objc,java --disable-win32-registry --disable-shared --enable-sjlj-exceptions --enable-libgcj --disable-java-awt --without-x --enable-java-gc=boehm --disable-libgcj-debug --enable-interpreter --enable-hash-synchronization --enable-libstdcxx-debug
Thread model: win32
gcc version 3.4.5 (mingw-vista special r3)
What I do wrong? How I get it compiled and linked :-)?
Cheers, gregor
Edit:
I forgot to write information about my Python installation: It's the official python.org installation 2.6.1
... and how I created the python.a file:
dlltool -z python.def --export-all-symbols -v c:\windows\system32\python26.dll
dlltool --dllname c:\Windows\system32\python26.dll --def python.def -v --output-lib python26.a
Well on Windows the python distribution comes already with a libpython26.a in the libs subdir so there is no need to generate .a files using dll tools.
I did try a little example with a single C file toto.c:
gcc -shared -o ./toto.dll ./toto.c -I/Python26/include/ -L/Python26/libs -lpython26
And it works like a charm. Hope it will help :-)
Python (at least my distribution) comes with a "python-config" program that automatically creates the correct compiler and linker options for various situations. However, I have never used it on Windows. Perhaps this tool can help you though?
IIRC, dlltool does not always work. Having python 2.6 + Wow makes things even more less likely to work. For numpy, here is how I did it. Basically, I use obdump.exe to build the table from the dll, which I parse to generate the .def. You should check whether your missing symbols are in the .def, or otherwise it won't work.