Develop Reference

unable to comile PostgreSQL with error message "could not find shared library for Python"

I've been trying to build the REL_10_STABLE version of PostgreSQL using MSYS2 in windows 10 environment.Here is the command that I execute for configuring the build:
./configure --enable-thread-safety --enable-nls --with-ldap
--with-openssl --with-ossp-uuid --with-libxml --with-libxslt --with-icu --with-tcl --with-perl --with-python
but I get this error message:
checking for python... no
checking for python3... no
checking for python2... /mingw64/bin/python2
checking for Python distutils module... yes
checking Python configuration directory... C:/msys64/mingw64/lib/python2.7/config
checking Python include directories... -IC:/msys64/mingw64/include/python2.7
checking how to link an embedded Python application... configure: error: could not find shared library for Python
You might have to rebuild your Python installation. Refer to
the documentation for details. Use --without-python to disable building
PL/Python.
As you can see python 2.7 is installed which is the version I need to use and I need the python to be built with python, but I keep getting the error message "could not find shared library for Python".Any help is very much appreciated.
Thank youSam

Most likely you have built your python executable without the --enable-shared flag.
The PostgreSQL documentation says the following:
Since PL/Python will be a shared library, the libpython library must be a shared library also on most platforms. This is not the case in a default Python installation built from source, but a shared library is available in many operating system distributions. configure will fail if building PL/Python is selected but it cannot find a shared libpython. That might mean that you either have to install additional packages or rebuild (part of) your Python installation to provide this shared library. When building from source, run Python's configure with the --enable-shared flag.

Error when using .so: "wrong ELF class: ELFCLASS64" on 32-bit machine

Using Python 2.7.15 Ubuntu 18.04.2 on a 64-bit machine to build per instructions at
https://github.com/Azure/azure-iot-sdk-python/blob/master/doc/python-devbox-setup.md#build-the-azure-iot-hub-sdks-for-python-on-linux, we get this error when running the resultant .so on a 32-bit Yocto machine running Python 2.7.12.
Per this post: gcc error: wrong ELF class: ELFCLASS64, I imagine the build can be done with a GCC flag. Simply using "-m32" when running ./build.sh, however, did not work.
I suspect I could also use/change some sort of configuration parameter. However, I can not find anything obvious to change in the SDK.
More specifically, the error is:
'ImportError: /home/root/build_folder/iothub_client.so: wrong ELF class: ELFCLASS64'

It looks to me that you should be cross compiling the Python SDK for your target machine. The -m32 flag according to gcc docs is for Nvidia PTX Options. I suspect you don't want that. You need to find an appropriate tool chain for your target device's architecture and use that to build the Python binary. There is documentation that will help you with this here: https://github.com/Azure/azure-iot-sdk-python/blob/master/doc/Example%20of%20Cross%20Compiling%20the%20Azure%20IoT%20SDK%20for%20Python.md.
You didn't mention what the architecture of your Yocto device is.

Force linking against Python release library in debug mode in Windows/Visual Studio from CMake

I'm developing a Python binding for a C++ library using Boost Python, for Linux and Windows (Visual Studio).
In Windows, the static Boost Python library has a dependency against Python (this is motive for another thread, here), so, in my CMake config I need to do:
if((${CMAKE_SYSTEM_NAME} STREQUAL "Linux") OR APPLE)
target_link_libraries(my_python_module ${Boost_LIBRARIES})
elseif(WIN32 AND MSVC)
add_definitions(/DBOOST_PYTHON_STATIC_LIB)
target_link_libraries(my_python_module ${Boost_LIBRARIES}) #This includes the Boost Python library
# Even though Boost Python library is included statically, in Windows it has a dependency to the Python library.
target_link_libraries(my_python_module ${Python_LIBRARIES})
endif()
This works fine in Linux, but in Windows, it only works in Release mode, not in Debug, in which case I always get a:
LINK : fatal error LNK1104: Can't open file 'python37.lib'
After some hair pulling I noticed the issue was caused by CMake instructing Visual Studio to link against 'python37_d.lib' instead of 'python37.lib' in the Debug mode.
However, as I described in the linked issue, the officially provided Boost Python debug library is linked against the Python release library, not the debug one. So, the solution would be to force the link against the Python release library, regardless of the build type. Unfortunately, ${Python_LIBRARIES} sets the library automatically depending on the mode, and I wouldn't like to explicitly specify python37.lib in my code (since I can upgrade Python and I don't want to have to change my CMake scripts because of that).
I found some similar issues here and here, but that doesn't reflect the exact situation I'm facing. Based on those, I tried setting:
target_link_libraries(my_python_module optimized ${Python_LIBRARIES})
But that didn't work either. So, the question is:
Is there a way to force the usage of the Python release library in Debug mode WITHOUT having to set it explicitly and leaving the Python CMake package to do it automatically instead. By explicit I mean doing:
target_link_libraries(my_python_module python37)
Thanks a lot for your help.

It seems that set(Python_FIND_ABI "OFF" "ANY" "ANY") as suggested in the comments by kanstar would be the correct way to do this. However, while Python_FIND_ABI is in CMake master, it hasn't been released yet in the latest version (v3.15.2 as of this writing).
In the meantime, there are solutions dependent on the CMake version.
CMake 3.12 and above
It's possible to link against FindPython's Python_LIBRARY_RELEASE, which isn't meant to be part of the module's public interface, but the variable is set correctly nonetheless.
cmake_minimum_required (VERSION 3.12)
find_package(Python ..<choose your COMPONENTS; refer to FindPython docs>..)
if(WIN32 AND MSVC)
target_link_libraries(my_python_module ${Python_LIBRARY_RELEASE})
endif()
CMake 3.0.4 to 3.11
Thanks to a comment by #Phil, we can expand the answer to include earlier CMake versions which had the FindPythonLibs module that sets the PYTHON_LIBRARY_RELEASE variable instead.
cmake_minimum_required (VERSION 3.0)
find_package(PythonLibs ..<refer to FindPythonLibs docs>..)
if(WIN32 AND MSVC)
target_link_libraries(my_python_module ${PYTHON_LIBRARY_RELEASE})
endif()

How can the Python runtime be embedded in an R package that runs on Windows

We wish to create an R package that wraps the Python runtime and is 'dependency free' (e.g. one need not install Python for the R package to work). There is already a package that allows R to call Python (CRAN package rPython) but it requires that Python be installed on the target machine. We would like all dependencies to be installed when the envisioned package is installed via the standard R package install mechanism. I forked rPython and created this variation: https://github.com/brucehoff/rWithPython. The package works on Unix and Mac. It downloads and builds the Python source and then accesses the Python runtime from R. By using a static version of libpython it builds a shared object that can be installed on another machine without leaving any dependencies behind. The problem is how to get it to work on Windows. On windows, R packages are built using "R Tools" (https://cran.r-project.org/bin/windows/Rtools/) which uses the cygwin/mingw stack. I tried running configure/make as on Unix, but it fails. I then tried linking a static libpython that I built on a Windows box using Visual Studio. Python is very helpful, providing instructions for changing the MVS build to create a static library:
PCBuild\readme.txt says, in part:
The solution has no configuration for static libraries. However it is
easy to build a static library instead of a DLL. You simply have to set
the "Configuration Type" to "Static Library (.lib)" and alter the
preprocessor macro "Py_ENABLE_SHARED" to "Py_NO_ENABLE_SHARED". You may
also have to change the "Runtime Library" from "Multi-threaded DLL
(/MD)" to "Multi-threaded (/MT)".
This works great and I get a static library for python. However when I try to link the library under cygwin/rtools the linker gives an error:
gcc -m32 -I"C:/bin/R/include" -DNDEBUG -I"d:/RCompile/CRANpkg/extralibs64/local/include" -I"C:/Python35/Include" -I"C:/Python35/PC" -O3 -Wall -std=gnu99 -mtune=core2 -c pycall.c -o pycall.o
gcc -m32 -shared -s -static-libgcc -o rWithPython.dll tmp.def pycall.o -LC:/Python35/PCbuild/win32 -lpython35 -Ld:/RCompile/CRANpkg/extralibs64/local/lib/i386 -Ld:/RCompile/CRANpkg/extralibs64/local/lib -LC:/bin/R/bin/i386 -lR
c:/rtools/gcc-4.6.3/bin/../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/bin/ld.exe: C:/Python35/PCbuild/win32/libpython35.a(C:/hgpy/cpython/PCbuild/obj//win32_Release/pythoncore/getbuildinfo.obj): Recognised but unhandled machine type (0x14c) in Import Library Format archive
pycall.o:pycall.c:(.text+0x5): undefined reference to '_imp__Py_Initialize'
pycall.o:pycall.c:(.text+0x1a): undefined reference to '_imp__PyRun_SimpleStringFlags'
pycall.o:pycall.c:(.text+0x31): undefined reference to '_imp__Py_Finalize'
pycall.o:pycall.c:(.text+0x56): undefined reference to '_imp__PyRun_SimpleStringFlags'
pycall.o:pycall.c:(.text+0x8c): undefined reference to '_imp__PyImport_AddModule'
pycall.o:pycall.c:(.text+0x95): undefined reference to '_imp__PyModule_GetDict'
pycall.o:pycall.c:(.text+0xa8): undefined reference to '_imp__PyDict_GetItemString'
pycall.o:pycall.c:(.text+0xb5): undefined reference to '_imp__PyUnicode_AsUTF8String'
collect2: ld returned 1 exit status
no DLL was created
ERROR: compilation failed for package 'rWithPython'
From what I've read "machine type" 0x14c is "Intel 386 or later, and compatible processors", i.e. the most common/expected machine type. So I'm guessing the error is a red herring, the problem is an incompatibility between compilers, not machines.
Any suggestions on how to proceed are appreciated!!
---- UPDATE ----
I verified that I can build/link (1) when linking in the library(ies) that are part of the standard Windows Python installation and (2) when building from source using MS Visual Studio, without modifying the build in any way. The problem is replicated as soon as I modify the Visual Studio build settings to produce a static library (python35.lib) following the guidelines in PCbuild\readme.txt. Those guidelines are a bit ambiguous. I will probe further, but if anyone has had success generating a static Python library in MS VS, please let me know!
---- ANOTHER UPDATE ----
We have a solution to this question:
https://github.com/Sage-Bionetworks/PythonEmbedInR
(The code is open source, under the GPL-3 license.)
You can see how we solved the problem of running on Windows:
https://github.com/Sage-Bionetworks/PythonEmbedInR/blob/master/configure.win
In short we do not try to compile from source or make a static library, rather we use Python's 'embeddable zip' and make sure its libraries are on the search path for our application. It seems to work great!

You need to put the Py_NO_ENABLE_SHARED define in your application code as well. Basically, anywhere the Python.h file is included. If not present, the include file assumes you have linked with a .dll and acts accordingly (and incorrectly.)

Installing graph-tool on Mac OS X (10.7) - already have Boost installed, but keep getting this error

I've been stuck on this issue for a while now. I'm trying to install graph-tool - http://graph-tool.skewed.de/download#macos - and I have the prereqs from following these steps, which the graph-tool site links to: https://gist.github.com/openp2pdesign/8864593
Instead of brew install, which didn't seem to give me all the files, I went to Boost's official site and downloaded from there properly, following these steps: http://www.boost.org/doc/libs/1_41_0/more/getting_started/unix-variants.html It's mainly getting a tar file and untarring it.
I then put my boost install here:
/usr/local/boost_1_55_0
I did a small C++ example and confirmed Boost works (using "Build a Simple Program Using Boost" from http://www.boost.org/doc/libs/1_41_0/more/getting_started/unix-variants.html.
Now the meat of the problem: trying to install graph-tool. In the very last step, I do
./configure PYTHON_EXTRA_LDFLAGS="-L/usr/local/bin"
(The PYTHON_EXTRA_LDFLAGS="-L/usr/local/bin" just makes the configure script find Python alright.)
But I get this error. (It finds Python fine, but not boost!)
...
================
Detecting python
================
checking for a Python interpreter with version >= 2.6... python
checking for python... /Users/daze/Library/Enthought/Canopy_64bit/User/bin/python
checking for python version... 2.7
checking for python platform... darwin
checking for python script directory... ${prefix}/lib/python2.7/site-packages
checking for python extension module directory... ${exec_prefix}/lib/python2.7/site-packages
checking for python2.7... (cached) /Users/daze/Library/Enthought/Canopy_64bit/User/bin/python
checking for a version of Python >= '2.1.0'... yes
checking for a version of Python == '2.7.3'... yes
checking for the distutils Python package... yes
checking for Python include path... -I/Applications/Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/include/python2.7
checking for Python library path... -L/Applications/Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/lib/python2.7/config -lpython2.7
checking for Python site-packages path... /Users/daze/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages
checking python extra libraries... -ldl -framework CoreFoundation
checking python extra linking flags... -L/usr/local/bin
checking consistency of all components of python development environment... yes
graph-tool will be installed at: /Users/daze/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages
===========================
Using python version: 2.7.3
===========================
checking for boostlib >= 1.38.0... configure: error: We could not detect the boost
libraries (version 1.38 or higher). If you have a staged boost library (still not installed)
please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option.
If you are sure you have boost installed, then check your version number looking in
<boost/version.hpp>. See http://randspringer.de/boost for more documentation.
Attempt 2: I then tried setting BOOST_ROOT properly:
In my ~/.bash_profile:
export BOOST_ROOT="/usr/local/boost_1_55_0"
But it still did no good, so I unset that.
Attempt 3: I then tried explicitly specifying where boost is installed:
./configure --with-boost="/usr/local/boost_1_55_0" PYTHON_EXTRA_LDFLAGS="-L/usr/local/bin"
But it still can't find boost, and yields that same error in the end of "We could not detect the boost libraries (version 1.38 or higher)."
It's been bugging me all day. I've read carefully, and went to the randspringer.de/boost site and saw this in the FAQ - http://www.randspringer.de/boost/faq.html#id2514912:
Q: I do not understand the configure error message
At configure time I get:
checking for boostlib >= 1.33... configure: error: We could not detect
the boost libraries (version 1.33 or higher). If you have a staged
boost library (still not installed) please specify $BOOST_ROOT in your
environment and do not give a PATH to --with-boost option. If you are
sure you have boost installed, then check your version number looking
in . See http://randspringer.de/boost for more
documentation.
I don't know if I use a staged version of boost. What is it and what
can I do ?
A: If you did not compile Boost by yourself you don't have a staged
version and you don't have to set BOOST_ROOT. Look here for an
explanation of different kind of installations.
If you are sure you have Boost installed then specify the directory
with
./configure --with-boost=your-boost-directory.
If it still does not work, please check the version number in
boost/version.hpp and compare it with the version requested in
configure.ac.
And I don't know what to see when comparing version numbers. There's nothing I found interesting there.
Hoping someone has at least an idea on what other approaches to take.

Hooray, my first chance to give back to Stack Overflow! I've been dealing with this issue myself the past 2 days.
Solution
Upgrade clang via Xcode
Make a symlink to boost that includes the version number
/usr/local/include/boost-1_55.0 -> ../Cellar/boost/1.55.0/include/boost
(included because I installed Boost using Brew and had this issue)
Edit the generation of CXXFLAGS in configure so that it looks like this:
old_cxxflags="$CXXFLAGS"
CXXFLAGS="${CXXFLAGS} -std=gnu++11 -stdlib=libc++"
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether C++ compiler supports -std=gnu++11" >&5
$as_echo_n "checking whether C++ compiler supports -std=gnu++11... " >&6; }
Run
./configure --disable-sparsehash CXX="/usr/bin/clang++" PYTHON_EXTRA_LDFLAGS="-L/usr/local/bin"
Versions
OS: Mac OS X 10.8.5
Clang: Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin12.5.0
Thread model: posix
Graph-tool: 2.2.29.1
Boost: 1.55.0
Explanation
If you go through the configure code and try and compile the confdefs.h files made in configure, you'll see clang error out upon encountering the -Wno-unused-local-typedefs flag. This is the actual cause of the "We could not detect the boost libraries (version 1.33 or higher)" error, not the fact that it can't find the boost files. This issue is fixed with newer versions of clang.
The configure test for version number is goofy. It expects the boost include directory to contain the version number.
While running make, you may run into the following errors:
./../graph_adjacency.hh:26:10: fatal error: 'tuple' file not found
This is caused by referencing the wrong standard library [1]
./../graph_adaptor.hh:655:39: error: expected ';' in 'for' statement specifier
for(typeof(removed_edges.begin()) iter = removed_edges.begin();
./../graph_adaptor.hh:655:39: error: use of undeclared identifier 'tier'
This is caused by referencing the wrong C++ standard (c++11 instead of gnu++11)
References
[1] No member named 'forward' in namespace 'std'
[2] I'm having some trouble with C++11 in Xcode

I think that you're currently pointing --with-boost to the boost parent directory, not the boost libraries.
Try
./configure --with-boost="/usr/local/boost_1_55_0/libs/" PYTHON_EXTRA_LDFLAGS="-L/usr/local/bin"