Develop Reference

Cython and clang on mac, "Python.h not found"

I'm running clang on mac to compile a c file created by running a very simple program through cython, but the compiler always give me a "Python.h not found" fatal error. I've tried every solution I could find, reinstalling python 3.9, using the -I/path/to/headerfile method, and rewriting the include statement in the code to contain the full filepath, but nothing has worked. When I do include the full filepath, I get fatal error: 'cpython/initconfig.h' file not found. What could the issue possibly be, and how would I fix it? The program itself works fine in the standard python interpreter, pyinstaller, and nuitka.

Today I needed to embed Python into a C application developed on XCode with Clang compiler, and I've met the same problems and solved them. Let me share the tips:
1. Python not found error
You should install a Python framework to your MacOS, and attach it into Frameworks and Libraries project settings tab. Also, specify it's path in Framework Search Paths inside Build Settings tab if XCode hasn't done it automatically.
2. Undefined symbols / Implicit declaration errors
Verify you're using a modern version of Python framework. Versions prior to 3.6 had different ABI which leads to the mentioned errors.
3. cpython/initconfig.h file not found
Versions prior to ~3.9 had some header inclusion problem for the cpython/initconfig.h file (issues 40642, 39026).
Check the builds of Python that your OS currently has. I found Pythons installed in the following paths:
# Default 2.7 MacOSX installation is here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/
/Applications/Xcode.app/Contents/Developer/Library/Frameworks/
/Library/Frameworks/
Or install a recent Python version. I recommend using "universal binary" build, see the docs.

Force linking against Python release library in debug mode in Windows/Visual Studio from CMake

I'm developing a Python binding for a C++ library using Boost Python, for Linux and Windows (Visual Studio).
In Windows, the static Boost Python library has a dependency against Python (this is motive for another thread, here), so, in my CMake config I need to do:
if((${CMAKE_SYSTEM_NAME} STREQUAL "Linux") OR APPLE)
target_link_libraries(my_python_module ${Boost_LIBRARIES})
elseif(WIN32 AND MSVC)
add_definitions(/DBOOST_PYTHON_STATIC_LIB)
target_link_libraries(my_python_module ${Boost_LIBRARIES}) #This includes the Boost Python library
# Even though Boost Python library is included statically, in Windows it has a dependency to the Python library.
target_link_libraries(my_python_module ${Python_LIBRARIES})
endif()
This works fine in Linux, but in Windows, it only works in Release mode, not in Debug, in which case I always get a:
LINK : fatal error LNK1104: Can't open file 'python37.lib'
After some hair pulling I noticed the issue was caused by CMake instructing Visual Studio to link against 'python37_d.lib' instead of 'python37.lib' in the Debug mode.
However, as I described in the linked issue, the officially provided Boost Python debug library is linked against the Python release library, not the debug one. So, the solution would be to force the link against the Python release library, regardless of the build type. Unfortunately, ${Python_LIBRARIES} sets the library automatically depending on the mode, and I wouldn't like to explicitly specify python37.lib in my code (since I can upgrade Python and I don't want to have to change my CMake scripts because of that).
I found some similar issues here and here, but that doesn't reflect the exact situation I'm facing. Based on those, I tried setting:
target_link_libraries(my_python_module optimized ${Python_LIBRARIES})
But that didn't work either. So, the question is:
Is there a way to force the usage of the Python release library in Debug mode WITHOUT having to set it explicitly and leaving the Python CMake package to do it automatically instead. By explicit I mean doing:
target_link_libraries(my_python_module python37)
Thanks a lot for your help.

It seems that set(Python_FIND_ABI "OFF" "ANY" "ANY") as suggested in the comments by kanstar would be the correct way to do this. However, while Python_FIND_ABI is in CMake master, it hasn't been released yet in the latest version (v3.15.2 as of this writing).
In the meantime, there are solutions dependent on the CMake version.
CMake 3.12 and above
It's possible to link against FindPython's Python_LIBRARY_RELEASE, which isn't meant to be part of the module's public interface, but the variable is set correctly nonetheless.
cmake_minimum_required (VERSION 3.12)
find_package(Python ..<choose your COMPONENTS; refer to FindPython docs>..)
if(WIN32 AND MSVC)
target_link_libraries(my_python_module ${Python_LIBRARY_RELEASE})
endif()
CMake 3.0.4 to 3.11
Thanks to a comment by #Phil, we can expand the answer to include earlier CMake versions which had the FindPythonLibs module that sets the PYTHON_LIBRARY_RELEASE variable instead.
cmake_minimum_required (VERSION 3.0)
find_package(PythonLibs ..<refer to FindPythonLibs docs>..)
if(WIN32 AND MSVC)
target_link_libraries(my_python_module ${PYTHON_LIBRARY_RELEASE})
endif()

Installing Boost Python on Windows 8.1, Setting up the toolchain correctly

I am trying to install Boost Python, however http://www.boost.org/doc/libs/1_55_0/libs/python/doc/tutorial/doc/html/python/hello.html is too compressed for me. I am switchig to Windows from Linux, setting up all the technical stuff drives me crazy :)
So when I follow the instructions http://www.boost.org/doc/libs/1_55_0/more/getting_started/windows.html#prepare-to-use-a-boost-library-binary and try to execute bootstrap.bat it complains about a missing VCVARS32.BAT and it does not recognice the "cl" command - I have however both MS VS Express 2012 and 2013 installed, both boost versions (32, 64) are installed in C:/local/, documentations says that both should work. So why does it not find the compiler?
Update: So http://msdn.microsoft.com/en-us/library/9s7c9wdw%28v=vs.110%29.aspx says that you cannot use the compiler from the command line... how do I build my bjam then? This must be something totally trivial and stupid... :/
Update: Okay, one step further. vcvars32.bat actually sets all environment variables to go, so i can invoke the command line compiler now! Also there are actually prebuild binaries of bjam from the official boost site http://sourceforge.net/projects/boost/files/, I got it and copied it to by boost_xxx/tools folder and added the site to the path, so i can invoke bjam from command line as well.
Next I put a user-config.jam in my home folder, it only says:
# MSVC configuration
using msvc : 11.0 ;
# Python configuration
using python : : C:/Users/me/Anaconda ;
In the Jamroot file provided with C:\local\boost_1_55_0\libs\python\example\tutorial, I only edited
use-project boost
: C:/local/boost_1_55_0 ;
then I call bjam from the very same directory which gives my some cryptic output
C:\local\boost_1_55_0\libs\python\example\tutorial>bjam
warning: mismatched versions of Boost.Build engine and core
warning: Boost.Build engine (bjam) is 03.1.18
warning: Boost.Build core (at C:/local/boost_1_55_0/tools/build/v2) is 2011.12-s
vn
link.jam: No such file or directory
C:/local/boost_1_55_0/tools/build/v2/util\path.jam:458: in path.makedirs
rule MAKEDIR unknown in module path.
C:/local/boost_1_55_0/tools/build/v2/build\configure.jam:233: in configure.set-l
og-file
C:/local/boost_1_55_0/tools/build/v2\build-system.jam:695: in load
C:\local\boost_1_55_0\libs\python\example\..\..\..\tools\build\v2/kernel\modules
.jam:289: in import
C:\local\boost_1_55_0\libs\python\example\..\..\..\tools\build\v2\kernel\bootstr
ap.jam:139: in boost-build
C:\local\boost_1_55_0\libs\python\example\boost-build.jam:7: in module scope
C:\local\boost_1_55_0\libs\python\example\tutorial>
Now, can you give me any hints as to what went wrong and where to go from here? I really want a stable, robust working copy of boost.python - it doesn't make sense to me if I need to hack something together. I thought it was quite standard/reliable, isn't there something like "boost.python for guys who did not study computer science"? Can I ignore the warnings?

Perhaps you can use the binary from http://www.lfd.uci.edu/~gohlke/pythonlibs/#boost.python
Regarding your compiler problems: there is a bunch of questions on this on SO already. Basically, the compiler you need depends on your version of python. You will probably need to install some Windows SDK and setup some stuff manually. It has always been a pain in the ass for me and in the end you might be better of with using some python distribution like Canopy (there are a few others out there).

Trouble installing JCC (required for pylucene) on mac

I am following closely the installation guide for pylucene. I am unable to get past the first step, which requires installing JCC.
To install JCC the instructions briefly note that mac users will need to:
Edit setup.py and review that values in the INCLUDES, CFLAGS,
DEBUG_CFLAGS, LFLAGS and JAVAC are correct for your system. These
values are also going to be compiled into JCC's config.py file and are
going to be used by JCC when invoking distutils or setuptools to
compile extensions it is generating code for.
I am not sure what exactly to edit. I have Java 1.6 installed. When I run the setup.py (without any edits), it gives me the error (which I expect because I haven't edited anything, as instructed):
Can't determine where the Java JDK has been installed on this machine. Please set the environment variable JCC_JDK to that location
before running setup.py.
I am a novice coder, so am having trouble finding what I should edit in the setup.py to make this work on a mac? I have tried putting in the file path to java, but this has not helped. Any advice would be much appreciated, thanks!

Building LLVM fails with empty error message

I'm trying to build LLVM 3.1 and Clang 3.1. I followed the Getting Started guide from Clang's website- check out the repositories in the requisite places, get Python, etc.
If I have Python 3.3 installed, it gives a Python semantic error- from main import main, no module called main. If I have Python 2.7 installed, it gives
CMake Error at CMakeLists.txt:307 (message):
Unexpected failure executing llvm-build:
Configuring incomplete, errors occurred!
This is most unhelpful. Any suggestions as to what I can do to build LLVM and Clang on Windows, or at least attempt to determine what the problem is?
Just as a note, I am attempting to build with Visual Studio 2012, which is officially supported by CMake but was not released when LLVM 3.1 and Clang 3.1 were created.
Edit: Here are the requisite lines from CMakeLists.txt
message(STATUS "Constructing LLVMBuild project information")
execute_process(
COMMAND ${PYTHON_EXECUTABLE} ${LLVMBUILDTOOL}
--native-target "${LLVM_NATIVE_ARCH}"
--enable-targets "${LLVM_TARGETS_TO_BUILD}"
--enable-optional-components "${LLVMOPTIONALCOMPONENTS}"
--write-library-table ${LLVMCONFIGLIBRARYDEPENDENCIESINC}
--write-cmake-fragment ${LLVMBUILDCMAKEFRAG}
ERROR_VARIABLE LLVMBUILDOUTPUT
ERROR_VARIABLE LLVMBUILDERRORS
OUTPUT_STRIP_TRAILING_WHITESPACE
ERROR_STRIP_TRAILING_WHITESPACE
RESULT_VARIABLE LLVMBUILDRESULT)
# On Win32, CMake doesn't properly handle piping the default output/error
# streams into the GUI console. So, we explicitly catch and report them.
if( NOT "${LLVMBUILDOUTPUT}" STREQUAL "")
message(STATUS "llvm-build output: ${LLVMBUILDOUTPUT}")
endif()
if( NOT "${LLVMBUILDRESULT}" STREQUAL "0" )
message(FATAL_ERROR
"Unexpected failure executing llvm-build: ${LLVMBUILDERRORS}")
endif()
Here, you can see that ${LLVMBUILDRESULT} is not good, but ${LLVMBUILDERRORS} which is obviously supposed to contain the build errors does not contain anything.

The problem is two-fold:
First, LLVM requires Python 2.x. If you try to force it to use 3.3, it will fail. The docs probably should say something about this (and I couldn't find any reference to it when I looked), but at this stage it's still pretty common for projects to just say, e.g., "Python" or "Python 2.6+" when they mean "2.6-2.7 but not 3.x".
Second, like most configuration/build tools, CMake only detects dependency changes in the code, not in your system configuration. So if you try to build, then change your system, then try to build again, it won't notice the change and adjust its configuration accordingly. It was already configured to use Python 3.3, and it didn't notice you'd replaced it with Python 2.7, hence the problem. You have to force it to reconfigure—which you can do by clearing the cache, but the simplest and cleanest answer is to just untar/git/whatever yourself a clean directory and start over. (Or, if you can build out-of-tree, as you can with many projects, just wipe out the build directory and start over.)
Finally, according to the docs, you really don't need Python to build llvm and clang, unless you want to run the tests. So, unless the docs are wrong (which is of course possible), if you had just done a clean build with no Python at all, it would have worked. The reason it didn't work when you tried it is the same reason it didn't work after you installed 2.7: You were in a partially-configured state, it thought it had Python, and therefore it insisted on using it.

I used something like
"c:\Program Files (x86)\CMake\bin\cmake.exe" -G "Visual Studio 12 2013" -DPYTHON_EXECUTABLE="C:\Python27\python27.exe" -DLLVM_TARGETS_TO_BUILD="X86" ..\llvm-3.5.0.src
and it worked for me. In my case cmake was not finding python so I had to use
-DPYTHON_EXECUTABLE="C:\Python27\python27.exe"
Notice that I didn't override my system folders then somebody with the same problem should use something different for each folder of the command.

Apparently, this is a CMake fail primarily, it would have succeeded if I had cleared the cache after installing Python 2.7