Develop Reference

Cmake cannot add "-fPIE" flag to makefile

I'm trying to compile C++ code with python embedded in it. The compiler does not compile my code with '-fPIE' flag even though I've added it in CMakeList.txt. Here's my CMakeList:
cmake_minimum_required(VERSION 3.3)
project(TestCython)
add_executable(main main.cpp)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED True)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-unused-result -Wsign-compare -g -fdebug-prefix-map=/build/python3.8-6QL2k7/python3.8-3.8.2=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -fPIE")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-unused-result -Wsign-compare -g -fdebug-prefix-map=/build/python3.8-6QL2k7/python3.8-3.8.2=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -fPIE")
# Eigen
find_package(Eigen3 3.3 REQUIRED NO_MODULE)
# Embedded Python Includes
include_directories(/usr/include/python3.8
/usr/include/eigen3/ )
# Embedded Python Linker
link_directories(/usr/lib/python3.8/config-3.8-x86_64-linux-gnu
/usr/lib)
target_link_libraries( main
python3.8
crypt
pthread
dl
util
m )
set(CMAKE_C_COMPILER gcc)
set(CMAKE_CXX_COMPILER g++)
If I run the generated make file, it will report a lot of similar errors and ask me to recompile with '-fPIE', like this one:
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libpython3.8.a(call.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a PIE object; recompile with -fPIE
The compilation succeeds if I don't use cmake:
g++ -I/usr/include/python3.8 -Wno-unused-result -Wsign-compare -g -fdebug-prefix-map=/build/python3.8-6QL2k7/python3.8-3.8.2=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -fPIE main.cpp -o main -L/usr/lib/python3.8/config-3.8-x86_64-linux-gnu -L/usr/lib -lpython3.8 -lcrypt -lpthread -ldl -lutil -lm
Cmake version is 3.16.3. Python version is 3.8.2. System is Ubuntu 20.04
How can I solve those errors?

Note that most of the time this error has nothing to do with the PIE flag but is rather something else screwy about your build, unfortunately it’s a bit hard to tell because your CMake config is a bit screwy.
This particular error however is probably because you’re setting your properties incorrectly; Setting CMAKE_CXX_FLAGS and similar variables does nothing if you do it after defining your target with add_executable. This can be avoided entirely by using a more modern CMake style (out of scope for this answer, just google Modern CMake for some examples), in this particular case instead of setting flags on those variables you could for example do
target_compile_options(main PRIVATE -fpie)

I had similar error when I linked my library env2048 (having a file called ppo_buffer) to my bindings module env2048bindings:
[build] FAILED: python/env2048bindings.cpython-37m-x86_64-linux-gnu.so
[build] : && /usr/bin/g++-7 -fPIC -g -shared -o python/env2048bindings.cpython-37m-x86_64-linux-gnu.so python/CMakeFiles/env2048bindings.dir/env2048/bindings.cpp.o src/libenv2048.a -pthread && :
[build] /usr/bin/ld: src/libenv2048.a(ppo_buffer.cpp.o): relocation R_X86_64_PC32 against symbol `_ZSt3minIlERKT_S2_S2_' can not be used when making a shared object; recompile with -fPIC
[build] /usr/bin/ld: final link failed: Bad value
I had to set the properties for the library to make it work with:
set_target_properties(env2048 PROPERTIES POSITION_INDEPENDENT_CODE ON)
From these, I would make the guess that you might need this:
set_target_properties(python3.8 PROPERTIES POSITION_INDEPENDENT_CODE ON)

Unable to install IPOPT for python (missing library dependency)

I built IPOPT from source at /usr/local/
Then I tried to install ipopt for python python setup.py install
However the installer complained (error) that it could not locate IpStdCInterface.h
I then modified the content of setup.py file as follows: (line 1 and 3 had wrong paths)
IPOPT_ICLUDE_DIRS=['/usr/local/include/coin']
IPOPT_LIBS=['ipopt', 'coinhsl', 'coinlapack', 'coinblas', 'coinmumps', 'coinmetis']
IPOPT_LIB_DIRS=['/usr/local/lib/']
IPOPT_DLL=None
I modified the first and third line to point to the correct directory.
Now, the lib directory path (in the third line above) contains a library file named libipopt.so, libipopt.so.0, libipopt.la (same name, different extensions).
The setup is now not complaining about IpStdCInterface.h but it is unable to find the libs
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/src/cyipopt.o -L/usr/local/lib/ -lipopt -lcoinhsl -lcoinlapack -lcoinblas -lcoinmumps -lcoinmetis -o build/lib.linux-x86_64-2.7/ipopt/cyipopt.so
/usr/bin/ld: cannot find -lcoinhsl
/usr/bin/ld: cannot find -lcoinlapack
/usr/bin/ld: cannot find -lcoinblas
/usr/bin/ld: cannot find -lcoinmumps
/usr/bin/ld: cannot find -lcoinmetis
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Those libs should be provided by the libipopt.so file? Or do I need to install some other package?

Well, i have a same problem as you. My solution was add the following commands in .bashrc:
export IPOPT_HOME="/opt/CoinIpopt"
export PATH="${PATH}:${IPOPT_HOME}/bin"
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${IPOPT_HOME}/lib"
/opt/CoinIpopt is the path which the IpOpt was installed.

Setup pjsip for Python

I'm trying to install pjsip's Python binding but am running into a build error that I feel is environmental but am just not able to figure out what's wrong.
I'm able to build pjsip without issue but run into a problem when trying to build the python bindings -- I'm getting an error from ld about a bad value in one of the static libraries.
Any thoughts?
root#0fcbc7b108af:/src/pjproject-2.5.5/pjsip-apps/src/python# python setup.py install
running install
running build
running build_py
running build_ext
building '_pjsua' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DPJ_AUTOCONF=1 -I/src/pjproject-2.5.5/pjlib/include -I/src/pjproject-2.5.5/pjlib-util/include -I/src/pjproject-2.5.5/pjnath/include -I/src/pjproject-2.5.5/pjmedia/include -I/src/pjproject-2.5.5/pjsip/include -I/usr/include/python2.7 -c _pjsua.c -o build/temp.linux-x86_64-2.7/_pjsua.o
_pjsua.c: In function 'py_pjsua_enum_transports':
_pjsua.c:1202:17: warning: variable 'status' set but not used [-Wunused-but-set-variable]
pj_status_t status;
^
_pjsua.c: In function 'py_pjsua_conf_get_port_info':
_pjsua.c:2338:9: warning: variable 'status' set but not used [-Wunused-but-set-variable]
int status;
^
_pjsua.c: In function 'py_pjsua_get_snd_dev':
_pjsua.c:2714:9: warning: variable 'status' set but not used [-Wunused-but-set-variable]
int status;
^
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/_pjsua.o -L/src/pjproject-2.5.5/pjlib/lib -L/src/pjproject-2.5.5/pjlib-util/lib -L/src/pjproject-2.5.5/pjnath/lib -L/src/pjproject-2.5.5/pjmedia/lib -L/src/pjproject-2.5.5/pjsip/lib -L/src/pjproject-2.5.5/third_party/lib -lpjsua-x86_64-unknown-linux-gnu -lpjsip-ua-x86_64-unknown-linux-gnu -lpjsip-simple-x86_64-unknown-linux-gnu -lpjsip-x86_64-unknown-linux-gnu -lpjmedia-codec-x86_64-unknown-linux-gnu -lpjmedia-x86_64-unknown-linux-gnu -lpjmedia-videodev-x86_64-unknown-linux-gnu -lpjmedia-audiodev-x86_64-unknown-linux-gnu -lpjmedia-x86_64-unknown-linux-gnu -lpjnath-x86_64-unknown-linux-gnu -lpjlib-util-x86_64-unknown-linux-gnu -lsrtp-x86_64-unknown-linux-gnu -lresample-x86_64-unknown-linux-gnu -lgsmcodec-x86_64-unknown-linux-gnu -lspeex-x86_64-unknown-linux-gnu -lilbccodec-x86_64-unknown-linux-gnu -lg7221codec-x86_64-unknown-linux-gnu -lyuv-x86_64-unknown-linux-gnu -lpj-x86_64-unknown-linux-gnu -lm -lrt -lpthread -lasound -o build/lib.linux-x86_64-2.7/_pjsua.so
/usr/bin/ld: /src/pjproject-2.5.5/pjsip/lib/libpjsua-x86_64-unknown-linux-gnu.a(pjsua_acc.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/src/pjproject-2.5.5/pjsip/lib/libpjsua-x86_64-unknown-linux-gnu.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

I was able to find an answer in an old mailing list entry (http://lists.pjsip.org/pipermail/pjsip_lists.pjsip.org/2008-August/004456.html).
Set CFLAGS=-fPIC and rebuild the PJSIP library (make clean && ./configure && make dep && make), then follow the directions for making the Python library.

Matplotlib 1.4.2 / 1.4.3 don't install from source

I have Matplotlib 1.3.1 installed on my Ubuntu 14.04 system and thought it was time to upgrade to 1.4.2. I never get the latest version for compatibility reasons (some packages that use Matplotlib might stop working). I then downloaded the tarball, uncompressed its contents into a folder and executed "sudo python setup.py build". However, I got the following error (last few lines only):
c++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/src/ft2font.o build/temp.linux-x86_64-2.7/src/mplutils.o build/temp.linux-x86_64-2.7/extern/CXX/cxxsupport.o build/temp.linux-x86_64-2.7/extern/CXX/cxx_extensions.o build/temp.linux-x86_64-2.7/extern/CXX/IndirectPythonInterface.o build/temp.linux-x86_64-2.7/extern/CXX/cxxextensions.o -L/usr/local/lib -L/usr/local/lib64 -lfreetype -lstdc++ -lm -o build/lib.linux-x86_64-2.7/matplotlib/ft2font.so
/usr/bin/ld: /usr/local/lib64/libstdc++.a(si_class_type_info.o): relocation R_X86_64_32S against `_ZTVN10__cxxabiv120__si_class_type_infoE' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib64/libstdc++.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
error: command 'c++' failed with exit status 1
I then tried the same with Matplotlib 1.4.3 (why not?) and got the same results.
What's going on here?
Thanks so much for any help.
Fausto

Cython compilation include his own target

I'm having troubles with an odd error. I use Cython to compile some c++ files. It works perfectly fine under OS X but under Linux i have the following error.
clang++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/test.o build/temp.linux-x86_64-2.7/ocv.o build/temp.linux-x86_64-2.7/fructueux.o build/temp.linux-x86_64-2.7/Face_keypoint_extractor_minimal.o build/temp.linux-x86_64-2.7/tools/mimic.o build/temp.linux-x86_64-2.7/Unrestricted_local_binary_pattern_modified.o build/temp.linux-x86_64-2.7/Over_sampling.o -L -o /home/AM/Documents/code_dw/Python_deep_dev/Cython_interface/First_import/test.so -L/usr/local/lib -lopencv_calib3d -lopencv_contrib -lopencv_core -lopencv_features2d -lopencv_flann -lopencv_gpu -lopencv_highgui -lopencv_imgproc -lopencv_legacy -lopencv_ml -lopencv_nonfree -lopencv_objdetect -lopencv_ocl -lopencv_photo -lopencv_stitching -lopencv_superres -lopencv_ts -lopencv_video -lopencv_videostab -ltbb -lrt -lpthread -lm -ldl -lboost_system -lboost_filesystem ./libFD.a
=> -L -o /home/AM/Documents/code_dw/Python_deep_dev/Cython_interface/First_import/test.so
clang: error: no such file or directory: '/home/AM/Documents/code_dw/Python_deep_dev/Cython_interface/First_import/test.so'
The -o is perfectly normal but i don't know the effect of the -L just before and i guess it is the reason it crashes.
Does someone has an idea of the reason why distutils and/or clang could produce such an output ?
PS : I have never written the string "test.so" in any of my file.

Stupid but it always happens
in the setup.py :
library_dirs=[""]
Create the -L in the compiler command and the -L ignore the -o and get the next path.