Note: Despite the mentioning of Python in the following there is a good chance for my problem not to be Python related at all. If I am not mistaken the “module” I mention is equivalent to a C library—at least for the concerns of my problem.
On Debian I am trying to create a Python module with C, which in turn uses the GSL. The following Makefile successfully compiles it:
CC = gcc -Wall -fPIC -O3
NAME = meinzeug
matrizenwuerfler: $(SRC)
$(CC) -o $(NAME).o -I/usr/lib/python2.5/site-packages/numpy/core/include -I/usr/include/python2.5 -c $(NAME).c
$(CC) -shared -o $(NAME).so -lgsl -lgslcblas -lm $(NAME).o
Because this module is supposed to be used by (Linux) machines other than mine, I want the GSL to be included into the module (or be shipped with it).
However, if I add -static as option to the last line of the Makefile, I get the following error:
gcc -Wall -fPIC -O3 -shared -static -o meinzeug.so -lgsl -lgslcblas -lm meinzeug.o
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.3.2/crtbeginT.o: relocation R_X86_64_32 against `__DTOR_END__' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/x86_64-linux-gnu/4.3.2/crtbeginT.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
Adding -Wl,-Bstatic before the library linking results in a different error:
gcc -Wall -fPIC -O3 -shared -o meinzeug.so -Wl,-Bstatic -lgsl -lgslcblas -lm meinzeug.o
/usr/bin/ld: cannot find -lgcc_s
collect2: ld returned 1 exit status
Other Stuff, that did not work: Recompiling GSL with fPIC, -static-libgcc, permutating the options.
What I did not try yet, is compiling gcc with fPIC or similar.
Try
gcc -Wall -fPIC -O3 -shared -o meinzeug.so /usr/lib/libgsl.a -lm meinzeug.
as you cannot do
gcc -Wall -fPIC -O3 -shared -static ... # shared and static at the same time ?
so you would provide the static library of GSL alongside with your code.
At the end of the day, I would punt and keep the dependency on the GSL. Just about everybody has it, and the API is pretty stable.
The ordering of the library calls is important. For me, it meant sending the /usr/lib/libgsl.a to the end of the command. That solved it.
Related
I'm trying to compile C++ code with python embedded in it. The compiler does not compile my code with '-fPIE' flag even though I've added it in CMakeList.txt. Here's my CMakeList:
cmake_minimum_required(VERSION 3.3)
project(TestCython)
add_executable(main main.cpp)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD_REQUIRED True)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-unused-result -Wsign-compare -g -fdebug-prefix-map=/build/python3.8-6QL2k7/python3.8-3.8.2=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -fPIE")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-unused-result -Wsign-compare -g -fdebug-prefix-map=/build/python3.8-6QL2k7/python3.8-3.8.2=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -fPIE")
# Eigen
find_package(Eigen3 3.3 REQUIRED NO_MODULE)
# Embedded Python Includes
include_directories(/usr/include/python3.8
/usr/include/eigen3/ )
# Embedded Python Linker
link_directories(/usr/lib/python3.8/config-3.8-x86_64-linux-gnu
/usr/lib)
target_link_libraries( main
python3.8
crypt
pthread
dl
util
m )
set(CMAKE_C_COMPILER gcc)
set(CMAKE_CXX_COMPILER g++)
If I run the generated make file, it will report a lot of similar errors and ask me to recompile with '-fPIE', like this one:
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libpython3.8.a(call.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a PIE object; recompile with -fPIE
The compilation succeeds if I don't use cmake:
g++ -I/usr/include/python3.8 -Wno-unused-result -Wsign-compare -g -fdebug-prefix-map=/build/python3.8-6QL2k7/python3.8-3.8.2=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -fPIE main.cpp -o main -L/usr/lib/python3.8/config-3.8-x86_64-linux-gnu -L/usr/lib -lpython3.8 -lcrypt -lpthread -ldl -lutil -lm
Cmake version is 3.16.3. Python version is 3.8.2. System is Ubuntu 20.04
How can I solve those errors?
Note that most of the time this error has nothing to do with the PIE flag but is rather something else screwy about your build, unfortunately it’s a bit hard to tell because your CMake config is a bit screwy.
This particular error however is probably because you’re setting your properties incorrectly; Setting CMAKE_CXX_FLAGS and similar variables does nothing if you do it after defining your target with add_executable. This can be avoided entirely by using a more modern CMake style (out of scope for this answer, just google Modern CMake for some examples), in this particular case instead of setting flags on those variables you could for example do
target_compile_options(main PRIVATE -fpie)
I had similar error when I linked my library env2048 (having a file called ppo_buffer) to my bindings module env2048bindings:
[build] FAILED: python/env2048bindings.cpython-37m-x86_64-linux-gnu.so
[build] : && /usr/bin/g++-7 -fPIC -g -shared -o python/env2048bindings.cpython-37m-x86_64-linux-gnu.so python/CMakeFiles/env2048bindings.dir/env2048/bindings.cpp.o src/libenv2048.a -pthread && :
[build] /usr/bin/ld: src/libenv2048.a(ppo_buffer.cpp.o): relocation R_X86_64_PC32 against symbol `_ZSt3minIlERKT_S2_S2_' can not be used when making a shared object; recompile with -fPIC
[build] /usr/bin/ld: final link failed: Bad value
I had to set the properties for the library to make it work with:
set_target_properties(env2048 PROPERTIES POSITION_INDEPENDENT_CODE ON)
From these, I would make the guess that you might need this:
set_target_properties(python3.8 PROPERTIES POSITION_INDEPENDENT_CODE ON)
I have Matplotlib 1.3.1 installed on my Ubuntu 14.04 system and thought it was time to upgrade to 1.4.2. I never get the latest version for compatibility reasons (some packages that use Matplotlib might stop working). I then downloaded the tarball, uncompressed its contents into a folder and executed "sudo python setup.py build". However, I got the following error (last few lines only):
c++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/src/ft2font.o build/temp.linux-x86_64-2.7/src/mplutils.o build/temp.linux-x86_64-2.7/extern/CXX/cxxsupport.o build/temp.linux-x86_64-2.7/extern/CXX/cxx_extensions.o build/temp.linux-x86_64-2.7/extern/CXX/IndirectPythonInterface.o build/temp.linux-x86_64-2.7/extern/CXX/cxxextensions.o -L/usr/local/lib -L/usr/local/lib64 -lfreetype -lstdc++ -lm -o build/lib.linux-x86_64-2.7/matplotlib/ft2font.so
/usr/bin/ld: /usr/local/lib64/libstdc++.a(si_class_type_info.o): relocation R_X86_64_32S against `_ZTVN10__cxxabiv120__si_class_type_infoE' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib64/libstdc++.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
error: command 'c++' failed with exit status 1
I then tried the same with Matplotlib 1.4.3 (why not?) and got the same results.
What's going on here?
Thanks so much for any help.
Fausto
I'm trying to install pip install python-docx but get the following error. Why doesn't it work?
building 'lxml.etree' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/src
creating build/temp.linux-x86_64-2.7/src/lxml
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip_build_plakshma/lxml/src/lxml/includes -I/opt/python2.7/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w
gcc -pthread -shared build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -L/usr/lib64 -L. -lxslt -lexslt -lxml2 -lz -lm -lpython2.7 -o build/lib.linux-x86_64-2.7/lxml/etree.so
/usr/bin/ld: /usr/local/lib/libpython2.7.a(abstract.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/libpython2.7.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
Type this in your terminal/cmd.
pip install python-docx.
For me when I made typed it in the apple terminal it was an error but type in the Pycharm terminal and it should work.
I'm trying to cross-compile the pycrypto package, and I'm getting closer and closer however, I've hit an issue I just can't figure out.
I want distutils to use the cross-compile specific gcc- so I set the CC env var and it seems to respect the setting for the first invocation of the compiler, but thats it.
export CC="/opt/teeos/buildroot/output/host/usr/bin/i586-linux-gcc"
/opt/teeos/buildroot/output/host/usr/bin/i586-linux-gcc -fno-strict-aliasing -fwrapv -Wall -Wstrict-prototypes -fPIC -std=c99 --sysroot=/opt/teeos/buildroot/output/staging -I/opt/teeos/buildroot/output/staging/usr/include/python2.7 -O3 -fomit-frame-pointer -Isrc/ -I/usr/include/python2.7 -c src/_fastmath.c -o build/temp.linux-i686-2.7/src/_fastmath.o
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions build/temp.linux-i686-2.7/src/_fastmath.o -lgmp -o build/lib.linux-i686-2.7/Crypto/PublicKey/_fastmath.so
unable to execute gcc: No such file or directory
I temporarily moved my systems gcc so it can't be found.
How do I make distutils respect the CC=/opt/buildroot... option for every invocation of the compiler / set the path to the GCC / LD I want distutils to use?
This sounds similar to another answer I recently gave for customizing the distutils compiler. You'll also need to define LDSHARED which is the command used to produce the final shared object. See if this works:
>>> from distutils import sysconfig
>>> sysconfig.get_config_var('LDSHARED')
'gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions'
>>> sysconfig.get_config_var('CC')
'gcc -pthread'
Then replace gcc with your desired compiler and options in the CC and LDSHARED environment variables:
% LDSHARED="i586-linux-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions" \
CC="i586-linux-gcc -pthread" python setup.py build
Two problems I'm having with copperhead at the minute, which I suspect are related.
Running a sample file (samples/axpy.py) generated lots of little warnings, but this one stood out.
g++ -pthread -fno-strict-aliasing -g -O2 -g -fwrapv -O2 -Wall -fPIC -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions -DNDEBUG -I/usr/include/python2.6 -I/usr/local/lib/python2.6/dist-packages/copperhead-0.1a1-py2.6.egg/copperhead/include -I/usr/local/cuda/include /tmp/codepy-compiler-cache-v5-uid1000/202478034fea29b82d046b259bd6f43e/module.o /tmp/codepy-compiler-cache-v5-uid1000/fdcb04ede426b146cfce8894e99eeb57/gpu.o -L/usr/lib -L/usr/local/cuda/lib -L/usr/local/cuda/lib64 -lcuda -lcudart -lboost_python-gcc43-mt -lpthread -ldl -lutil -o /tmp/codepy-compiler-cache-v5-uid1000/202478034fea29b82d046b259bd6f43e/codepy.temp.202478034fea29b82d046b259bd6f43e.module.so
/usr/bin/ld: skipping incompatible /usr/local/cuda/lib/libcudart.so when searching for -lcudart
/usr/bin/ld: cannot find -lboost_python-gcc43-mt
collect2: ld returned 1 exit status
Few things to notice;
g++ is has correctly picked up the correct CUDA lib dir to use (lib64), but doesn't see libcudart.so in there, which it is.
/usr/local/cuda/lib64/libcudart.so.4.0.12: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, stripped
Next, boost_python isn't found; I've heard from a few places that this is due to ubuntu breaking a package convention somewhere, and people has said they've fixed it by changing the pycuda configure options, but haven't said what to change it to... (Example)
Anyone got any ideas for either the cudart or boot issues?
It is finding libcudart. What you
are seeing is only an informational
warning: -L options are
searched in order and the linker is
finding the 32 bit version first,
because you gave
-L/usr/local/cuda/lib before -L/usr/loca/cuda/lib64.
For the libboost_python problem,
just link with -lboost_python. The
Ubuntu systems I use (64 bit
10.04LTS with boost-python 1.40) have a series of cascading symbolic
links to that canonical
library name that make the linker
find the correct library without any
further intervention.