Develop Reference

Reportlab installation failed after upgrading to Macos Big Sur

I'm trying to reinstall my virtual env after upgrade to MacOS Big Sur.
But error appears:
4 warnings generated.
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -Qunused-arguments -Qunused-arguments -DRENDERPM_FT -DLIBART_COMPILATION -DLIBART_VERSION=2.3.21 -Isrc/rl_addons/renderPM -Isrc/rl_addons/renderPM/libart_lgpl -Isrc/rl_addons/renderPM/gt1 -I/usr/local/include/freetype2 -I/usr/local/include -I/usr/local/opt/openssl#1.1/include -I/usr/local/opt/sqlite/include -I/Users/zulfugar/PycharmProjects/AppForm/.venv/include -I/usr/local/Cellar/python#3.8/3.8.6_1/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c src/rl_addons/renderPM/gt1/gt1-parset1.c -o build/temp.macosx-11.0-x86_64-3.8/src/rl_addons/renderPM/gt1/gt1-parset1.o
src/rl_addons/renderPM/gt1/gt1-parset1.c:604:28: warning: for loop has empty body [-Wempty-body]
for (i = 0; i < size; i++);
^
src/rl_addons/renderPM/gt1/gt1-parset1.c:604:28: note: put the semicolon on a separate line to silence this warning
src/rl_addons/renderPM/gt1/gt1-parset1.c:1907:16: warning: comparison of integers of different signs: 'int' and 'unsigned long' [-Wsign-compare]
for (i = 0; i < sizeof(internal_procs) / sizeof(InternalGt1ProcListing); i++)
~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/rl_addons/renderPM/gt1/gt1-parset1.c:710:1: warning: function 'print_value_deep' is not needed and will not be emitted [-Wunneeded-internal-declaration]
print_value_deep (Gt1PSContext *psc, Gt1Value *val, int nest)
^
3 warnings generated.
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -Qunused-arguments -Qunused-arguments -DRENDERPM_FT -DLIBART_COMPILATION -DLIBART_VERSION=2.3.21 -Isrc/rl_addons/renderPM -Isrc/rl_addons/renderPM/libart_lgpl -Isrc/rl_addons/renderPM/gt1 -I/usr/local/include/freetype2 -I/usr/local/include -I/usr/local/opt/openssl#1.1/include -I/usr/local/opt/sqlite/include -I/Users/zulfugar/PycharmProjects/AppForm/.venv/include -I/usr/local/Cellar/python#3.8/3.8.6_1/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c src/rl_addons/renderPM/gt1/gt1-dict.c -o build/temp.macosx-11.0-x86_64-3.8/src/rl_addons/renderPM/gt1/gt1-dict.o
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -Qunused-arguments -Qunused-arguments -DRENDERPM_FT -DLIBART_COMPILATION -DLIBART_VERSION=2.3.21 -Isrc/rl_addons/renderPM -Isrc/rl_addons/renderPM/libart_lgpl -Isrc/rl_addons/renderPM/gt1 -I/usr/local/include/freetype2 -I/usr/local/include -I/usr/local/opt/openssl#1.1/include -I/usr/local/opt/sqlite/include -I/Users/zulfugar/PycharmProjects/AppForm/.venv/include -I/usr/local/Cellar/python#3.8/3.8.6_1/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c src/rl_addons/renderPM/gt1/gt1-namecontext.c -o build/temp.macosx-11.0-x86_64-3.8/src/rl_addons/renderPM/gt1/gt1-namecontext.o
src/rl_addons/renderPM/gt1/gt1-namecontext.c:100:9: error: implicitly declaring library function 'strlen' with type 'unsigned long (const char *)' [-Werror,-Wimplicit-function-declaration]
len = strlen (s);
^
src/rl_addons/renderPM/gt1/gt1-namecontext.c:100:9: note: include the header <string.h> or explicitly provide a declaration for 'strlen'
src/rl_addons/renderPM/gt1/gt1-namecontext.c:102:3: error: implicitly declaring library function 'memcpy' with type 'void *(void *, const void *, unsigned long)' [-Werror,-Wimplicit-function-declaration]
memcpy (new, s, len);
^
src/rl_addons/renderPM/gt1/gt1-namecontext.c:102:3: note: include the header <string.h> or explicitly provide a declaration for 'memcpy'
src/rl_addons/renderPM/gt1/gt1-namecontext.c:172:10: error: implicitly declaring library function 'strcmp' with type 'int (const char *, const char *)' [-Werror,-Wimplicit-function-declaration]
if (!strcmp (nc->table[i & mask].name, name))
^
src/rl_addons/renderPM/gt1/gt1-namecontext.c:172:10: note: include the header <string.h> or explicitly provide a declaration for 'strcmp'
3 errors generated.
error: command 'clang' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /Users/zulfugar/PycharmProjects/AppForm/.venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/_w/8xmqn_ys20n91w63y9lcl6nr0000gn/T/pip-install-_q2xgbny/reportlab/setup.py'"'"'; __file__='"'"'/private/var/folders/_w/8xmqn_ys20n91w63y9lcl6nr0000gn/T/pip-install-_q2xgbny/reportlab/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/_w/8xmqn_ys20n91w63y9lcl6nr0000gn/T/pip-record-08bq_70o/install-record.txt --single-version-externally-managed --compile --install-headers /Users/zulfugar/PycharmProjects/AppForm/.venv/include/site/python3.8/reportlab Check the logs for full command output.
I tried:
Reinstall xcode-select --install;
Reinstalled Python brew install python#3.8;
Installed Xcode itself;
None of above helped.
What can be the issue?

this worked for me
CFLAGS="-Wno-error=implicit-function-declaration" pip install reportlab

I found that updating my pip requirements.txt pinned version of reportlab from 3.3.x to 3.5.59 (latest as of today) resolved this issue for me.
I have not yet done any validation of the changes between those versions. YMMV!

Fixed issue by downloading source code and updating src/rl_addons/renderPM/gt1/gt1-namecontext.c
There was
#if defined(_WIN32) || defined(macintosh)
# include <string.h>
#endif
Seems macintosh is not defined on MacOS Big Sur, added include <string.h> without if statement and build using python setup.py install
Worked.

Linker flag positions in CMake 3.5

I have a pretty strange question: how do I make CMake 3.5 put the linker flags AFTER the files?
I am trying to compile a C++ which calls up a Python script. I have these two lines in my CMakeList.cpp :
add_executable (Python_trivium PyCall.cpp)
set_target_properties (Python_trivium PROPERTIES COMPILE_FLAGS "${PYTHON_CFLAGS}" Python_trivium PRIVATE "${PYTHON_LDFLAGS}")
This generates a Linking error. With "make VERBOSE=1" I can see that the linker command is as follows:
/usr/bin/c++ -L/usr/lib/python2.7/config-x86_64-linux-gnu -L/usr/lib -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions CMakeFiles/Python_trivium.dir/PyCall.cpp.o -o Python_trivium
Where the flags are BEFORE the files. I get a "undefined reference" error for each command from Python.h, such as
/home/portolan/emanuele/MAST_project/pyhelper.hpp:12: undefined reference to `Py_Initialize'
If I invert and put the flags AFTER the files everything works:
/usr/bin/c++ CMakeFiles/Python_trivium.dir/PyCall.cpp.o -o Python_trivium -L/usr/lib/python2.7/config-x86_64-linux-gnu -L/usr/lib -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions
Is there a way to ask CMake to format the linker command line this way?
Thanks,
Michele

I discovered that my error was in trying to do it manually. CMake can do it better for me:
find_package(PythonLibs REQUIRED)
include_directories(${PYTHON_INCLUDE_DIRS})
message("PYTHONLIBS_VERSION_STRING: ${PYTHONLIBS_VERSION_STRING}")
Unfortunately it finds 3.5, while I would need 2.7....

boost_python hello example not working

I am trying to get the hello world example for boost python working. I am using OSX, boost 1.55 and python 2.7
Here is my hello.cpp
#include <boost/python/module.hpp>
#include <boost/python/def.hpp>
char const* greet()
{
return "hello, world";
}
BOOST_PYTHON_MODULE(hello_ext)
{
using namespace boost::python;
def("greet", greet);
}
and I have compiled it with the following two lines:
g++ -fPIC -I/usr/include/python2.7/ -I/usr/local/include/ -c hello.cpp
g++ -shared -Wl, -o hello.so hello.o -L/usr/lib -L/usr/local/lib -lpython2.7 -lboost_python
When I try to import it into python by doing import hello.so I get the following error:
ImportError: dynamic module does not define init function (inithello)
Any ideas?

Turns out the name in the BOOST_PYTHON_MODULE must match the name of the library so I changed the linking step to
g++ -shared -Wl, -o hello_ext.so hello.o -L/usr/lib -L/usr/local/lib -lpython2.7 -lboost_python

Embedding Python in C, linking fails with undefined reference to `Py_Initialize'

I am trying to compile the example from the docs https://docs.python.org/2.7/extending/embedding.html and my code looks exactly like the one under 5.1:
#include <Python.h>
int
main(int argc, char *argv[])
{
Py_SetProgramName(argv[0]);
Py_Initialize();
PyRun_SimpleString("from time import time, ctime\n"
"print 'Today is', ctime(time())\n");
Py_Finalize();
return 0;
}
I use the following command to compile it which works fine for me and gives me the desired object file:
gcc -c $(python2.7-config --cflags) embedpy.c
To link it I use the following command which ends up in the following error:
gcc $(/usr/bin/python2.7-config --ldflags) embedpy.o
embedpy.o: In function `main':
/home/miguellissimo/embedpy.c:6: undefined reference to `Py_SetProgramName'
/home/miguellissimo/embedpy.c:7: undefined reference to `Py_Initialize'
/home/miguellissimo/embedpy.c:8: undefined reference to `PyRun_SimpleStringFlags'
/home/miguellissimo/embedpy.c:11: undefined reference to `Py_Finalize'
collect2: error: ld returned 1 exit status
I can't find out what I am doing wrong or what I forget to get the example working.
PS: The python2.7-config command gives the following output on my Xubuntu machine:
>>> python2.7-config --cflags
-I/usr/include/python2.7 -I/usr/include/x86_64-linux-gnu/python2.7 -fno-stri
ct-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=
4 -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-pr
ototypes
>>> python2.7-config --ldflags
-L/usr/lib/python2.7/config-x86_64-linux-gnu -L/usr/lib -lpthread -ldl -luti
l -lm -lpython2.7 -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions

Libraries have to come after the object files when you are linking, so do:
gcc embedpy.o $(/usr/bin/python2.7-config --ldflags)

Also add --embed to python3-config
On Ubuntu 20.04, Python 3.8, I also needed to pass --embed to python3-config as in:
gcc -std=c99 -ggdb3 -O0 -pedantic-errors -Wall -Wextra \
-fpie $(python3-config --cflags --embed) -o 'eval.out' \
'eval.c' $(python3-config --embed --ldflags)
otherwise -lpython3.8 is not added which leads to missing definitions.
This is my test program:
eval.c
#define PY_SSIZE_T_CLEAN
#include <Python.h>
int main(int argc, char *argv[]) {
(void)argc;
wchar_t *program = Py_DecodeLocale(argv[0], NULL);
if (program == NULL) {
fprintf(stderr, "Fatal error: cannot decode argv[0]\n");
exit(1);
}
Py_SetProgramName(program);
Py_Initialize();
PyRun_SimpleString(argv[1]);
if (Py_FinalizeEx() < 0) {
exit(120);
}
PyMem_RawFree(program);
return 0;
}
test run:
./eval.out 'print(2 ** 3)'

Got the same error on WSL, Ubuntu 18.04, Python3.8, g++ 7.5.
Thanks to Ciro Santilli comment, adding --embed option to python3.8-config solved unresolved symbols problem, but after that I got the following error:
g++ `python3.8-config --cflags --embed` -o cpython.out cpython.cpp `python3.8-config --ldflags --embed`
lto1: fatal error: bytecode stream in file ‘/home/rpovelik/installed/miniconda3/envs/cython/lib/python3.8/config-3.8-x86_64-linux-gnu/libpython3.8.a’ generated with LTO version 6.0 instead of the expected 6.2
compilation terminated.
lto-wrapper: fatal error: g++ returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
I suppose some people are going to face the same issue so I inspected the similar symptoms here where conda people said something like "don't use default compilers - g++ - because it could cause compatibility issues - use conda specific compilers".
By the way, adding -fno-lto solved my problem with system-wide g++ 7.5. Possibly you can try to change version of the compiler.

I replicated the same problem on Lubuntu14.04 32bit virtual machine running on Win7 machine.
I did the following things to replicate the problem in the first place where the C++ code is written in Eclipse C/C++ IDE. Defined a C++ Eclipse project titled "test". The source file contains the following C++ code which is the same as mentioned above by Miguellissimo.
C++ Code ::
#include "python2.7/Python.h"
int main(int argc, char *argv[]) {
Py_Initialize();
PyRun_SimpleString("print \"Hello, world!\"");
Py_Finalize();
return 0;
}
Errors ::
test.o: In function main':
/home/ros/workspace/test/src/test.cpp:15: undefined reference toPy_Initialize'
/home/ros/workspace/test/src/test.cpp:17: undefined reference to PyRun_SimpleStringFlags'
/home/ros/workspace/test/src/test.cpp:18: undefined reference toPy_Finalize'
collect2: error: ld returned 1 exit status
The output of the following commands was the same as mentioned before by Miguellissimo,
ros#rosPC:~/workspace/test/src$ python2.7-config --cflags
-I/usr/include/python2.7 -I/usr/include/i386-linux-gnu/python2.7 -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
ros#rosPC:~/workspace/test/src$ python2.7-config --ldflags
-L/usr/lib/python2.7/config-i386-linux-gnu -L/usr/lib -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions
Project Properties of C++ Eclipse Project titled "test"::
Inside, I had the following C++ Build Settings
GCC C++ Compiler
Command: g++
All options: -I/opt/ros/indigo/include -O0 -g3 -Wall -c -fmessage-length=0
Command line pattern: ${COMMAND} ${FLAGS} ${OUTPUT_FLAG} ${OUTPUT_PREFIX}${OUTPUT} ${INPUTS}
Includes
include paths: /opt/ros/indigo/include
GCC C Compiler
Command: gcc
All options: -I/opt/ros/indigo/include -I/usr/lib/python2.7/config-i386-linux-gnu -O0 -g3 -Wall -c -fmessage-length=0
Command line pattern: ${COMMAND} ${FLAGS} ${OUTPUT_FLAG} ${OUTPUT_PREFIX}${OUTPUT} ${INPUTS}
Includes
include paths: /opt/ros/indigo/include
/usr/lib/python2.7/config-i386-linux-gnu
GCC C++ Linker
Command: g++
All options: Command line pattern: ${COMMAND} ${FLAGS} ${OUTPUT_FLAG} ${OUTPUT_PREFIX}${OUTPUT} ${INPUTS}
Libraries:
Libraries(-I):
Library search path(-L):
Solution::
Specified the following C++ Build Settings in the Project Properties of C++ Eclipse Project "test"
GCC C++ Compiler
Command: g++
All options: -I/opt/ros/indigo/include -I/usr/include/python2.7 -O0 -g3 -Wall -c -fmessage-length=0
Command line pattern: ${COMMAND} ${FLAGS} ${OUTPUT_FLAG} ${OUTPUT_PREFIX}${OUTPUT} ${INPUTS}
Includes
include paths: /opt/ros/indigo/include
/usr/include/python2.7
GCC C Compiler
Command: gcc
All options: -I/opt/ros/indigo/include -I/usr/include/python2.7 -O0 -g3 -Wall -c -fmessage-length=0
Command line pattern: ${COMMAND} ${FLAGS} ${OUTPUT_FLAG} ${OUTPUT_PREFIX}${OUTPUT} ${INPUTS}
Includes
include paths: /opt/ros/indigo/include
/usr/include/python2.7
GCC C++ Linker
Command: g++
All options: -L/usr/lib/python2.7/config-i386-linux-gnu
Command line pattern: ${COMMAND} ${FLAGS} ${OUTPUT_FLAG} ${OUTPUT_PREFIX}${OUTPUT} ${INPUTS}
Libraries:
Libraries(-I): python2.7
Library search path(-L): /usr/lib/python2.7/config-i386-linux-gnu
Result :: Linker errors which resulted previously from the compilation of C++ Eclipse project doesn't occur anymore.

The accepted answer uses system's default library location.
If you are within an anaconda environment, library location may not be found. I tend to give locations more explicitly during linking, like:
gcc embedpy.o -L$(python-config --prefix)/lib $(python-config --ldflags)
This will not work outside anaconda environment in 64-bit library, in which case:
gcc embedpy.o -L$(python-config --prefix)/lib64 $(python-config --ldflags)

Linking GSL (or other library) statically into a shared library

Note: Despite the mentioning of Python in the following there is a good chance for my problem not to be Python related at all. If I am not mistaken the “module” I mention is equivalent to a C library—at least for the concerns of my problem.
On Debian I am trying to create a Python module with C, which in turn uses the GSL. The following Makefile successfully compiles it:
CC = gcc -Wall -fPIC -O3
NAME = meinzeug
matrizenwuerfler: $(SRC)
$(CC) -o $(NAME).o -I/usr/lib/python2.5/site-packages/numpy/core/include -I/usr/include/python2.5 -c $(NAME).c
$(CC) -shared -o $(NAME).so -lgsl -lgslcblas -lm $(NAME).o
Because this module is supposed to be used by (Linux) machines other than mine, I want the GSL to be included into the module (or be shipped with it).
However, if I add -static as option to the last line of the Makefile, I get the following error:
gcc -Wall -fPIC -O3 -shared -static -o meinzeug.so -lgsl -lgslcblas -lm meinzeug.o
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.3.2/crtbeginT.o: relocation R_X86_64_32 against `__DTOR_END__' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/x86_64-linux-gnu/4.3.2/crtbeginT.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
Adding -Wl,-Bstatic before the library linking results in a different error:
gcc -Wall -fPIC -O3 -shared -o meinzeug.so -Wl,-Bstatic -lgsl -lgslcblas -lm meinzeug.o
/usr/bin/ld: cannot find -lgcc_s
collect2: ld returned 1 exit status
Other Stuff, that did not work: Recompiling GSL with fPIC, -static-libgcc, permutating the options.
What I did not try yet, is compiling gcc with fPIC or similar.

Try
gcc -Wall -fPIC -O3 -shared -o meinzeug.so /usr/lib/libgsl.a -lm meinzeug.
as you cannot do
gcc -Wall -fPIC -O3 -shared -static ... # shared and static at the same time ?
so you would provide the static library of GSL alongside with your code.
At the end of the day, I would punt and keep the dependency on the GSL. Just about everybody has it, and the API is pretty stable.

The ordering of the library calls is important. For me, it meant sending the /usr/lib/libgsl.a to the end of the command. That solved it.