Develop Reference

Embedded python fails to compile on Raspberry Pi

I installed python 3.9.1 on my Raspberry Pi following the instructions here https://www.ramoonus.nl/2020/10/06/how-to-install-python-3-9-on-raspberry-pi/ and set it as the default python interpreter. I got my compiling and linking parameters for embedded Python following the instructions here https://docs.python.org/3.9/extending/embedding.html#compiling-and-linking-under-unix-like-systems
I tried a simple test with the following code (test.c) :
#include <Python.h>
int
main(int argc, char *argv[])
{
wchar_t *program = Py_DecodeLocale(argv[0], NULL);
if (program == NULL) {
fprintf(stderr, "Fatal error: cannot decode argv[0]\n");
exit(1);
}
Py_SetProgramName(program); /* optional but recommended */
Py_Initialize();
PyRun_SimpleString("from time import time,ctime\n"
"print('Today is', ctime(time()))\n");
Py_Finalize();
PyMem_RawFree(program);
return 0;
}
and then
gcc -I/usr/local/opt/python-3.9.1/include/python3.9 -I/usr/local/opt/python-3.9.1/include/python3.9 -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -c test.c -o test.o
and
gcc -L/usr/local/opt/python-3.9.1/lib/python3.9/config-3.9-arm-linux-gnueabihf -L/usr/local/opt/python-3.9.1/lib -lcrypt -lpthread -ldl -lutil -lm -o test.o
and got
/usr/lib/gcc/arm-linux-gnueabihf/4.9/../../../arm-linux-gnueabihf/crt1.o: In function '_start': /build/glibc-P1SmLh/glibc-2.19/csu/../ports/sysdeps/arm/start.S:119: undefined reference to 'main' collect2: error: ld returned 1 exit status
Trying to compile the example at https://docs.python.org/3.9/extending/embedding.html#pure-embedding throws the same error. What could the problem be?
Edit:
After Expolarity's comment I changed the linker command to:
gcc test.o -L/usr/local/opt/python-3.9.1/lib/python3.9/config-3.9-arm-linux-gnueabihf -L/usr/local/opt/python-3.9.1/lib -lcrypt -lpthread -ldl -lutil -lm -o test
which seems to have worked but threw me a bunch of other errors:
test.o: In function main': /home/pi/Downloads/test.c:6: undefined reference to Py_DecodeLocale'
/home/pi/Downloads/test.c:11: undefined reference to Py_SetProgramName' /home/pi/Downloads/test.c:12: undefined reference to Py_Initialize'
/home/pi/Downloads/test.c:13: undefined reference to PyRun_SimpleStringFlags' /home/pi/Downloads/test.c:15: undefined reference to Py_Finalize'
/home/pi/Downloads/test.c:16: undefined reference to `PyMem_RawFree'
collect2: error: ld returned 1 exit status
This seems more serious. Any ideas?

After tttapa's answer over here it finally worked by adjusting the linker command as so:
gcc test.o -L/usr/local/opt/python-3.9.1/lib/python3.9/config-3.9-arm-linux-gnueabihf -L/usr/local/opt/python-3.9.1/lib -lcrypt -lpthread -ldl -lutil -lm -lpython3.9 -o test
Edit: Expolarity also answered correctly just after tttapa. Thanks a lot everyone!

Error while cross compiling Tensorflow for Raspberry Pi 3 On ubuntu 16.04lts

I'm trying to cross compile Tensorflow for Raspberry PI3 with help of Tensorflow build_raspberry_pi.sh but I'm getting error and I have followed the below steps:-
1: Installing Bazel 0.10.0,
2: Cloning Tensorflow from git and doing a checkout of r1.5,
3: Running the script present inside /tensorflow/tools/ci_build/pi/, The script is build_raspberry_pi.sh.
I'm cross compiling it for raspberry Pi 3 and getting the below error during the compilation:
ERROR: /home/baladev/.cache/bazel/_bazel_baladev/da0e175f87e998fe3d550279550cec2c/external/grpc/BUILD:431:1: C++ compilation of rule '#grpc//:gpr_base' failed (Exit 1): arm-linux-gnueabihf-gcc failed: error executing command
(cd /home/baladev/.cache/bazel/_bazel_baladev/da0e175f87e998fe3d550279550cec2c/execroot/org_tensorflow && \ exec env - \
PATH=/home/baladev/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin \
PWD=/proc/self/cwd \
PYTHON_BIN_PATH=/usr/bin/python \
PYTHON_LIB_PATH=/usr/local/lib/python2.7/dist-packages \
TF_NEED_CUDA=0 \
TF_NEED_OPENCL_SYCL=0 \ /home/baladev/.cache/bazel/_bazel_baladev/da0e175f87e998fe3d550279550cec2c/external/arm_compiler/bin/arm-linux-gnueabihf-gcc
-U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -DRASPBERRY_PI -g0 -O2 -DNDEBUG -ffunction-sections -fdata-sections -DGEMMLOWP_ALLOW_SLOW_SCALAR_FALLBACK '-march=armv7-a' '-mfpu=neon-vfpv4' '-std=gnu11' '-DS_IREAD=S_IRUSR' '-DS_IWRITE=S_IWUSR' -O3 -U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_1
-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_2 -U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_8 -funsafe-math-optimizations -ftree-vectorize -fomit-frame-pointer '-std=c++11' -isystem /usr/include/arm-linux-gnueabihf -isystem /usr/include/python2.7
-isystem /usr/include/ -MD -MF bazel-out/armeabi-opt/bin/external/grpc/_objs/gpr_base/external/grpc/src/core/lib/support/subprocess_posix.pic.d '-frandom-seed=bazel-out/armeabi-opt/bin/external/grpc/_objs/gpr_base/external/grpc/src/core/lib/support/subprocess_posix.pic.o'
-fPIC '-DGRPC_ARES=0' -D__CLANG_SUPPORT_DYN_ANNOTATION__ -iquote external/grpc -iquote bazel-out/armeabi-opt/genfiles/external/grpc
-iquote external/bazel_tools -iquote bazel-out/armeabi-opt/genfiles/external/bazel_tools -iquote external/com_google_absl -iquote bazel-out/armeabi-opt/genfiles/external/com_google_absl -isystem external/grpc/include -isystem bazel-out/armeabi-opt/genfiles/external/grpc/include -isystem external/bazel_tools/tools/cpp/gcc3 -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -no-canonical-prefixes
-fno-canonical-system-headers -c external/grpc/src/core/lib/support/subprocess_posix.cc -o bazel-out/armeabi-opt/bin/external/grpc/_objs/gpr_base/external/grpc/src/core/lib/support/subprocess_posix.pic.o) cc1plus: warning: command line option '-std=gnu11' is valid for C/ObjC but not for C++ In file included from external/grpc/src/core/lib/support/subprocess_posix.cc:33:0: /home/baladev/.cache/bazel/_bazel_baladev/da0e175f87e998fe3d550279550cec2c/external/arm_compiler/bin/../arm-linux-gnueabihf/sysroot/usr/include/sys/wait.h:101:3: error: redeclaration of 'P_ALL' P_ALL, /* Wait for any child. */ ^ In file included from /usr/include/stdlib.h:41:0,
from external/grpc/src/core/lib/support/subprocess_posix.cc:30: /usr/include/bits/waitflags.h:52:3: note: previous declaration 'idtype_t P_ALL' P_ALL, /* Wait for any child. */ ^ In file included from external/grpc/src/core/lib/support/subprocess_posix.cc:33:0: /home/baladev/.cache/bazel/_bazel_baladev/da0e175f87e998fe3d550279550cec2c/external/arm_compiler/bin/../arm-linux-gnueabihf/sysroot/usr/include/sys/wait.h:102:3: error: redeclaration of 'P_PID' P_PID, /* Wait for specified process. */ ^ In file included from /usr/include/stdlib.h:41:0,
from external/grpc/src/core/lib/support/subprocess_posix.cc:30: /usr/include/bits/waitflags.h:53:3: note: previous declaration 'idtype_t P_PID' P_PID, /* Wait for specified process. */ ^ In file included from external/grpc/src/core/lib/support/subprocess_posix.cc:33:0: /home/baladev/.cache/bazel/_bazel_baladev/da0e175f87e998fe3d550279550cec2c/external/arm_compiler/bin/../arm-linux-gnueabihf/sysroot/usr/include/sys/wait.h:103:3: error: redeclaration of 'P_PGID' P_PGID /* Wait for members of process group. */ ^ In file included from /usr/include/stdlib.h:41:0,
from external/grpc/src/core/lib/support/subprocess_posix.cc:30: /usr/include/bits/waitflags.h:54:3: note: previous declaration 'idtype_t P_PGID' P_PGID /* Wait for members of process group. */ ^ In file included from external/grpc/src/core/lib/support/subprocess_posix.cc:33:0: /home/baladev/.cache/bazel/_bazel_baladev/da0e175f87e998fe3d550279550cec2c/external/arm_compiler/bin/../arm-linux-gnueabihf/sysroot/usr/include/sys/wait.h:104:3: error: conflicting declaration 'typedef enum idtype_t idtype_t' } idtype_t; ^ In file included from /usr/include/stdlib.h:41:0,
from external/grpc/src/core/lib/support/subprocess_posix.cc:30: /usr/include/bits/waitflags.h:55:3: note: previous declaration as 'typedef enum idtype_t idtype_t' } idtype_t; ^ INFO: Elapsed time:
984.156s, Critical Path: 40.58s FAILED: Build did NOT complete successfully
Am I missing something which is needed for cross compilation?

Embedding Python in C: Error in linking - undefined reference to PyString_AsString

I am trying to embed a python program inside a C program. My OS is Ubuntu 14.04
I try to embed python 2.7 and python 3.4 interpreter in the same C code base (as separate applications). The compilation and linking works when embedding python 2.7 but not for the python 3.4. It fails during the linker stage.
Here is my C code (just an example not real code)
simple.c
#include <stdio.h>
#include <Python.h>
int main(int argc, char *argv[])
{
PyObject *pName, *pModule, *pFunc, *pValue;
char module[] = "get_version";
char func[] = "get_version";
char module_path[] = ".";
Py_Initialize();
PyObject *sys_path = PySys_GetObject("path");
PyList_Append(sys_path, PyUnicode_FromString(module_path));
pName = PyUnicode_FromString(module);
pModule = PyImport_Import(pName);
Py_DECREF(pName);
if(pModule != NULL)
{
pFunc = PyObject_GetAttrString(pModule, func);
if (pFunc && PyCallable_Check(pFunc))
{
pValue = PyObject_CallObject(pFunc, NULL);
if (pValue != NULL) {
printf("Python version: %s\n", PyString_AsString(pValue));
Py_DECREF(pValue);
}
else {
Py_DECREF(pFunc);
Py_DECREF(pModule);
PyErr_Print();
fprintf(stderr,"Call failed\n");
return 1;
}
}
}
Py_Finalize();
return 0;
}
get_version.py
import sys
def get_version():
version = '.'.join(str(v) for v in sys.version_info[:3])
print("version: ", version)
return version
I compile the program using gcc. First with compiling and linking flags set to python 2.7 I run the compilation and linking by using following command:
gcc `python-config --cflags` simple.c `python-config --ldflags`
The flags expand as:
python-config --cflags: -I/usr/include/python2.7 -I/usr/include/x86_64-linux-gnu/python2.7 -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
python-config --ldflags: -L/usr/lib/python2.7/config-x86_64-linux-gnu -L/usr/lib -lpthread -ldl -lutil -lm -lpython2.7 -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions
It works fine without any issues. When I try to compile the same with python3.4 flags it fails:
gcc `python3-config --cflags` simple.c `python3-config --ldflags`
The flags expand as:
python-config --cflags: -I/usr/include/python3.4m -I/usr/include/python3.4m -Wno-unused-result -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
python-config --ldflags: -L/usr/lib/python3.4/config-3.4m-x86_64-linux-gnu -L/usr/lib -lpython3.4m -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions
Error message:
simple.c: In function ‘main’:
simple.c:27:17: warning: implicit declaration of function ‘PyString_AsString’ [-Wimplicit-function-declaration]
printf("Python version: %s\n", PyString_AsString(pValue));
^
simple.c:27:17: warning: format ‘%s’ expects argument of type ‘char *’, but argument 2 has type ‘int’ [-Wformat=]
/tmp/ccaoMdTo.o: In function `main':
/home/vagrant/c_python_api/simple.c:27: undefined reference to `PyString_AsString'
collect2: error: ld returned 1 exit status
I tried by changing the order in which linker objects are specified. But no luck. Any idea why this would be the case?
Thanks for the help!!

Python 3 does not have PyString_AsString any more; the Python 3 str correspond to Python 2 unicode objects; the names of the functions for handling str in Python 3 are PyUnicode_-prefixed in the C-API.
Thus this line:
printf("Python version: %s\n", PyString_AsString(pValue));
could be changed to use PyUnicode_AsUTF8 on Python 3:
#if PY_MAJOR_VERSION >= 3
printf("Python version: %s\n", PyUnicode_AsUTF8(pValue));
#else
printf("Python version: %s\n", PyString_AsString(pValue));
#endif
(Not that passing NULL to printf %s will have undefined behaviour, so you'd want to check that a non-NULL pointer was returned)

python termination error when ctypes dll calls printf

I am developing a python system with some core dlls accessed via ctypes. I have reduced the problem to this condition: execute a module that loads (no need to call) two dlls, one of which calls printf -- this error will occur in exit.
This application has requested the Runtime to terminate it in an
unusual way. Please contact the application's support team for more
information.
My environment:
- Windows 7, SP1
- Python 2.7.8
- MinGW v 3.20
This test case is adapted from a tutorial on writing dlls with MinGW:
/* add_core.c */
__declspec(dllexport) int sum(int a, int b) {
return a + b;
}
/* sub_core.c */
#include <stdio.h>
__declspec(dllexport) int sum(int a, int b) {
printf("Hello from sub_core.c");
return a - b;
}
prog.py
import ctypes
add_core_dll = ctypes.cdll.LoadLibrary('add_core.dll')
add_core_dll = ctypes.cdll.LoadLibrary('sub_core.dll')
> make
gcc -Wall -O3 -g -ansi -c add_core.c -o add_core.o
gcc -g -L. -ansi -shared add_core.o -o add_core.dll
gcc -Wall -O3 -g -ansi -c sub_core.c -o sub_core.o
gcc -g -L. -ansi -shared sub_core.o -o sub_core.dll
>python prog.py
This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
and pops up a message dialog to the same effect: "python.exe has stopped working ...".
Note that the programs execute as expected and produce normal output. This error at termination is just a big nuisance I'd like to be rid of.

the same happens for:
Windows 7 Enterprise, SP1
Python 2.7.11
mingw32-g++.exe 5.3.0

Python extension: symbol(s) not found for architecture x86_64 error

I want to write a python extension in c. I work on Mac, I took a code from here:
#include <Python.h>
static PyObject* say_hello(PyObject* self, PyObject* args)
{
const char* name;
if (!PyArg_ParseTuple(args, "s", &name))
return NULL;
printf("Hello %s!\n", name);
Py_RETURN_NONE;
}
static PyMethodDef HelloMethods[] =
{
{"say_hello", say_hello, METH_VARARGS, "Greet somebody."},
{NULL, NULL, 0, NULL}
};
PyMODINIT_FUNC
inithello(void)
{
(void) Py_InitModule("hello", HelloMethods);
}
I compile it:
gcc -c -o py_module.o py_module.c -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/
gcc -o py_module py_module.o -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/ -lm
But I get this error:
Undefined symbols for architecture x86_64:
"_PyArg_ParseTuple", referenced from:
_say_hello in py_module.o
"_Py_InitModule4_64", referenced from:
_inithello in py_module.o
"__Py_NoneStruct", referenced from:
_say_hello in py_module.o
"_main", referenced from:
start in crt1.10.6.o
ld: symbol(s) not found for architecture x86_64
collect2: ld returned 1 exit status
make: *** [py_module] Error 1
How comes python doesn't support X86_64 architecture?

Two things:
You need to link your extension as a shared object (you're attempting to link an executable, which is why the linker is looking for main());
You need to link against the Python static library (-lpython).

Query the python env path with these commands:
$python-config --includes
-I/usr/include/python2.6 -I/usr/include/python2.6
$python-config --ldflags
-lpthread -ldl -lutil -lm -lpython2.6
generate .o file:
$ g++ -fPIC -c -I/usr/include/python2.6 -I/usr/include/python2.6 xx.cpp
generate .so file:
g++ -shared xx.o -o xx.so

Thanks to #NPE #glglgl and anatoly here is my Makefile:
DIR=/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/
CC=gcc
CFLAGS=-I$(DIR)
ODIR=.
LIBS_DIR=/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/
LIBS=-lpython2.7
_DEPS =
DEPS = $(patsubst %,$(IDIR)/%,$(_DEPS))
_OBJ = py_module.o
OBJ = $(patsubst %,$(ODIR)/%,$(_OBJ))
$(ODIR)/%.o: %.c $(DEPS)
$(CC) -c -o $# $< $(CFLAGS)
py_module: $(OBJ)
gcc -shared $^ $(CFLAGS) -I$(LIBS_DIR) $(LIBS) -o $#
.PHONY: clean
clean:
rm -f $(ODIR)/*.o *~ core $(INCDIR)/*~
makefile template had been taken from here.
In order to find the paths, one may use python-config --ldflags
and python-config --includes