I am running Python 3.4.4 :: Anaconda 4.0.0 (x86_64) on OS X Yosemite. My Cython's version is 0.23.4. I'm trying to embed some very trivial Cython code test.pyx into C code testcc.c. The problem is, if I use python2.7-config then everything works well (Python 2.7 is the built-in version on OS X). However if I use python3.4-config the following errors raised:
Undefined symbols for architecture x86_64:
"_inittest", referenced from:
_main in testcc-b22dcf.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
I have to use Python3 since all my other codes are written in it. Please help me solve this problem.
The following are my source files:
test.pyx:
cdef public void pythonAdd(int[] a):
a[1] = 5
a[0] = 4
testcc.c:
#include "Python.h"
#include "test.h"
#include <math.h>
#include <stdio.h>
int main(int argc, char **argv) {
Py_Initialize();
inittest();
int a [2] = {0 , 0};
pythonAdd(a);
printf("fist: %d, second: %d", a[0], a[1]);
Py_Finalize();
return 0;
}
And Compiling those two files using following setup.py:
from distutils.core import setup, Extension
from Cython.Build import cythonize
ext = Extension("testc", sources=["test.pyx"])
setup(name="testc", ext_modules=cythonize(ext))
The following is the command I compile those c files:
ldflags:=$(shell $(python3.4-config) --ldflags)
cflags:=$(shell $(python3.4-config) --cflags)
python setup.py build_ext --inplace
cython test.pyx
gcc $(cflags) $(ldflags) test.c testcc.c -o cysvm.out
Update:
I changed the inittest() to PyInit_test() as Jim suggested. The code compiles successfully. However when I run ./cysvm.out the following errors occured:
./cysvm.out
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: Py_Initialize: unable to load the file system codec
ImportError: No module named 'encodings'
Current thread 0x00007fff772f5300 (most recent call first):
Update
I solved this by adding the following line before Py_Initialize(); in my c code:
Py_SetPythonHome(L"/PATH/to/python3");
This is probably due to the fact that in Python 3.x initialization of modules is not performed by calling init<module_name> but rather with PyInit_<module_name> (See PEP 3121). So, if you are linking with Python 3.x and executing via 3.x you need to change the initialization call.
In short, changing the call that initializes the module from:
inittest();
To:
PyInit_test();
and recompiling, should do the trick.
As for your second problem, an alternate solution other than using Py_SetPythonHome is setting PYTHONHOME to the output of python3.4-config --exec-prefix (or sys.exec_prefix) prior to compilation.
Related
I have a simple example python extenstion I want to use from C/C++. The code is as follows
example.pxy:
from numpy import random
cdef public void somefunc():
print(random.randint(500))
setup.py:
from setuptools import setup
cfom Cython.Build import cythonize
import numpy
setup(
ext_modules=cythonize("example.pyx"),
zip_safe=False,
include_dirs=[numpy.get_include()]
)
Running python3 setup.py build_ext --inplace --compiler="mingw32" -DMS_WIN64 then creates example.c, example.h and example.cp310-win_amd64.pyd. The C++ code I am using to call someFunc is:
example.cpp:
#include <Python.h>
#include "example.h"
int main()
{
Py_Initialize();
somefunc();
return 0;
}
This I compile using g++ example.cpp -DMS_WIN64. But that command seems to be incomplete. There are sill objects left that need to be linked, namely the ones from the example.pyx. How do I do this? I do not see any generated .dllor.lib` or similar.
Additionally, if I use #include "exmaple.c" in
example.cpp, I get a very long list of missing symbols from the linker. The objects are all named __im_Py*.
I am using MINGW64 on Windows 10. The python installation I am trying to link against is a regular python installation from the system. I have an environment variable CPLUS_INCLUDE_PATH=C:\Program Files\Python310\include.
I am trying to learn how to embed a Python interpreter into my C/C++ programs.
I like this concept because it may enable me to extend C programs at run time as well as enable users to write custom scripts/plugins.
Detailed instruction on embedding Python in C is provided at: https://docs.python.org/3/extending/embedding.html
I'm using example code provided in the Python documentation to figure out the mechanics involved:
embedded.c
#include <Python.h>
int
main(int argc, char *argv[])
{
Py_SetProgramName(argv[0]); /* optional but recommended */
Py_Initialize();
PyRun_SimpleString("from time import time,ctime\n"
"print 'Today is',ctime(time())\n");
Py_Finalize();
return 0;
}
The problem is I can't figure out how to compile it on Windows using gcc.
C:\Users\user\embedded_python> gcc embedded.c
embedded.c:2:10: fatal error: Python.h: No such file or directory
#include <Python.h>
^~~~~~~~~~
compilation terminated.
I think the problem is I need to link to the Python.h file, but I can't seem to get the compiler flags right:
C:\Users\user\Desktop\embedded_python>gcc -IC:\Users\user\AppData\Local\Programs\Python\Python38 -lPython38 embedded.c
embedded.c:2:10: fatal error: Python.h: No such file or directory
#include <Python.h>
^~~~~~~~~~
compilation terminated.
How can I get this program to compile on Windows (preferably with gcc/g++ to avoid Visual Studio bloat)?
In Linux, but I suppose if you activate your Bash from windows and can perform a work around. First you have to install python3-dev
Then use this command line where your file is located, to compile (test is your filename)
gcc -Wall test.c -o test $(pkg-config --cflags --libs python3)
I'm trying to embed Cython code into C following O'reilly Cython book chapter 8. I found this paragraph on Cython's documentation but still don't know what should I do:
If the C code wanting to use these functions is part of more than one shared library or executable, then import_modulename() function needs to be called in each of the shared libraries which use these functions. If you crash with a segmentation fault (SIGSEGV on linux) when calling into one of these api calls, this is likely an indication that the shared library which contains the api call which is generating the segmentation fault does not call the import_modulename() function before the api call which crashes.
I'm running Python 3.4, Cython 0.23 and GCC 5 on OS X. The source code are transcendentals.pyx and main.c:
main.c
#include "transcendentals_api.h"
#include <math.h>
#include <stdio.h>
int main(int argc, char **argv)
{
Py_SetPythonHome(L"/Users/spacegoing/anaconda");
Py_Initialize();
import_transcendentals();
printf("pi**e: %f\n", pow(get_pi(), get_e()));
Py_Finalize();
return 0;
}
transcendentals.pyx
cdef api double get_pi():
return 3.1415926
cdef api double get_e():
print("calling get_e()")
return 2.718281828
I'm compiling those files using setup.py and Makefile:
setup.py:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
setup(
ext_modules=cythonize([
Extension("transcendentals", ["transcendentals.pyx"])
])
)
Makefile
python-config=/Users/spacegoing/anaconda/bin/python3-config
ldflags:=$(shell $(python-config) --ldflags)
cflags:=$(shell $(python-config) --cflags)
a.out: main.c transcendentals.so
gcc-5 $(cflags) $(ldflags) transcendentals.c main.c
transcendentals.so: setup.py transcendentals.pyx
python setup.py build_ext --inplace
cython transcendentals.pyx
clean:
rm -r a.out a.out.dSYM build transcendentals.[ch] transcendentals.so transcendentals_api.h
However, I came to error Segmentation fault: 11. Any idea can help with this? Thanks!
In that Makefile there is
transcendentals.so: setup.py transcendentals.pyx
python setup.py build_ext --inplace
Unless python refers to /Users/spacegoing/anaconda/bin/python3 it should be replaced since the module may be compiled for wrong python version, and cannot thus be loaded.
In main.c there is call import_transcendentals() that does not check the return value i.e. if the import fails or succeeds. In case of failure, get_pi() and get_e() point to invalid memory locations and trying to call them causes a segmentation fault.
Also, the module has to be located somewhere where it can be found. It seems that when embedding, the current directory is not searched for python modules. PYTHONPATH environment variable could be changed to include the directory where transcendentals.so is located.
The following is an altenative way of embedding the code to the C program and sidesteps the import issues since the module code is linked to the executable.
Essentially, a call to PyInit_transcendentals() is missing.
File transcendentals.h will be generated when the cython functions are defined public i.e.
cdef public api double get_pi():
...
cdef public api double get_e():
Your main.c should have the include directives
#include <Python.h>
#include "transcendentals.h"
and then in main
Py_Initialize();
PyInit_transcendentals();
There should be no #include "transcendentals_api.h" and no import_transcendentals()
The first reason is that according to the documentation
However, note that you should include either modulename.h or
modulename_api.h in a given C file, not both, otherwise you may get
conflicting dual definitions.
The second reason is, that since transcendentals.c is linked to the program in
gcc $(cflags) $(ldflags) transcendentals.c main.c
there is no reason to import transcendentals module. The module has to be initialized though, PyInit_transcendentals() does that for Python 3
I am having some problems building PyQt5 on Ubuntu 14.04. I am working with some code that has a hard dependency on Python 2.7; so, I am unable to use the python3 packages from Ubuntu's repository. Further searches of Ubuntu's packages reveal that there are dev and doc packages for Python 2 pyqt5. But, nothing to install the libraries necessary to write code.
This has led me to creating a custom build for PyQt5. I obtained the source for version 5.5 from here: https://www.riverbankcomputing.com/software/pyqt/download5 and I am using sip as provided by the Ubuntu repos (installation of kubuntu-desktop requires sip).
I read that its easy to have mismatched versions of sip so I did the following check:
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sip
>>> print(sip, sip.SIP_VERSION_STR)
(<module 'sip' from '/usr/lib/python2.7/dist-packages/sip.so'>, '4.16.9')
And:
$ sip -V
4.16.9
Also I am using the Qt5 tools provided by the Ubuntu repos. This included installing qtdeclarative5-* (probably overkill) and qt5-default. Here is some information about qmake:
qmake --version
QMake version 3.0
Using Qt version 5.2.1 in /usr/lib/x86_64-linux-gnu
I currently have PyQt4 installed and read on the installation notes that this would be fine as long as they were both compiled against the same version of sip.
After downloading, I unpacked the tarball and attempted a build as follows:
sudo ln -s /usr/include/python2.7 /usr/local/include/python2.7
python configure.py --sip-incdir=/usr/include/python2.7/
make
The configuration output appeared to identify the correct version of sip and I get the following (seemingly) sip related compile errors from make:
make[1]: Entering directory `~/Downloads/PyQt-gpl-5.5/QtWebKit'
g++ -c -m64 -pipe -fno-exceptions -O2 -Wall -W -D_REENTRANT -fPIC -DSIP_PROTECTED_IS_PUBLIC -Dprotected=public -DQT_NO_DEBUG -DQT_PLUGIN -DQT_WEBKIT_LIB -DQT_NETWORK_LIB -DQT_GUI_LIB -DQT_CORE_LIB -I/usr/lib/x86_64-linux-gnu/qt5/mkspecs/linux-g++-64 -I. -I. -I/usr/include/python2.7 -I/usr/include/qt5 -I/usr/include/qt5/QtWebKit -I/usr/include/qt5/QtNetwork -I/usr/include/qt5/QtGui -I/usr/include/qt5/QtCore -I. -o sipQtWebKitQWebSecurityOrigin.o sipQtWebKitQWebSecurityOrigin.cpp
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp: In function ‘PyObject* meth_QWebSecurityOrigin_addAccessWhitelistEntry(PyObject*, PyObject*)’:
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp:384:9: error: ‘SubdomainSetting’ is not a member of ‘QWebSecurityOrigin’
QWebSecurityOrigin::SubdomainSetting a2;
^
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp:384:46: error: expected ‘;’ before ‘a2’
QWebSecurityOrigin::SubdomainSetting a2;
^
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp:387:214: error: ‘a2’ was not declared in this scope
if (sipParseArgs(&sipParseErr, sipArgs, "BJ1J1E", &sipSelf, sipType_QWebSecurityOrigin, &sipCpp, sipType_QString,&a0, &a0State, sipType_QString,&a1, &a1State, sipType_QWebSecurityOrigin_SubdomainSetting, &a2))
^
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp:389:21: error: ‘class QWebSecurityOrigin’ has no member named ‘addAccessWhitelistEntry’
sipCpp->addAccessWhitelistEntry(*a0,*a1,a2);
^
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp: In function ‘PyObject* meth_QWebSecurityOrigin_removeAccessWhitelistEntry(PyObject*, PyObject*)’:
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp:417:9: error: ‘SubdomainSetting’ is not a member of ‘QWebSecurityOrigin’
QWebSecurityOrigin::SubdomainSetting a2;
^
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp:417:46: error: expected ‘;’ before ‘a2’
QWebSecurityOrigin::SubdomainSetting a2;
^
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp:420:214: error: ‘a2’ was not declared in this scope
if (sipParseArgs(&sipParseErr, sipArgs, "BJ1J1E", &sipSelf, sipType_QWebSecurityOrigin, &sipCpp, sipType_QString,&a0, &a0State, sipType_QString,&a1, &a1State, sipType_QWebSecurityOrigin_SubdomainSetting, &a2))
^
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp:422:21: error: ‘class QWebSecurityOrigin’ has no member named ‘removeAccessWhitelistEntry’
sipCpp->removeAccessWhitelistEntry(*a0,*a1,a2);
^
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp: In function ‘void* init_type_QWebSecurityOrigin(sipSimpleWrapper*, PyObject*, PyObject*, PyObject**, PyObject**, PyObject**)’:
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp:477:48: error: no matching function for call to ‘QWebSecurityOrigin::QWebSecurityOrigin(const QUrl&)’
sipCpp = new QWebSecurityOrigin(*a0);
^
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp:477:48: note: candidates are:
In file included from ~/Downloads/PyQt-gpl-5.5/sip/QtWebKit/qwebsecurityorigin.sip:26:0:
/usr/include/qt5/QtWebKit/qwebsecurityorigin.h:64:5: note: QWebSecurityOrigin::QWebSecurityOrigin(QWebSecurityOriginPrivate*)
QWebSecurityOrigin(QWebSecurityOriginPrivate* priv);
^
/usr/include/qt5/QtWebKit/qwebsecurityorigin.h:64:5: note: no known conversion for argument 1 from ‘const QUrl’ to ‘QWebSecurityOriginPrivate*’
/usr/include/qt5/QtWebKit/qwebsecurityorigin.h:58:5: note: QWebSecurityOrigin::QWebSecurityOrigin(const QWebSecurityOrigin&)
QWebSecurityOrigin(const QWebSecurityOrigin& other);
^
/usr/include/qt5/QtWebKit/qwebsecurityorigin.h:58:5: note: no known conversion for argument 1 from ‘const QUrl’ to ‘const QWebSecurityOrigin&’
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp: At global scope:
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp:516:48: error: ‘AllowSubdomains’ is not a member of ‘QWebSecurityOrigin’
{sipName_AllowSubdomains, static_cast<int>(QWebSecurityOrigin::AllowSubdomains), 21},
^
~/Downloads/PyQt-gpl-5.5/QtWebKit/sipQtWebKitQWebSecurityOrigin.cpp:517:51: error: ‘DisallowSubdomains’ is not a member of ‘QWebSecurityOrigin’
{sipName_DisallowSubdomains, static_cast<int>(QWebSecurityOrigin::DisallowSubdomains), 21},
^
make[1]: *** [sipQtWebKitQWebSecurityOrigin.o] Error 1
make[1]: Leaving directory `~/Downloads/PyQt-gpl-5.5/QtWebKit'
make: *** [sub-QtWebKit-make_first-ordered] Error 2
The outcome is that I can make install and get some of the functionality that I'd expect; however, I am missing some required functionality with the WebKit widgets. I hope that I have supplied enough information to describe where i'm stuck. I feel just shy of digging into the code; however, I'm assuming that the answer is actually much simpler.
Thanks in advance!
So,
I started digging through the source package for the file that is failing to be compiled. In the sip directory there is a sip file QWebSecurityOrigin that contains the following:
%If (Qt_5_2_0 -)
enum SubdomainSetting
{
AllowSubdomains,
DisallowSubdomains,
};
%End
I can reasonably expect that this code be included as qmake tells me the following:
qmake --version
QMake version 3.0
Using Qt version 5.2.1 in /usr/lib/x86_64-linux-gnu
Next I wanted to look into the qwebsecurityorigin.h that was provided by Qt to see if the error could come from there. Mine is installed here: /usr/include/qt5/QtWebKit/qwebsecurityorigin.h
#ifndef _WEBSECURITYORIGIN_H_
#define _WEBSECURITYORIGIN_H_
#include <QtCore/qurl.h>
#include <QtCore/qshareddata.h>
#include "qwebkitglobal.h"
namespace WebCore {
class SecurityOrigin;
class ChromeClientQt;
}
class QWebSecurityOriginPrivate;
class QWebDatabase;
class QWebFrame;
class QWEBKIT_EXPORT QWebSecurityOrigin {
public:
static QList<QWebSecurityOrigin> allOrigins();
static void addLocalScheme(const QString& scheme);
static void removeLocalScheme(const QString& scheme);
static QStringList localSchemes();
~QWebSecurityOrigin();
QString scheme() const;
QString host() const;
int port() const;
qint64 databaseUsage() const;
qint64 databaseQuota() const;
void setDatabaseQuota(qint64 quota);
void setApplicationCacheQuota(qint64 quota);
QList<QWebDatabase> databases() const;
QWebSecurityOrigin(const QWebSecurityOrigin& other);
QWebSecurityOrigin &operator=(const QWebSecurityOrigin& other);
private:
friend class QWebDatabase;
friend class QWebFrameAdapter;
friend class WebCore::ChromeClientQt;
QWebSecurityOrigin(QWebSecurityOriginPrivate* priv);
private:
QExplicitlySharedDataPointer<QWebSecurityOriginPrivate> d;
};
Note, no enum is defined. A search of Qt 5.5 suggests that the enum should be there: http://doc.qt.io/qt-5/qwebsecurityorigin.html#SubdomainSetting-enum
Finally, had recalled that I installed libqt5webkit separately from the bulk of the Qt libraries; so, I did a version check on the package:
dpkg -s libqt5webkit5
Package: libqt5webkit5
Status: install ok installed
Priority: optional
Section: libs
Installed-Size: 34225
Maintainer: Ubuntu Developers <ubuntu-devel-discuss#lists.ubuntu.com>
Architecture: amd64
Multi-Arch: same
Source: qtwebkit-opensource-src
Version: 5.1.1-1ubuntu8
This output is almost identical for the dev package. So this makes it appear as the bulk of the Qt5 distribution, in the repos, is on a different version than webkit. Furthermore, if QWebKit is on 5.1.1, it would explain why the enum is missing as the sip file seems to suggest it was an addition in 5.2.0.
So my solution was to download and install Qt 5.5 from the Qt website using the automated installer (ran with sudo, using defaults). I then started fresh with the PyQt5 source by blowing away the build directory and unpacking source again:
python configure.py --sip-incdir=/usr/include/python2.7/ --qmake=/opt/Qt/5.5/gcc_64/bin/qmake
make
sudo make install
The licenses are not compatible; however, a quick search through the pyqt5 configure.py script, using the error output, may give some insight into getting the code configured and compiling.
I want to build simple app with pybind11, pybind is already installed in my Ubuntu system with cmake (and make install). I use this simple cmake file:
cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
project(trt_cpp_loader )
find_package(pybind11 REQUIRED)
add_executable(trt_cpp_loader main.cpp)
set_property(TARGET trt_cpp_loader PROPERTY CXX_STANDARD 11)
This is main.cpp:
#include <iostream>
#include <pybind11/embed.h>
namespace py = pybind11;
using namespace std;
int main(){return 0;}
when I build it, I get:
In file included from /usr/local/include/pybind11/pytypes.h:12:0,
from /usr/local/include/pybind11/cast.h:13,
from /usr/local/include/pybind11/attr.h:13,
from /usr/local/include/pybind11/pybind11.h:44,
from /usr/local/include/pybind11/embed.h:12,
from /home/stiv/lpr/trt_cpp_loader/main.cpp:2:
/usr/local/include/pybind11/detail/common.h:112:10: fatal error: Python.h: No such file or directory
#include <Python.h>
^~~~~~~~~~
compilation terminated.
how can I fix this problem? (python-dev and python3-dev are already installed, Python.h is available)
You'll want to use the pybind11_add_module command (see https://pybind11.readthedocs.io/en/stable/compiling.html#building-with-cmake) for the default case of creating an extension module.
If the goal is indeed to embed Python in an executable, it is your reponsibility to explicitly add python headers & libraries to the compiler/linker commands in CMake. (see https://pybind11.readthedocs.io/en/stable/compiling.html#embedding-the-python-interpreter on how to do that)
Following the Wenzel Jakob's answer I want to put an example of CMakeLists.txt for compiling the example provided in this tutorial:
// example.cpp
#include <pybind11/pybind11.h>
int add(int i, int j) {
return i + j;
}
PYBIND11_MODULE(example, m) {
m.doc() = "pybind11 example plugin"; // optional module docstring
m.def("add", &add, "A function which adds two numbers");
}
and
# example.py
import example
print(example.add(1, 2))
and
# CMakeLists.txt
cmake_minimum_required(VERSION 2.8.12)
project(example)
find_package(pybind11 REQUIRED)
pybind11_add_module(example example.cpp)
now in the root run
cmake .
make
now run the python code by
python3 example.py
P.S. I have also written some instructions here for compiling/installing the pybind11.
Maybe just install the Python headers? For example, on Ubuntu you can install the sudo apt-get install python-dev (or python3-dev or pythonX.Y-dev) package. That could resolve this.