Develop Reference

Call function from python in C++

I'm currently working on a project with a separate cli and gui. The gui part is written in C++ to keep the exectuable small. I compiled my python program with pyinstaller and need to call some functions from the C++ program.
Python part:
import configparser
def getconfig() -> list:
config = configparser.ConfigParser()
config.read("config.ini")
section1 = config["general"]
section2 = section["section2"]
return [section1, section2] #should be a list of dict?
Compiling it via pyinstaller --onefile --clean myprogram.py
What i would like to do in C++ is :
//please correct me if this is the wrong type,
//i know i probably need to do a bit of parsing between python and C++ types
std::vector<std::unordered_map<std::string, std::string>> > = myprogram.getconfig()
I just don't want to do the config parsing in C++ again, or would you recommend this as it's probably easier to call the compiled python binary?
This program just needs to run on linux, windows is not necessary.

If you are only planning to run on POSIX-compatible systems (ie. GNU/Linux), you could spawn a new process to run the Python script, e.g. using the C function exec. Example:
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
int main()
{
char* args[] = { "pythonscript.py", NULL };
int t = execv("pythonscript.py", args);
if(t < 0) printf("%s\n", strerror(errno));
return 0;
}
And then in pythonscript.py:
#!/usr/bin/python3
print("hello, world!")
This trick works with bash scripts as well.
The script must be executable though, so remember to run chmod +x pythonscript.py.
EDIT:
You will likely need to use pipes or other mechanisms for inter-process communication. (this is not my expertise!)

How can I save a python function to a static c++ container using pybind11?

Essentially, on the C++ side I have a container that holds a certain type of function. Now I would like to expose this container to python with the possibility for the users to provide their own python functions.
The simplest example would look like this:
#include "pybind/common/Common.h"
using CppFunc = std::function< int (int) >;
PYBIND11_MODULE( test, m )
{
m.def("addFunc", [](const pybind11::function& f){
static std::vector<CppFunc> vec{};
vec.push_back(f.cast<CppFunc>());
});
}
Then in python I would like to just do something like this.
import test
def myFunc(number):
return number+1
test.addFunc(myFunc)
Interestingly enough, this works fine. However, if I run the script with "python script.py" it runs through and then never terminates. In an interactive console, the same code works fine until you try to close the console: the process gets stuck.
How can I safely store this python function in the C++ container?

static std::vector<CppFunc> vec{} stores references to python objects (user functions) which never get released due to static storage, therefore interpreter cannot terminate.
To ensure interpreter termination you can call a cleanup function at module termination:
#include "pybind11/pybind11.h"
#include "pybind11/functional.h"
namespace py = pybind11;
using CppFunc = std::function< int (int) >;
PYBIND11_MODULE( test , m )
{
static std::vector<CppFunc> vec{};
m.def("addFunc", [](CppFunc f){
vec.push_back(std::move(f));
});
m.add_object("_cleanup", py::capsule([]{ vec.clear(); }));
}
See doc for more details: https://pybind11.readthedocs.io/en/stable/advanced/misc.html#module-destructors

How do I construct python code from a give .pyd output in Visual Studio?

I am trying to develop a project to integrate C++ function with python using Pybind11. I am quite familiar with C++, but not so with Python. I have files with following format which I developed for a C++ project.
Output of C++ as: cppproject.pyd
C++ function I want to integrate: int add(int i, int j)
Pybind11 module: PYBIND11_MODULE(example,m){....}
I have all the files I need. But I need to run the add function inside Python now and I am stuck with how to code.
I tried
from cppproject import example
example.add(1, 2)
but it is throwing me an exception as follows:
dynamic module does not define module export function (PyInit_cppproject)
Where am I making mistake with the python code?
If it helps, this is my C++ code:
#include <Python.h>
#include <pybind11/pybind11.h>
int add(int i, int j) {
return i + j;
}
PYBIND11_MODULE(example, m) {
m.def("add", &add, "A function which adds two numbers");
}
The output of this file is in .pyd format to facilitate for python integration.
Edit: By the way, I am trying to run both, C++ and Python projects, as one solution in Visual Studio.

Should work if you name your out-file example.pyd.
then:
from example import add
Alternatively:
PYBIND11_MODULE(cppproject, m) {
auto example = m.def_submodule("example");
...
}

Embedding python 3.4 into C++ Qt Application?

I'm making an Qt Quick GUI application(for windows), which uses OpenGL and C++ for some computationally intensive stuff. I want to embed python code into the app, for doing some stuff which is comparatively easier in python.
Basically, I just want the c++ code to call a function in a python script and let the script do the job, then store the returned data in a variable(string, or float etc.) for further use. I'm using Qt creator, and I got python3 lib for MinGW compiler. I tried some code, but its looks like python lib is not quite compatible with Qt creator. IS using pyqt for this will be a good idea? What will be the best and easiest way to do this ?
EDIT: This is the basic code I tried, first it gave me an error saying, cannot find pyconfig.h. Then I added an INCUDEPATH to my python34 include directory.
#include "mainwindow.h"
#include <QApplication>
#include <boost/python.hpp>
int main(int argc, char *argv[])
{
QApplication a(argc, argv);
MainWindow w;
w.show();
using namespace boost::python;
PyObject *pName, *pModule, *pDict, *pFunc, *pValue;
Py_Initialize();
pName = PyString_FromString(argv[1]);
pModule = PyImport_Import(pName);
pDict = PyModule_GetDict(pModule);
pFunc = PyDict_GetItemString(pDict, argv[2]);
if (PyCallable_Check(pFunc))
{
PyObject_CallObject(pFunc, NULL);
} else
{
PyErr_Print();
}
// Clean up
Py_DECREF(pModule);
Py_DECREF(pName);
Py_Finalize();
return a.exec();
}
My .pro file:
QT += core gui
greaterThan(QT_MAJOR_VERSION, 4): QT += widgets
TARGET = TestWidgetApp
TEMPLATE = app
INCLUDEPATH += C:/boost_1_57_0
INCLUDEPATH += C:/Python34/include
SOURCES += main.cpp\
mainwindow.cpp
HEADERS += mainwindow.h
FORMS += mainwindow.ui
OTHER_FILES +=
Then the following errors:
C:\Python34\include\object.h:435: error: C2059: syntax error : ';'
C:\Python34\include\object.h:435: error: C2238: unexpected token(s) preceding ';'
C:\Users\Amol\Desktop\TestWidgetApp\main.cpp:19: error: C3861: 'PyString_FromString': identifier not found

The problem here is that Python 3.4 has a struct member called "slots", (file object.h, in the typedef for PyType_Spec), which Qt defines out from under you so that you can say things like:
public slots:
in your code. The solution is to add:
#undef slots
just before you include Python.h, and to redefine it before you include anything that uses "slots" in the way that Qt does:
#undef slots
#include <Python.h>
#define slots
#include "myinclude.h"
#include <QString>
A bit of a hack (because you're depending on a particular definition of slots in Qt), but it should get you going.

I have removed all the Qt code from your example and then I tried to compile it (Qt has nothing to do with your compile error). And it compiles for me. The difference was I used the include files from Python 2.7.
So I did a little search for the string PyString_FromString in the folders: C:\Python33\includes (I noted you use python 3.4 and not 3.3 but I suspect this is a 3.x thing) and C:\Python27\includes.
Results:
Python 3.3
Python 2.7
So, apparently, Python 3.4 is not supported by your BoostPython version.

Python3 has no PyString_FromString function. Python3 str type internally is unicode objects with complex structure.
Use PyUnicode_FromString or PyUnicode_FromStringAndSize for constructing str object from UTF-8 encoded C string (char*).

Move your
#include "boost/python.hpp"
...to be before your other includes and it will resolve your problem.
The actual issue is as Scott Deerwester described in his answer.

Passing a list through Python to C++

So I have a Python program that's finding .txt file directories and then passing those directories as a list(I believe) to my C++ program. The problem I am having is that I am not sure how to pass the list to C++ properly. I have used :
subprocess.call(["path for C++ executable"] + file_list)
where file_list is the [] of txt file directories.
My arguments that my C++ code accepts are:
int main (int argc, string argv[])
Is this correct or should I be using a vector? When I do use this as my argument and try to print out the list I get the directory of my executable, the list, and then smiley faces, symbols, and then the program crashes.
Any suggestions? My main point that I am trying to find out is the proper syntax of utilizing subprocess.call. Any help would be appreciated! thanks!

Another option is to use cython, (not a direct answer). Here is a simple complete example:
Suppose you have the following files:
cython_file.cpp
python_file.py
setup.py
sum_my_vector.cpp
sum_my_vector.h
setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [Extension(
name="cython_file",
sources=["cython_file.pyx", "sum_my_vector.cpp"],
extra_compile_args=["-std=c++11"],
language="c++",
)]
setup(
name = 'cython_file',
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules,
)
cython_file.pyx
from libcpp.vector cimport vector
cdef extern from "sum_my_vector.h":
int sum_my_vector(vector[int] my_vector)
def sum_my_vector_cpp(my_list):
cdef vector[int] my_vector = my_list
return sum_my_vector(my_vector)
sum_my_vector.cpp
#include <iostream>
#include <vector>
#include "sum_my_vector.h"
using namespace::std;
int sum_my_vector(vector<int> my_vector)
{
int my_sum = 0;
for (auto iv = my_vector.begin(); iv != my_vector.end(); iv++)
my_sum += *iv;
return my_sum;
}
sum_my_vector.h
#ifndef SUM_MY_VECTOR
#define SUM_MY_VECTOR
using namespace::std;
int sum_my_vector(vector<int> my_vector);
#endif
python_file.py
from cython_file import sum_my_vector_cpp
print sum_my_vector_cpp([1,2,3,5])
Now run
python setup.py build_ext --inplace
and the you can run the python file
python python_file.py
11

"Passing a list through Python to C++"
An alternative approach would be to use Boost.Python, this may not answer your question directly, but still its worth pointing out another solution.
#include <boost/python.hpp>
#include <vector>
#include <string>
void get_dir_list( boost::python::list dir_list )
{
for (int i = 0; i < len(dir_list); ++i)
{
std::string x = boost::python::extract<std::string>(dir_list[i]);
// perform stuffs
std::cout << "This is " << x << std::endl ;
}
}
BOOST_PYTHON_MODULE(get_dir_list)
{
def("get_dir_list", get_dir_list);
}
Compiled Using :
g++ main.cpp -shared -fPIC -o get_dir_list.so -I/usr/include/python2.7 -lboost_python
Usage :
import get_dir_list
import os
get_dir_list.get_dir_list(os.listdir('.'))
Live Demo Here

I'll post this alternative solution since it would also work for other long lists of strings that needed to be passed.
In your Python script create a text file (I'll call it "masterFile") and write the file paths to the masterFile. You could give each file path a separate line. Then pass the masterFile's file path to your C++ program. This way you don't have to worry about the length of your command line arguments. Let your C++ program open and read the file for processing.
You can use something like os.remove() to get rid of the masterFile in your Python script once the C++ program has finished.
Also, you mentioned in the comments that you need to do different tasks dependent on different file paths: A suggestion would be to add a char at the beginning of each line in the masterFile to signal what needs to be done for the particular file. Example:
a Random/path/aFile.txt # a could mean do task 1
b Random2/path2/differentFile.c # b could mean do task 2

You pass a list to subprocess.call. subprocess.call converts this to what is needed for the system (which may vary, but certainly isn't a Python list). The system then arranges for this to be copied somewhere in the new process, and sets up the standard arguments to main, which are int, char**. In your C++ program, you must define main as int main( int argc, char** argv ); nothing else will work. (At least... a system could support int main( std::string const& ) or some such as an extension. But I've never heard of one that did.)