Introduction
I'm trying to do a simple thing - in theory, but so far not in practise. I have C++ console application, with embedded python script. It has to pass some variables inside python, do calculation, and send it back.
Example code
my cpp file look like this (based on THIS answer):
#include <iostream>
#include <boost/python.hpp>
using namespace boost::python;
int main()
{
int a = 2;
int b = 3;
Py_Initialize();
try
{
object module = import("__main__");
object name_space = module.attr("__dict__");
exec_file("Script.py", name_space, name_space);
object MyFunc = name_space["MyFunc"];
object result = MyFunc(a,b);
int sum = extract<int>(result["sum"]);
std::cout << sum;
}
catch (error_already_set)
{
PyErr_Print();
}
Py_Finalize();
}
My Script.py file is very simple
def MyFunc(a,b):
result = a+b
return result
The Issue
I expected, that python will accept two variables, sum it, and let extract it back do my C++. Then C++ print the results. However it ends with following error:
TypeError: 'int' object has no attribute '__getitem__'
I think that MyFunc(a,b) is not correct call, however I can't figure why. Should I pass variables in other way? If yes, pleas explain me how to do this.
Related
I am using Python code within C++ code and trying to pass a list argument to a function written in Python. I tried the normal way of executing the Python code without passing any argument and it was all working fine but when I pass a list as an argument, I get a segmentation fault.
Here is my code:
#define PY_SSIZE_T_CLEAN
#include</usr/include/python3.6/Python.h>
#include <bits/stdc++.h>
using namespace std;
int callModuleFunc(int array[], size_t size) {
PyObject *mymodule = PyImport_ImportModule("test");
PyObject *myfunc = PyObject_GetAttrString(mymodule, "get_lists");
cout<<"Imported"<<endl;
PyObject *mylist = PyList_New(size);
cout<<"Imported3"<<endl;
for (size_t i = 0; i != size; ++i) {
PyList_SET_ITEM(mylist, i, PyLong_FromLong(array[i]));
}
PyObject *arglist = Py_BuildValue("(O)", mylist);
cout<<"Imported1"<<endl;
PyObject *result = PyObject_CallObject(myfunc, arglist); // getting segmentation fault here
cout<<"Imported5"<<endl;
int retval = (int)PyLong_AsLong(result);
Py_DECREF(result);
Py_DECREF(arglist);
Py_DECREF(mylist);
Py_DECREF(myfunc);
Py_DECREF(mymodule);
return retval;
}
int main(int argc, char const *argv[])
{
wchar_t * program = Py_DecodeLocale(argv[0], NULL);
if(!program){
cout<<"***Error***"<<endl;
exit(1);
}
Py_SetProgramName(program);
Py_Initialize();
PyObject *module = NULL, *result = NULL;
PyRun_SimpleString("print('Hello from python')\n"
"print('Hiii')");
int arr[5] = {1,3,4,5,6};
callModuleFunc(arr, 5);
if(Py_FinalizeEx() < 0){
cout<<"***Error***"<<endl;
exit(120);
}
PyMem_RawFree(program);
return 0;
}
When I call PyObject_CallObject(myfunc, arglist), I get a segmentation fault.
I am totally new to it so I'm just trying stuff from the internet.
I'm using Python version 3.6 with g++ compiler 7.5.
Here is my test.py:
def get_lists(l1):
print("Lists: ", l1)
return 10
Please let me know how I can resolve this.
Thanks
The Python function get_lists is not known during execution.
Note: The test package is meant for internal use by Python only. It is documented for the benefit of the core developers of Python. Any use of this package outside of Python’s standard library is discouraged as code mentioned here can change or be removed without notice between releases of Python.
see https://docs.python.org/3/library/test.html
If the name of the imported Python file is renamed to another name (e.g. list.py) the Python function can be found.
Additional Hints
For the environment of the OP the problem is then solved. For my environment (Python 3.9.5) I need additionally replace the PyRun_SimpleString with:
PyRun_SimpleString("import sys");
PyRun_SimpleString("sys.path.append(\".\")");
Otherwise the module can't be imported.
Finally, it is advisable to check each function call for errors (e.g. whether it returns NULL) and to use PyErr_Print so that the actual cause of the error is printed instead of a crash due to a segmentation fault.
For example an error message could look like this:
AttributeError: module 'test' has no attribute 'get_lists'
Test
The output of the line Lists: [1, 3, 4, 5, 6] on the console shows that the Python function is called correctly when the above points are taken into account.
I'm learning C++, and in particular C interface to Python. Right now, my focus is on calling or importing python objects from C++ main program.
I've been studying the following link but couldn't understand some concepts. (https://www.codeproject.com/Articles/820116/Embedding-Python-program-in-a-C-Cplusplus-code)
Following is the sections of the tutorial that I can't understand fully.
My questions are:
calling of module:
Is it correct for me to assume "CPyObject pModule = PyImport_Import(pName)" is doing this job?
importing of objects:
i. Is it correct for me to assume "CPyObject pFunc = PyObject_GetAttrString(pModule, "getInteger")" is doing this job?
ii.If I want to import a dataframe from python to C++ as a CPyObject, how can I manipulate this object in C++. I'm asking because there is no equivalent object to dataframe in C++.
3) Is there anything else I need to do to make sure my Python module file is visible and callable to C++ ? Such as saving them in the same folder?
Consider the following Python program, stored in pyemb3.py:
def getInteger():
print('Python function getInteger() called')
c = 100*50/30
return c
Now we want to call the function getInteger() from the following C++ code and print the value returned this function. This is the client C++ code:
#include <stdio.h>
#include <Python.h>
#include <pyhelper.hpp>
int main()
{
CPyInstance hInstance;
CPyObject pName = PyUnicode_FromString("pyemb3");
CPyObject pModule = PyImport_Import(pName);
if(pModule)
{
CPyObject pFunc = PyObject_GetAttrString(pModule, "getInteger");
if(pFunc && PyCallable_Check(pFunc))
{
CPyObject pValue = PyObject_CallObject(pFunc, NULL);
printf_s("C: getInteger() = %ld\n", PyLong_AsLong(pValue));
}
else
{
printf("ERROR: function getInteger()\n");
}
}
else
{
printf_s("ERROR: Module not imported\n");
}
return 0;
}
The problem is that 100*50/30 is not an integer, it is a float.
to get an integer use integer division: 100*50//30
If you are not sure about the returned type, you can use the Py_TYPE macro on pValue or just simply check for the type you are looking for with: PyLong_Check or PyLong_CheckExact
1: if PyImport_Import does not return null then the import was successful and the module was already executed by the time the function returned.
2: The PyObject_GetAttrString or the PyObject_GetAttr is the right way to get the imported module's objects.
3: Use these flags to ensure Python is embedded. Use Py_SetPath before Py_Initialize to add your module's path to sys.path.
This question is about how to pass a C++ object to a python function that is called in a (C++) embedded Python interpreter.
The following C++ class (MyClass.h) is designed for testing:
#ifndef MyClassH
#define MyClassH
#include <string>
using std::string;
class MyClass
{
public:
MyClass(const string& lbl): label(lbl) {}
~MyClass(){}
string getLabel() {return label;}
private:
string label;
};
#endif
A python module, exposing the C++ class, can be generated by the following Swig interface file:
%module passmetopython
%{ #include "MyClass.h" %}
%include "std_string.i"
//Expose to Python
%include "MyClass.h"
Below is a Python script using the python module
import passmetopython as pmtp
def execute(obj):
#This function is to be called from C/C++, with a
#MyClass object as an argument
print ("Entering execute function")
lbl = obj.getLabel();
print ("Printing from within python execute function. Object label is: " + lbl)
return True
def main():
c = pmtp.MyClass("Test 1")
retValue = execute(c)
print("Return value: " + str(retValue))
#Test function from within python
if __name__ == '__main__':
main()
This question is about how to get the python execute() function working, when called from c++, with a C++ object as an argument.
The following C++ program was written to test the functions (minimum amount of error checking):
#include "Python.h"
#include <iostream>
#include <sstream>
#include "MyClass.h"
using namespace std;
int main()
{
MyClass obj("In C++");
cout << "Object label: \"" << obj.getLabel() << "\"" << endl;
//Setup the Python interpreter and eventually call the execute function in the
//demo python script
Py_Initialize();
//Load python Demo script, "passmetopythonDemo.py"
string PyModule("passmetopythonDemo");
PyObject* pm = PyUnicode_DecodeFSDefault(PyModule.c_str());
PyRun_SimpleString("import sys");
stringstream cmd;
cmd << "sys.path.append(\"" << "." << "\")";
PyRun_SimpleString(cmd.str().c_str());
PyObject* PyModuleP = PyImport_Import(pm);
Py_DECREF(pm);
//Now create PyObjects for the Python functions that we want to call
PyObject* pFunc = PyObject_GetAttrString(PyModuleP, "execute");
if(pFunc)
{
//Setup argument
PyObject* pArgs = PyTuple_New(1);
//Construct a PyObject* from long
PyObject* pObj(NULL);
/* My current attempt to create avalid argument to Python */
pObj = PyLong_FromLong((long) &obj);
PyTuple_SetItem(pArgs, 0, pObj);
/***** Calling python here *****/
cout<<endl<<"Calling function with an MyClass argument\n\n";
PyObject* res = PyObject_CallObject(pFunc, pArgs);
if(!res)
{
cerr << "Failed calling function..";
}
}
return 0;
}
When running the above code, the execute() python function, with a MyClass object as an argument, fails and returns NULL. However, the Python function is entered, as I can see the output (Entering execute function) in the console output, indicating that the object passed is not, indeed, a valid MyClass object.
There are a lot of examples on how to pass simple types, like ints, doubles or string types to Python from C/C++. But there are very few example showing how to pass a C/C++ object/ pointer, which is kind of puzzling.
The above code, with a CMake file, can be checked out from github:
https://github.com/TotteKarlsson/miniprojects/tree/master/passMeToPython
This code is not to use any boost python or other API's. Cython sounds interesting though, and if it can be used to simplify on the C++ side, it could be acceptable.
This is a partial answer to my own question. I'm saying partial, because I do believe there is a better way.
Building on this post http://swig.10945.n7.nabble.com/Pass-a-Swig-wrapped-C-class-to-embedded-Python-code-td8812.html
I generated the swig runtime header, as described here, section 15.4: http://www.swig.org/Doc2.0/Modules.html#Modules_external_run_time
Including the generated header in the C++ code above, allow the following code to be written:
PyObject* pObj = SWIG_NewPointerObj((void*)&obj, SWIG_TypeQuery("_p_MyClass"), 0 );
This code is using information from the Swig python wrap source files, namely the "swig" name of the type MyClass, i.e. _p_MyClass.
With the above PyObject* as an argument to the PyObject_CallObject function, the python execute() function in the code above executes fine, and the Python code, using the generated python module, do have proper access to the MyClass objects internal data. This is great.
Although the above code illustrate how to pass, and retrieve data between C++ and Python in a quite simple fashion, its not ideal, in my opinion.
The usage of the swig header file in the C++ code is really not that pretty, and in addition, it requires a user to "manually" look into swig generated wrapper code in order to find the "_p_MyClass" code.
There must be a better way!? Perhaps something should be added to the swig interface file in order to get this looking nicer?
PyObject *pValue;
pValue = PyObject_CallMethod(pInstance, "add","(i)",x);
if (pValue)
Py_DECREF(pValue);
else
PyErr_Print();
I try to expose two different classes to python, but I don't get it to compile. I tried to follow the boost::python example, which works quite well. But if I try to write the wrapper classes for my classes it doesn't work. I have provided two minimal examples below:
struct Base
{
virtual ~Base() {}
virtual std::unique_ptr<std::string> f() = 0;
};
struct BaseWrap : Base, python::wrapper<Base>
{
std::unique_ptr<std::string> f()
{
return this->get_override("f")();
}
};
and
struct Base
{
virtual ~Base() {}
virtual void f() = 0;
};
struct BaseWrap : Base, python::wrapper<Base>
{
void f()
{
return this->get_override("f")();
}
};
The first one does not compile because of the unique pointer(I think boost::python does not use unique pointers?) and the second example complains about the return statement inside the void function. Can someone help me how to solve this problems?
The examples are failing to compile because:
The first example attempts to convert an unspecified type (the return type of override::operator()) to an incompatible type. In particular, Boost.Python does not currently support std::unique_ptr, and hence will not convert to it.
The second example attempts to return the unspecified type mentioned above when the calling function declares that it returns void.
From a Python perspective, strings are immutable, and attempting to transferring ownership of a string from Python to C++ violates semantics. However, one could create a copy of a string within C++, and pass ownership of the copied string to C++. For example:
std::unique_ptr<std::string> BaseWrap::f()
{
// This could throw if the Python method throws or the Python
// method returns a value that is not convertible to std::string.
std::string result = this->get_override("f")();
// Adapt the result to the return type.
return std::unique_ptr<std::string>(new std::string(result));
}
The object returned from this->get_override("f")() has an unspecified type, but can be used to convert to C++ types. The invocation of the override will throw if Python throws, and the conversion to the C++ type will throw if the object returned from Python is not convertible to the C++ type.
Here is a complete example demonstrating two ways to adapt the returned Python object to a C++ object. As mentioned above, the override conversion can be used. Alternatively, one can use boost::python::extract<>, allowing one to check if the conversion will fail before performing the conversion:
#include <memory> // std::unique_ptr
#include <boost/algorithm/string.hpp> // boost::to_upper_copy
#include <boost/python.hpp>
struct base
{
virtual ~base() {}
virtual std::unique_ptr<std::string> perform() = 0;
};
struct base_wrap : base, boost::python::wrapper<base>
{
std::unique_ptr<std::string> perform()
{
namespace python = boost::python;
// This could throw if the Python method throws or the Python
// method returns a value that is not convertible to std::string.
std::string result = this->get_override("perform")();
// Alternatively, an extract could be used to defer extracting the
// result.
python::object method(this->get_override("perform"));
python::extract<std::string> extractor(method());
// Check that extractor contains a std::string without throwing.
assert(extractor.check());
// extractor() would throw if it did not contain a std::string.
assert(result == extractor());
// Adapt the result to the return type.
return std::unique_ptr<std::string>(new std::string(result));
}
};
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<base_wrap, boost::noncopyable>("Base", python::init<>())
.def("perform", python::pure_virtual(&base::perform))
;
python::def("make_upper", +[](base* object) {
auto result = object->perform(); // Force dispatch through base_wrap.
assert(result);
return boost::to_upper_copy(*result);
});
}
Interactive usage:
>>> import example
>>> class Derived(example.Base):
... def perform(self):
... return "abc"
...
>>> derived = Derived()
>>> assert("ABC" == example.make_upper(derived))
I have problem wrapping an Enum for Python using Boost-Python.
Initially I intended to do something like the following in the try-catch (I've inserted my whole code below) statement:
main_namespace["Motion"] = enum_<TestClass::Motion>("Motion")
.value("walk", TestClass::walk)
.value("bike", TestClass::bike)
;
Everything was fine and compilation was done. At run time I got this error (which makes no sense to me):
AttributeError: 'NoneType' object has no attribute 'Motion'
Afterwards I decided to write a Python module using BOOST_PYTHON_MODULE in my code.
After initializing Python interpreter I wanted to use this module right away but didn't know how(?). The following is my whole code:
#include <boost/python.hpp>
#include <iostream>
using namespace std;
using namespace boost::python;
BOOST_PYTHON_MODULE(test)
{
enum_<TestClass::Motion>("Motion")
.value("walk", TestClass::walk)
.value("bike", TestClass::bike)
;
}
int main()
{
Py_Initialize();
try
{
object pyMainModule = import("__main__");
object main_namespace = pyMainModule.attr("__dict__");
//What previously I intended to do
//main_namespace["Motion"] = enum_<TestClass::Motion>("Motion")
// .value("walk", TestClass::walk)
// .value("bike", TestClass::bike)
//;
//I want to use my enum here
//I need something like line below which makes me able to use the enum!
exec("print 'hello world'", main_namespace, main_namespace);
}
catch(error_already_set const&)
{
PyErr_Print();
}
Py_Finalize();
return 0;
}
Anything useful to know about wrapping and using Enums in Python will be appreciated!
Thanks in advance
The AttributeError is the result of trying to create a Python extension type without first setting scope. The boost::python::enum_ constructor states:
Constructs an enum_ object holding a Python extension type derived from int which is named name. The named attribute of the current scope is bound to the new extension type.
When embedding Python, to use a custom Python module, it is often easiest to use PyImport_AppendInittab, then import the module by name.
PyImport_AppendInittab("example", &initexample);
...
boost::python::object example = boost::python::import("example");
Here is a complete example showing two enumerations being exposed through Boost.Python. One is included in a separate module (example) that is imported by main, and the other is exposed directly in main.
#include <iostream>
#include <boost/python.hpp>
/// #brief Mockup class with a nested enum.
struct TestClass
{
/// #brief Mocked enum.
enum Motion
{
walk,
bike
};
// #brief Mocked enum.
enum Color
{
red,
blue
};
};
/// #brief Python example module.
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::enum_<TestClass::Motion>("Motion")
.value("walk", TestClass::walk)
.value("bike", TestClass::bike)
;
}
int main()
{
PyImport_AppendInittab("example", &initexample); // Add example to built-in.
Py_Initialize(); // Start interpreter.
// Create the __main__ module.
namespace python = boost::python;
try
{
python::object main = python::import("__main__");
python::object main_namespace = main.attr("__dict__");
python::scope scope(main); // Force main scope
// Expose TestClass::Color as Color
python::enum_<TestClass::Color>("Color")
.value("red", TestClass::red)
.value("blue", TestClass::blue)
;
// Print values of Color enumeration.
python::exec(
"print Color.values",
main_namespace, main_namespace);
// Get a handle to the Color enumeration.
python::object color = main_namespace["Color"];
python::object blue = color.attr("blue");
if (TestClass::blue == python::extract<TestClass::Color>(blue))
std::cout << "blue enum values matched." << std::endl;
// Import example module into main namespace.
main_namespace["example"] = python::import("example");
// Print the values of the Motion enumeration.
python::exec(
"print example.Motion.values",
main_namespace, main_namespace);
// Check if the Python enums match the C++ enum values.
if (TestClass::bike == python::extract<TestClass::Motion>(
main_namespace["example"].attr("Motion").attr("bike")))
std::cout << "bike enum values matched." << std::endl;
}
catch (const python::error_already_set&)
{
PyErr_Print();
}
}
Output:
{0: __main__.Color.red, 1: __main__.Color.blue}
blue enum values matched.
{0: example.Motion.walk, 1: example.Motion.bike}
bike enum values matched.