I had one c++ program that inside for loop, calling a function.
The function is doing a heavy process, it is embedded with python and performing image processing.
My question is, why can it only run at the first instance of the variable?
Main function (I only show the part of code require in this title):
int main(){
for(int a = 0;a<5;a++){
for(int b=0;b<5;b++){
// I want every increment it go to PyRead() function, doing image processing, and compare
if(PyRead()==1){
// some application might be occur
}
else {
}
}
}
PyRead() function, the function in c++ to go into python environment performing image processing:
bool PyRead(){
string data2;
Py_Initialize();
PyRun_SimpleString("print 'hahahahahawwwwwwwwwwwww' ");
char filename[] = "testcapture";
PyRun_SimpleString("import sys");
PyRun_SimpleString("sys.path.append(\".\")");
PyObject * moduleObj = PyImport_ImportModule(filename);
if (moduleObj)
{
PyRun_SimpleString("print 'hahahahaha' ");
char functionName[] = "test";
PyObject * functionObj = PyObject_GetAttrString(moduleObj, functionName);
if (functionObj)
{
if (PyCallable_Check(functionObj))
{
PyObject * argsObject = PyTuple_New(0);
if (argsObject)
{
PyObject * resultObject = PyEval_CallObject(functionObj, argsObject);
if (resultObject)
{
if ((resultObject != Py_None)&&(PyString_Check(resultObject)))
{
data2 = PyString_AsString(resultObject);
}
Py_DECREF(resultObject);
}
else if (PyErr_Occurred()) PyErr_Print();
Py_DECREF(argsObject);
}
}
Py_DECREF(functionObj);
}
else PyErr_Clear();
Py_DECREF(moduleObj);
}
Py_Finalize();
std::cout << "The Python test function returned: " << data2<< std::endl;
cout << "Data2 \n" << data2;
if(compareID(data2) == 1)
return true;
else
return false;
}
This is second time I ask this question in stack overflow. I hope this time this question will be more clear!
I can successful compile with no error.
When I run the program, I realize at a=0, b=0 it will go to PyRead() function and return value, after that it go to a=0, b=1, at that moment the whole program will end.
It supposes to go to PyRead() function again, but it does not do that and straight ending the program.
I must strongly mention that PyRead() function needed a long time to run (30seconds).
I had no idea what happens, seeking for somehelp. Please focus on the Bold part to understand my question.
Thanks.
See the comment in https://docs.python.org/2/c-api/init.html#c.Py_Finalize
Ideally, this frees all memory allocated by the Python interpreter.
Dynamically loaded extension modules loaded by Python are not unloaded.
Some extensions may not work properly if their initialization routine is called more than once
It seems your module, does not play well with this function.
A workaround can be - create the script on the fly and call it with python subprocess.
Related
I'm having difficulty with creating a PyTupleObject using the Python C api.
#include "Python.h"
int main() {
int err;
Py_ssize_t size = 2;
PyObject *the_tuple = PyTuple_New(size); // this line crashes the program
if (!the_tuple)
std::cerr << "the tuple is null" << std::endl;
err = PyTuple_SetItem(the_tuple, (Py_ssize_t) 0, PyLong_FromLong((long) 5.7));
if (err < 0) {
std::cerr << "first set item failed" << std::endl;
}
err = PyTuple_SetItem(the_tuple, (Py_ssize_t) 1, PyLong_FromLong((long) 5.7));
if (err < 0) {
std::cerr << "second set item failed" << std::endl;
}
return 0;
}
crashes with
Process finished with exit code -1073741819 (0xC0000005)
But so does everything else i've tried so far. Any ideas what I'm doing wrong? Not that I'm just trying to run the as a C++ program, as I'm just trying to do tests on the code before adding a swig typemap.
The commenter #asynts is correct in that you need to initialize the interpreter via Py_Initialize if you want to interact with Python objects (you are, in fact, embedding Python). There are a subset of functions from the API that can safely be called without initializing the interpreter, but creating Python objects do not fall within this subset.
Py_BuildValue may "work" (as in, not creating a segfault with those specific arguments), but it will cause issues elsewhere in the code if you try to do anything with it without having initialized the interpreter.
It seems that you're trying to extend Python rather than embed it, but you're embedding it to test the extension code. You may want to refer to the official documentation for extending Python with C/C++ to guide you through this process.
I am trying to embed Python in a C++ multi-threading program using the Python/C API (version 3.7.3) on a quad-core ARM 64 bit architecture. A dedicated thread-safe class "PyHandler" takes care of all the Python API calls:
class PyHandler
{
public:
PyHandler();
~PyHandler();
bool run_fun();
// ...
private:
PyGILState_STATE _gstate;
std::mutex _mutex;
}
In the constructor I initialize the Python interpreter:
PyHandler::PyHandler()
{
Py_Initialize();
//PyEval_SaveThread(); // UNCOMMENT TO MAKE EVERYTHING WORK !
}
And in the destructor I undo all initializations:
PyHandler::~PyHandler()
{
_gstate = PyGILState_Ensure();
if (Py_IsInitialized()) // finalize python interpreter
Py_Finalize();
}
Now, in order to make run_fun() callable by one thread at a time, I use the mutex variable _mutex (see below). On top of this, I call PyGILState_Ensure() to make sure the current thread holds the python GIL, and call PyGILState_Release() at the end to release it. All the remaining python calls happen within these two calls:
bool PyHandler::run_fun()
{
std::lock_guard<std::mutex> lockGuard(_mutex);
_gstate = PyGILState_Ensure(); // give the current thread the Python GIL
// Python calls...
PyGILState_Release(_gstate); // release the Python GIL till now assigned to the current thread
return true;
}
Here is how the main() looks like:
int main()
{
PyHandler py; // constructor is called !
int n_threads = 10;
std::vector<std::thread> threads;
for (int i = 0; i < n_threads; i++)
threads.push_back(std::thread([&py]() { py.run_fun(); }));
for (int i = 0; i < n_threads; i++)
if (threads[i].joinable())
threads[i].join();
}
Although all precautions, the program always deadlocks at the PyGILState_Ensure() line in run_fun() during the very first attempt. BUT when I uncomment the line with PyEval_SaveThread() in the constructor everything magically works. Why is that ?
Notice that I am not calling PyEval_RestoreThread() anywhere. Am I supposed to use the macros Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS instead ? I thought these macros and PyEval_SaveThread() are only used dealing with Python threads and NOT with non-Python threads, as in my case! Am I missing something ?
The documentation for my case, only mentions the use of PyGILState_Ensure() and PyGILState_Release. Any help is highly appreciated.
I'm been struggling with this problem pretty long. I have to communicate with an Arduino and C++ program. I know it is easy in Python, but it has to be in C++ for school project. So I'm reading a lot of things, but no one help me further.
So I have a short Python script that communicate with the arduino very well and fast written in Python.
communication.py:
import serial
ser = serial.Serial('/dev/ttyACM0', 9600)
def communicate(number):
ser.write(number)
return ser.readline()
The Arduino gets the number and returns values of light sensors in a string. Now I want this to communicate with C++ program. My C++ code (this is only a small part of the full program)
mainwindow.cpp:
void MainWindow::run(){
while (ui->startstop->value() == 1){// as long that the program must run
blockchange = 0;
// get data from arduino
Py_Initialize();
const char* modulename = "communication";
PyObject *pName = PyUnicode_FromString(modulename);
PyObject *pModule = PyImport_Import(pName);
if (pModule != NULL){
PyObject *pDict = PyModule_GetDict(pModule);
PyObject *pFunc = PyDict_GetItem(pDict, PyUnicode_FromString("communicate"));
if (pFunc != NULL){
PyObject_CallObject(pFunc, PyLong_FromLong(blockchange));
}else{
std::cout << "couldn't find func\n";
}
}else{
std::cout << "pyhton module not found\n";
}
}
It only gives "python module not found". Which means that PyImport_Import(pName) returns NULL. What is wrong?
I use Ubuntu 18.04 and my standard version of Python is 3.5 and the program is written in Qt Creator. I tried a lot of things, also without Python, but I haven't found anything that works. I only want that my Arduino reads one int from 0 to 6, and that the C++ program reads a string of 6 numbers separated with a ",".
I'm using Zbar with it's Processor option in Python. I've been trying to figure out how to limit the symbology to QR-code only, but have only found answers for C as it follows:
scanner = new ImageScanner();
scanner.setConfig(Symbol.QRCODE, Config.ENABLE, 1);
I understand that the original code is written for C but is there anyway to do it in Python? Python isn't my main language and it's a bit difficult for me to understand what the arguments are in this case for processor.parse_config() (which I have currently set to 'enable'):
From https://github.com/npinchot/zbar/blob/master/processor.c
static PyObject*
processor_parse_config (zbarProcessor *self,
PyObject *args,
PyObject *kwds)
{
const char *cfg = NULL;
static char *kwlist[] = { "config", NULL };
if(!PyArg_ParseTupleAndKeywords(args, kwds, "s", kwlist, &cfg))
return(NULL);
if(zbar_processor_parse_config(self->zproc, cfg)) {
PyErr_Format(PyExc_ValueError, "invalid configuration setting: %s",
cfg);
return(NULL);
}
Py_RETURN_NONE;
}
I don't even understand why 'enable' is a valid argument.
Took me some time to figure this out since there's no documentation and the config format is counter-intuitive, IMO, but here you go:
proc.parse_config('disable')
proc.parse_config('qrcode.enable')
The first line, disable, disables all scanners.
The second line enables the qrcode scanner.
I've written some C-code to call scipy functions. The body, including variable declarations and using EXIT FAIL to denote messages and cleanup steps, is:
PyObject *module_name, *module = NULL;
PyObject *funct = NULL;
PyObject *output = NULL;
int j;
double dInVal, dOutVal;
Py_Initialize();
module_name = PyString_FromString("scipy.stats");
module = PyImport_Import(module_name);
Py_DECREF(module_name);
if (!module)
EXIT FAIL
funct = PyObject_GetAttrString(module, "beta");
if (!funct)
EXIT FAIL
Py_DECREF(module);
for (j=0; j<=10; j++)
{
dInVal = (double)j/10.0;
output = PyObject_CallMethod(funct, "ppf", "(f,f,f)", dInVal, 50.0, 50.0);
if (!output)
EXIT FAIL
dOutVal = PyFloat_AsDouble(output);
Py_DECREF(output);
printf("%6.3f %6.3f\n", dInVal, dOutVal);
}
Py_DECREF(funct);
Py_Finalize();
When I run this as the main routine, it appears to work fine. However, when I run it as a subroutine, it works for a first call, but fails on any subsequent call.
The code does work as a subroutine after the first call, if I make all of the PyObject pointers static (and include the appropriate flags, so that Python is initialized and the "beta" object is imported only once), but make-everything-static seems like a brute force solution to the problem, especially if the program will eventually include more than one subroutine that calls Python.
My question is, what is the best practice for setting up a C-program, to call scipy from a subroutine?