Related
I am calling a Python function from my C++ code. The Python function takes two numbers as input and returns the sum. I am unable to retrieve the sum value in C++ code. Below are code for both C++ and Python files.
C++ code:
#include<iostream>
#include "Python.h"
int main()
{
Py_Initialize();
PyRun_SimpleString("import sys; sys.path.insert(0, '/home/aakash/Desktop/code/testpc')");
//Run a python function
PyObject *pName, *pModule, *pFunc, *pArgs, *pValue;
pName = PyUnicode_FromString((char*)"sample");
pModule = PyImport_Import(pName);
pFunc = PyObject_GetAttrString(pModule, (char*)"main2");
// pArgs = PyTuple_Pack(1, PyUnicode_FromString((char*)"Greg"));
pArgs = PyTuple_New(2);
PyTuple_SetItem(pArgs, 0, PyLong_FromLong(5));
PyTuple_SetItem(pArgs, 1, PyLong_FromLong(6));
pValue = PyObject_CallObject(pFunc, pArgs);
auto result = _PyUnicode_AsString(pValue);
std::cout << result << std::endl;
Py_Finalize();
return 0;
}
Python code in sample.py file:
def main2(a, b):
print("getting sum ")
c = a + b
return c
Getting error:- TypeError: bad argument type for built-in operation.
Request to help for other datatypes too.
I got the solution by replacing auto result = _PyUnicode_AsString(pValue); with int result = PyFloat_AsDouble(pValue);. But I am not fully convinced as I needed an integer, but I had to use a function which looks to be meant for dealing with Float/Double. So, waiting for an accurate answer. Thanks!
Introduction
I'm trying to update QT GUI element basing on the state of calculations in embedded python script. I'm able to extract required values from python, but can't set a reference to c++ object to make it work.
The details
Let's assume python code is called (in calc.cpp) this way:
void class_name::transfer(varA, varB, varC)
{
Py_Initialize();
emit inprogress(70); //HERE IT WORKS
object module = import("__main__");
object name_space = module.attr("__dict__");
exec_file("MyModule.py", name_space, name_space);
object MyFunc = name_space["MyFunc"];
object result = MyFunc(varA, varB, varC, callback);
double ret = extract<double>(result);
Py_Finalize();
}
void class_name::callback(double t_prog, double t_final)
{
progr = (double)t_prog / t_final * 100;
cout << progr; //To check if value is updating (It is)
emit inprogress(progr); //HERE IT FAIL
}
callback is a static member function (in calc.cpp) I use to extract some values, indicating on which stage is calculation inside python script. It is called in loop from python script (MyModule.py):
while r.successful() and k < num_steps:
r.integrate(r.t + delta_t)
callback(r.t, t_final)
However the compilation fails with following error:
illegal call to nonstatic member function
nonstatic member reference must be relative to specific object
It is related to emit inprogress(progr);
The Question
I think I should pass a reference to the object from my c++ to python, and back to c++ with callback. But I can't find a way how to do this. What's the correct way to do this?
Tested ideas (still not working)
I was trying to pass it simple like:
void class_name::callback(double t_prog, double t_final, class_name &cssd) but python seems to can't convert it automatically.
Creating of new class object:
class_name cs;
emit cs.inprogress(progr);
Compile without error, but signal never reach the slot - it creates new object instead of refer to existing.
We need to add some additional state to the callback, namely a reference to the instance of the class whose member function we want to invoke.
To do this, we can use a class. To make the functionality equivalent to just using a simple static callback function, let's define operator() (i.e. make it a functor), and also expose this operator to Python.
Let's suppose we have the following application class:
class app
{
public:
explicit app(std::string name) : name_(std::move(name)) {}
int run();
void callback(double t_prog, double t_final);
private:
std::string name_;
};
In run() we execute our Python script, and we want it to call the member function callback of the current instance.
Let's define the following callback handler class:
class callback_handler
{
public:
explicit callback_handler(app& a) : app_(a) {}
void operator()(double t_prog, double t_final)
{
app_.callback(t_prog, t_final);
}
private:
app& app_;
};
We need to expose this class to Python, but don't want to be able to create new instances from Python, nor do we really want it to be copied (although it doesn't really matter here, since our state only consists of references).
BOOST_PYTHON_MODULE(cbtest)
{
bp::class_<callback_handler, boost::noncopyable>("callback_handler", bp::no_init)
.def("__call__", &callback_handler::operator())
;
};
At the start of our application, we need to make sure to initialize our module before we use it -- call initcbtest(); right after initializing the Python interpreter.
Now we can use our callback handler in the following manner (The Python code stays the same, since the object is callable):
callback_handler cbh(*this);
bp::object result = MyFunc(1, 10, 2, boost::ref(cbh));
std::cout << "result = " << bp::extract<double>(result) << "\n";
Sample Code
#include <boost/noncopyable.hpp>
#include <boost/python.hpp>
#include <iostream>
// ============================================================================
namespace bp = boost::python;
// ============================================================================
class app
{
public:
explicit app(std::string name) : name_(std::move(name)) {}
int run();
void callback(double t_prog, double t_final);
private:
std::string name_;
};
// ============================================================================
class callback_handler
{
public:
explicit callback_handler(app& a) : app_(a) {}
void operator()(double t_prog, double t_final)
{
app_.callback(t_prog, t_final);
}
private:
app& app_;
};
// ----------------------------------------------------------------------------
BOOST_PYTHON_MODULE(cbtest)
{
bp::class_<callback_handler, boost::noncopyable>("callback_handler", bp::no_init)
.def("__call__", &callback_handler::operator())
;
};
// ============================================================================
void app::callback(double t_prog, double t_final)
{
std::cout << "CB(" << name_ << ") " << t_prog << " " << t_final << "\n";
}
// ----------------------------------------------------------------------------
int app::run()
{
Py_Initialize();
initcbtest();
try {
bp::object module = bp::import("__main__");
bp::object name_space = module.attr("__dict__");
bp::exec_file("MyModule.py", name_space, name_space);
bp::object MyFunc = name_space["MyFunc"];
callback_handler cbh(*this);
bp::object result = MyFunc(1, 10, 2, boost::ref(cbh));
std::cout << "result = " << bp::extract<double>(result) << "\n";
} catch (bp::error_already_set&) {
PyErr_Print();
}
Py_Finalize();
return 0;
}
// ============================================================================
int main()
{
app a("TestApp");
return a.run();
}
// ============================================================================
Python Script
File MyModule.py:
def MyFunc(a, b, c, callback):
result = 0
for i in range(a, b, c):
result += i
callback(i, b)
return result
Console Output
CB(TestApp) 0 10
CB(TestApp) 2 10
CB(TestApp) 4 10
CB(TestApp) 6 10
CB(TestApp) 8 10
result = 20
Maybe the problem is that you are calling emit c_n.inprogress(progr);, where the argument of the signal progr is of type double, whereas in the connect(sender, SIGNAL( inprogress(int) ), ui->progressBar, SLOT( setValue(int) ) ); the signal takes an integer as argument. In older Qt versions(older than Qt5), signals and slots must use exactly the same types, meaning that the implicit conversion may not occur.
https://forum.qt.io/topic/23302/connect-diferent-signals-and-slost-each-other/4
I'm using the JCC Python-Java bridge and for the most part it works. However, I'm getting the following error:
JRE version: 7.0_17-b02
Java VM: Java HotSpot(TM) Client VM (23.7-b01 mixed mode linux-x86 )
Problematic frame:
C [_ciasliveschema.so+0x21e75c] boxJObject(_typeobject*, _object*, java::lang::Object*)+0x22c
The stack dump is as follows:
Stack: [0xbfbe5000,0xbfc35000], sp=0xbfc33820, free space=314k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [_ciasliveschema.so+0x21e75c] boxJObject(_typeobject*, _object*, java::lang::Object*)+0x22c
C [_ciasliveschema.so+0x221977] boxObject(_typeobject*, _object*, java::lang::Object*)+0x27
C [_ciasliveschema.so+0x225149] _parseArgs(_object*, unsigned int, char, ...)+0x2f69
C [_ciasliveschema.so+0x17e21c] schema::util::t_IndividualCaster_asMessage
(schema::util::t_IndividualCaster*, _object*)+0xac
C [python+0x8bda4] PyEval_EvalFrameEx+0x6494
C [python+0x8ccb1] PyEval_EvalCodeEx+0x871
C [python+0xe0a0c] fileno##GLIBC_2.0+0xe0a0c
C [python+0x143b5] PyObject_Call+0x45
C [python+0x1b107] fileno##GLIBC_2.0+0x1b107
C [python+0x143b5] PyObject_Call+0x45
C [python+0x84a72] PyEval_CallObjectWithKeywords+0x42
C [python+0x1eec1] PyInstance_New+0x71
C [python+0x143b5] PyObject_Call+0x45
C [python+0x86923] PyEval_EvalFrameEx+0x1013
C [python+0x8b347] PyEval_EvalFrameEx+0x5a37
C [python+0x8ccb1] PyEval_EvalCodeEx+0x871
C [python+0x8cd47] PyEval_EvalCode+0x57
The code for the boxJObject function is as follows:
static int boxJObject(PyTypeObject *type, PyObject *arg,
java::lang::Object *obj)
{
if (arg == Py_None)
{
if (obj != NULL)
*obj = Object(NULL);
}
else if (PyObject_TypeCheck(arg, &PY_TYPE(Object)))
{
if (type != NULL && !is_instance_of(arg, type))
return -1;
if (obj != NULL)
*obj = ((t_Object *) arg)->object;
}
else if (PyObject_TypeCheck(arg, &PY_TYPE(FinalizerProxy)))
{
arg = ((t_fp *) arg)->object;
if (PyObject_TypeCheck(arg, &PY_TYPE(Object)))
{
if (type != NULL && !is_instance_of(arg, type))
return -1;
if (obj != NULL)
*obj = ((t_Object *) arg)->object;
}
else
return -1;
}
else
return 1;
return 0;
}
This is called in the following manner:
int result = boxJObject(type, arg, obj);
Also, I have modified the following section of the jcc.cpp:initVM() method:
if (JNI_CreateJavaVM(&vm, (void **) &vm_env, &vm_args) < 0)
{
for (unsigned int i = 0; i < nOptions; i++)
delete vm_options[i].optionString;
PyErr_Format(PyExc_ValueError,
"An error occurred while creating Java VM");
return NULL;
}
As follows:
vm_args.nOptions = nOptions;
vm_args.ignoreUnrecognized = JNI_FALSE;
vm_args.options = vm_options;
vmInitSuccess = JNI_CreateJavaVM(&vm, (void **) &vm_env, &vm_args);
if (vmInitSuccess < 0)
{
for (unsigned int i = 0; i < nOptions; i++)
delete vm_options[i].optionString;
//Set up basic error message
sprintf(strVMInitSuccess, "%d", vmInitSuccess);
strcpy(strVMError, "An error occurred while creating Java VM (No Exception): ");
strcat(strVMError, strVMInitSuccess);
//Get exception if there is one
if((exc = vm_env->ExceptionOccurred()))
{
//Clear the exception since we have it now
vm_env->ExceptionClear();
//Get the getMessage() method
if ((java_class = vm_env->FindClass ("java/lang/Throwable")))
{
if ((method = vm_env->GetMethodID(java_class, "getMessage", "()Ljava/lang/String;")))
{
int size;
strExc = static_cast<jstring>(vm_env->CallObjectMethod(exc, method));
charExc = vm_env->GetStringUTFChars(strExc, NULL);
size = sizeof(strVMError) + sizeof(charExc);
char strVMException[size];
strcpy(strVMException, "An error occurred while creating Java VM (Exception): ");
strcat(strVMException, charExc);
PyErr_Format(PyExc_ValueError, strVMException);
return NULL;
}
}
}
PyErr_Format(PyExc_ValueError, strVMError);
return NULL;
}
This was to attempt to get a more detailed error message back from JCC when errors occur, so it is possible that this is the source of the error (although the segfault error and stack trace above suggest otherwise).
Finally, I am currently invoking the initVM() method from within Python as follows:
self.__cslschemalib = ciasliveschema.initVM(classpath=ciasliveschema.CLASSPATH)
However, when I attempt to invoke the method as follows (to increase the amount of memory available):
self.__cslschemalib = ciasliveschema.initVM(classpath=ciasliveschema.CLASSPATH, initialheap='512m', maxheap='2048m', maxstack='2048m', vmargs='-Xcheck:jni,-verbose:jni,-verbose:gc')
I get the following error:
JRE version: 7.0_17-b02
Java VM: Java HotSpot(TM) Client VM (23.7-b01 mixed mode linux-x86 )
Problematic frame:
C 0x00000000
And stack trace:
Stack: [0xbf6e0000,0xbf8e0000], sp=0xbf8dd580, free space=2037k
Any suggestions?
The problem has been resolved. The problem actually arose in the call to the above boxJObject method:
if (boxObject(NULL, arg, obj) < 0) return -1;
Which is the _parseArgs function in functions.cpp of the JCC source code.
The problem arose because (at least from a quick scan of this function), _parseArgs checks to see whether more args have been passed than the method accepts, but does not check for the case in which fewer args have been passed.
In the call to the IndividualCaster().asMessage() method in my Python code I was passing just one argument when the method actually takes two. While I haven't located the precise source of the error in the _parseArgs method, I have corrected the call in my Python code so that it now takes the correct number of args, and the seg fault is no longer occurring.
I have an if clause within a for loop in which I have defined state_out beforehand with:
state_out = (PyArrayObject *) PyArray_FromDims(1,dims_new,NPY_BOOL);
And the if conditions are like this:
if (conn_ctr<sum*2){
*(state_out->data + i*state_out->strides[0]) = true;
}
else {
*(state_out->data + i*state_out->strides[0]) = false;
}
When commenting these out, state_out returns as an all-False Numpy array. There is a problem with this assignment that I fail to see. As far as I know, all within the struct PyArrayObject that are called here in this code are pointers, so after the pointer arithmetic, it should be pointing to the address I intend to write. (All if conditions in the code are built by reaching values in this manner, and I know it works, since I managed to printf input arrays' values.) Then if I want to assign a bool to one of these parts in the memory, I should assign it via *(pointer_intended) = true What am I missing?
EDIT: I have spotted that even if I don't reach those values even if I put some printf functions within:
if (conn_ctr<sum*2){
printf("True!\n");
}
else {
printf("False!\n");
}
I get a SegFault again.
Thanks a lot, an the rest of the code is here.
#include <Python.h>
#include "numpy/arrayobject.h"
#include <stdio.h>
#include <stdbool.h>
static PyObject* trace(PyObject *self, PyObject *args);
static char doc[] =
"This is the C extension for xor_masking routine. It interfaces with Python via C-Api, and calculates the"
"next state with C pointer arithmetic";
static PyMethodDef TraceMethods[] = {
{"trace", trace, METH_VARARGS, doc},
{NULL, NULL, 0, NULL}
};
PyMODINIT_FUNC
inittrace(void)
{
(void) Py_InitModule("trace", TraceMethods);
import_array();
}
static PyObject* trace(PyObject *self, PyObject *args){
PyObject *adjacency ,*mask, *state;
PyArrayObject *adjacency_arr, *mask_arr, *state_arr, *state_out;
if (!PyArg_ParseTuple(args,"OOO:trace", &adjacency, &mask, &state)) return NULL;
adjacency_arr = (PyArrayObject *)
PyArray_ContiguousFromObject(adjacency, NPY_BOOL,2,2);
if (adjacency_arr == NULL) return NULL;
mask_arr = (PyArrayObject *)
PyArray_ContiguousFromObject(mask, NPY_BOOL,2,2);
if (mask_arr == NULL) return NULL;
state_arr = (PyArrayObject *)
PyArray_ContiguousFromObject(state, NPY_BOOL,1,1);
if (state_arr == NULL) return NULL;
int dims[2], dims_new[1];
dims[0] = adjacency_arr -> dimensions[0];
dims[1] = adjacency_arr -> dimensions[1];
dims_new[0] = adjacency_arr -> dimensions[0];
if (!(dims[0]==dims[1] && mask_arr -> dimensions[0] == dims[0]
&& mask_arr -> dimensions[1] == dims[0]
&& state_arr -> dimensions[0] == dims[0]))
return NULL;
state_out = (PyArrayObject *) PyArray_FromDims(1,dims_new,NPY_BOOL);
int i,j;
for(i=0;i<dims[0];i++){
int sum = 0;
int conn_ctr = 0;
for(j=0;j<dims[1];j++){
bool adj_value = (adjacency_arr->data + i*adjacency_arr->strides[0]
+j*adjacency_arr->strides[1]);
if (*(bool *) adj_value == true){
bool mask_value = (mask_arr->data + i*mask_arr->strides[0]
+j*mask_arr->strides[1]);
bool state_value = (state_arr->data + j*state_arr->strides[0]);
if ( (*(bool *) mask_value ^ *(bool *)state_value) == true){
sum++;
}
conn_ctr++;
}
}
if (conn_ctr<sum*2){
}
else {
}
}
Py_DECREF(adjacency_arr);
Py_DECREF(mask_arr);
Py_DECREF(state_arr);
return PyArray_Return(state_out);
}
if (conn_ctr<sum*2){
*(state_out->data + i*state_out->strides[0]) = true;
}
else {
*(state_out->data + i*state_out->strides[0]) = false;
}
Here, I naively make a pointer arithmetic, state_out->data is a pointer to the beginning of data, it is defined to be a pointer of char:SciPy Doc - Python Types and C-Structures
typedef struct PyArrayObject {
PyObject_HEAD
char *data;
int nd;
npy_intp *dimensions;
npy_intp *strides;
...
} PyArrayObject;
Which a portion of I copied here. state_out->strides is a pointer to an array of length of the dimension of the array we have. This is a 1d array in this case. So when I make the pointer arithmetic (state_out->data + i*state_out->strides[0]) I certainly aim to calculate the pointer that points the ith value of the array, but I failed to give the type of the pointer, so the
I had tried :
NPY_BOOL *adj_value_ptr, *mask_value_ptr, *state_value_ptr, *state_out_ptr;
which the variables are pointing towards the values that I am interested in my for loop, and state_out_ptr is the one that I am writing to. I had thought that since I state that the
constituents of these arrays are of type NPY_BOOL, the pointers that point to the data within the array would be of type NPY_BOOL also. This fails with a SegFault when one is working with data directly manipulating the memory. This is from the fact that NPY_BOOL is an enum for an integer (as pv kindly stated in the comments.) for NumPy to use internally,.There is a C typedef npy_bool in order to use within the code for boolean values. Scipy Docs. When I introduced my pointers with the type
npy_bool *adj_value_ptr, *mask_value_ptr, *state_value_ptr, *state_out_ptr;
Segmentation fault disappeared, and I succeeded in manipulating and returning a Numpy Array.
I'm not an expert, but this solved my issue, point out if I'm wrong.
The part that has changed in the source code is:
state_out = (PyArrayObject *) PyArray_FromDims(1,dims_new,NPY_BOOL);
npy_bool *adj_value_ptr, *mask_value_ptr, *state_value_ptr, *state_out_ptr;
npy_intp i,j;
for(i=0;i<dims[0];i++){
npy_int sum = 0;
npy_int conn_ctr = 0;
for(j=0;j<dims[1];j++){
adj_value_ptr = (adjacency_arr->data + i*adjacency_arr->strides[0]
+j*adjacency_arr->strides[1]);
if (*adj_value_ptr == true){
mask_value_ptr = (mask_arr->data + i*mask_arr->strides[0]
+j*mask_arr->strides[1]);
state_value_ptr = (state_arr->data + j*state_arr->strides[0]);
if ( (*(bool *) mask_value_ptr ^ *(bool *)state_value_ptr) == true){
sum++;
}
conn_ctr++;
}
}
state_out_ptr = (state_out->data + i*state_out->strides[0]);
if (conn_ctr < sum*2){
*state_out_ptr = true;
}
else {
*state_out_ptr = false;
}
}
I'm having some trouble figuring out the proper way to walk a Python traceback using the C API. I'm writing an application that embeds the Python interpreter. I want to be able to execute arbitrary Python code, and if it raises an exception, to translate it to my own application-specific C++ exception. For now, it is sufficient to extract just the file name and line number where the Python exception was raised. This is what I have so far:
PyObject* pyresult = PyObject_CallObject(someCallablePythonObject, someArgs);
if (!pyresult)
{
PyObject* excType, *excValue, *excTraceback;
PyErr_Fetch(&excType, &excValue, &excTraceback);
PyErr_NormalizeException(&excType, &excValue, &excTraceback);
PyTracebackObject* traceback = (PyTracebackObject*)traceback;
// Advance to the last frame (python puts the most-recent call at the end)
while (traceback->tb_next != NULL)
traceback = traceback->tb_next;
// At this point I have access to the line number via traceback->tb_lineno,
// but where do I get the file name from?
// ...
}
Digging around in the Python source code, I see they access both the filename and module name of the current frame via the _frame structure, which looks like it is a privately-defined struct. My next idea was to programmatically load the Python 'traceback' module and call its functions with the C API. Is this sane? Is there a better way to access a Python traceback from C?
This is an old question but for future reference, you can get the current stack frame from the thread state object and then just walk the frames backward. A traceback object isn't necessary unless you want to preserve the state for the future.
For example:
PyThreadState *tstate = PyThreadState_GET();
if (NULL != tstate && NULL != tstate->frame) {
PyFrameObject *frame = tstate->frame;
printf("Python stack trace:\n");
while (NULL != frame) {
// int line = frame->f_lineno;
/*
frame->f_lineno will not always return the correct line number
you need to call PyCode_Addr2Line().
*/
int line = PyCode_Addr2Line(frame->f_code, frame->f_lasti);
const char *filename = PyString_AsString(frame->f_code->co_filename);
const char *funcname = PyString_AsString(frame->f_code->co_name);
printf(" %s(%d): %s\n", filename, line, funcname);
frame = frame->f_back;
}
}
I prefer calling into python from C:
err = PyErr_Occurred();
if (err != NULL) {
PyObject *ptype, *pvalue, *ptraceback;
PyObject *pystr, *module_name, *pyth_module, *pyth_func;
char *str;
PyErr_Fetch(&ptype, &pvalue, &ptraceback);
pystr = PyObject_Str(pvalue);
str = PyString_AsString(pystr);
error_description = strdup(str);
/* See if we can get a full traceback */
module_name = PyString_FromString("traceback");
pyth_module = PyImport_Import(module_name);
Py_DECREF(module_name);
if (pyth_module == NULL) {
full_backtrace = NULL;
return;
}
pyth_func = PyObject_GetAttrString(pyth_module, "format_exception");
if (pyth_func && PyCallable_Check(pyth_func)) {
PyObject *pyth_val;
pyth_val = PyObject_CallFunctionObjArgs(pyth_func, ptype, pvalue, ptraceback, NULL);
pystr = PyObject_Str(pyth_val);
str = PyString_AsString(pystr);
full_backtrace = strdup(str);
Py_DECREF(pyth_val);
}
}
I've discovered that _frame is actually defined in the frameobject.h header included with Python. Armed with this plus looking at traceback.c in the Python C implementation, we have:
#include <Python.h>
#include <frameobject.h>
PyTracebackObject* traceback = get_the_traceback();
int line = traceback->tb_lineno;
const char* filename = PyString_AsString(traceback->tb_frame->f_code->co_filename);
But this still seems really dirty to me.
One principal I've found useful in writing C extensions is to use each language where it's best suited. So if you have a task to do that would be best implemented in Python, implement in Python, and if it would be best implemented in C, do it in C. Interpreting tracebacks is best done in Python for two reasons: first, because Python has the tools to do it, and second, because it isn't speed-critical.
I would write a Python function to extract the info you need from the traceback, then call it from C.
You could even go so far as to write a Python wrapper for your callable execution. Instead of invoking someCallablePythonObject, pass it as an argument to your Python function:
def invokeSomeCallablePythonObject(obj, args):
try:
result = obj(*args)
ok = True
except:
# Do some mumbo-jumbo with the traceback, etc.
result = myTraceBackMunger(...)
ok = False
return ok, result
Then in your C code, call this Python function to do the work. The key here is to decide pragmatically which side of the C-Python split to put your code.
I used the following code to extract Python exception's error body. strExcType stores the exception type and strExcValue stores the exception body. Sample values are:
strExcType:"<class 'ImportError'>"
strExcValue:"ImportError("No module named 'nonexistingmodule'",)"
Cpp code:
if(PyErr_Occurred() != NULL) {
PyObject *pyExcType;
PyObject *pyExcValue;
PyObject *pyExcTraceback;
PyErr_Fetch(&pyExcType, &pyExcValue, &pyExcTraceback);
PyErr_NormalizeException(&pyExcType, &pyExcValue, &pyExcTraceback);
PyObject* str_exc_type = PyObject_Repr(pyExcType);
PyObject* pyStr = PyUnicode_AsEncodedString(str_exc_type, "utf-8", "Error ~");
const char *strExcType = PyBytes_AS_STRING(pyStr);
PyObject* str_exc_value = PyObject_Repr(pyExcValue);
PyObject* pyExcValueStr = PyUnicode_AsEncodedString(str_exc_value, "utf-8", "Error ~");
const char *strExcValue = PyBytes_AS_STRING(pyExcValueStr);
// When using PyErr_Restore() there is no need to use Py_XDECREF for these 3 pointers
//PyErr_Restore(pyExcType, pyExcValue, pyExcTraceback);
Py_XDECREF(pyExcType);
Py_XDECREF(pyExcValue);
Py_XDECREF(pyExcTraceback);
Py_XDECREF(str_exc_type);
Py_XDECREF(pyStr);
Py_XDECREF(str_exc_value);
Py_XDECREF(pyExcValueStr);
}
I had reason to do this recently while writing an allocation tracker for numpy. The previous answers are close but frame->f_lineno will not always return the correct line number--you need to call PyFrame_GetLineNumber(). Here's an updated code snippet:
#include "frameobject.h"
...
PyFrameObject* frame = PyEval_GetFrame();
int lineno = PyFrame_GetLineNumber(frame);
PyObject *filename = frame->f_code->co_filename;
The full thread state is also available in the PyFrameObject; if you want to walk the stack keep iterating on f_back until it's NULL. Checkout the full data structure in frameobject.h: http://svn.python.org/projects/python/trunk/Include/frameobject.h
See also: https://docs.python.org/2/c-api/reflection.html
You can access Python traceback similar to tb_printinternal function. It iterates over PyTracebackObject list. I have tried also suggestions above to iterate over frames, but it does not work for me (I see only the last stack frame).
Excerpts from CPython code:
static int
tb_displayline(PyObject *f, PyObject *filename, int lineno, PyObject *name)
{
int err;
PyObject *line;
if (filename == NULL || name == NULL)
return -1;
line = PyUnicode_FromFormat(" File \"%U\", line %d, in %U\n",
filename, lineno, name);
if (line == NULL)
return -1;
err = PyFile_WriteObject(line, f, Py_PRINT_RAW);
Py_DECREF(line);
if (err != 0)
return err;
/* ignore errors since we are not able to report them, are we? */
if (_Py_DisplaySourceLine(f, filename, lineno, 4))
PyErr_Clear();
return err;
}
static int
tb_printinternal(PyTracebackObject *tb, PyObject *f, long limit)
{
int err = 0;
long depth = 0;
PyTracebackObject *tb1 = tb;
while (tb1 != NULL) {
depth++;
tb1 = tb1->tb_next;
}
while (tb != NULL && err == 0) {
if (depth <= limit) {
err = tb_displayline(f,
tb->tb_frame->f_code->co_filename,
tb->tb_lineno,
tb->tb_frame->f_code->co_name);
}
depth--;
tb = tb->tb_next;
if (err == 0)
err = PyErr_CheckSignals();
}
return err;
}
As of python 3.11, accessing the frame objects seems to need a different approach. Anyway, this works in 3.11, hth someone
py_err(void)
{
PyObject *err = PyErr_Occurred();
if (! err) {
return;
}
PyObject *ptype, *pvalue, *pbacktrace, *pyobj_str;
PyObject *ret, *list, *string;
PyObject *mod;
char *py_str;
PyErr_Fetch(&ptype, &pvalue, &pbacktrace);
PyErr_NormalizeException(&ptype, &pvalue, &pbacktrace);
PyErr_Display(ptype, pvalue, pbacktrace);
PyTraceBack_Print(pbacktrace, pvalue);
pyobj_str = PyObject_Str(pvalue);
py_str = py_obj_to_string(pyobj_str);
printf("%s", py_str);
myfree(py_str);
mod = PyImport_ImportModule("traceback");
list = PyObject_CallMethod(mod, "format_exception", "OOO", ptype, pvalue, pbacktrace);
if (list) {
string = PyUnicode_FromString("\n");
ret = PyUnicode_Join(string, list);
Py_DECREF(list);
Py_DECREF(string);
py_str = py_obj_to_string(ret);
printf("%s", py_str);
myfree(py_str);
Py_DECREF(ret);
}
PyErr_Clear();
}
and you will probably need this too
char *py_obj_to_string(const PyObject *py_str)
{
PyObject *py_encstr;
char *outstr = nullptr;
char *str;
py_encstr = nullptr;
str = nullptr;
if (! PyUnicode_Check((PyObject *) py_str)) {
goto err_out;
}
py_encstr = PyUnicode_AsEncodedString((PyObject *) py_str, "utf-8", nullptr);
if (! py_encstr) {
goto err_out;
}
str = PyBytes_AS_STRING(py_encstr);
if (! str) {
goto err_out;
}
outstr = strdup(str);
err_out:
if (py_encstr) {
Py_XDECREF(py_encstr);
}
return outstr;
}
actual working code if someone needs it can be found in my larger project https://github.com/goblinhack/zorbash