I have a python extension function
static void EXTrender_effect(EffectGlobals_t *effect_handle,
std::string preset,
bp::object dest, int xdim, int ydim)
{ ... }
that I export as usual in my boost.python extension:
def("render_effect", EXTrender_effect);
When I call that from python(2.7), I get a C++ exception boost::python::error_already_set. Tracing that down in Visual Studio, I can see it's coming from a boost::python::objects::function::argument_error. So OK, I have an argument error; I'm probably calling it with the wrong args. What I'd like to do is print or throw something sensible in my python extension when this happens, so users of my extension will see the nice message that I know is lurking in PyErr_Fetch. (I can see the message getting built in the boost.python argument_error code.)
But I can't catch that error_already_set; boost.python does that internally around the funcall. And I can't check for python errors in my extension code, since my function never gets called (the argument error is detected by boost.python). What ends up happening is it just silently fails.
What can I hook up so I (or my users) can see the argument error messages? And ideally convert those into python exceptions?
This is all Win7, Python2.7.
Related
I have a library which is a python wrapper for a C/C++ library, which needs to be imported to Python in Linux. Let's call the library abc.so. This library is dependent on another C/C++ library: xyz. Both these libraries used to have/have facilities that are dependent on Windows, Borland compiler or similar compilers. I am able to successfully build abc.so, after fixing some of the windows compiler related issues. However, I cannot import it to my python code. I receive the error:
ImportError: /usr/local/lib/abc.so: undefined symbol: _strnicmp
or a variant of this. I tried various import methods involving packages like ctpes, os, sys and flags like RTLD_LAZY, RTLD_GLOBAL, RTLD_NOW under the assumption that the method of import will fix this problem. However, none of them worked. This answer: undefined reference to stricmp (and the comment above) suggests that strnicmp should be replaced. It also points out that this is a link time error. However, I have not been able to identify part of these libraries expecting an implementation of strnicmp. What would be a good approach to find the source of this issue? Also, should I be trying some alternative path to fix this issue?
The stricmp() and strnicmp() functions are specific to Windows.
POSIX (Linux) uses the <strings.h> header and strcasecmp() and strncasecmp().
You can write a simple cover function or change the calls via a macro. For the cover function, you'd want to use appropriate conditional compilation (#ifdef SOMETHING / #endif — or maybe #ifndef SOMETHING / #endif). You might use an inline function if it only appears in one file. If it appears in many files, you may want a regular function, even though it's a smidgeon less efficient. If there's a convenient header included by all the files that you can modify to add (conditionally) the static inline function definition, that can work well — it's also about the only time you normally add a function body to a header file.
static inline int strnicmp(const char *s1, const char *s2, size_t len)
{
return strncasecmp(s1, s2, len);
}
or
#undef strnicmp
#define strnicmp(s1, s2, len) strncasecmp(s1, s2, len)
I have a C++ project which was written by others, which calls python code at the end of the execution.
In the C++ initializer, it defines a pHandle:
pHandle_ = PyObject_CallObject(pLoad_, NULL);
PyGILState_Release(gstate);
Then it calls python code this way:
PyObject *pyValue = PyObject_CallObject(pProcess_, pArgs);
pProcess_ and pArgs are created earlier. The python code's file name is 'runLogic.py' and the function executed in runLogic.py is 'process()'.
Is there a way to break into the python's process() function while I debug in c++ through GDB? In C++ code, I can step through until the line above, copied again below:
PyObject *pyValue = PyObject_CallObject(pProcess_, pArgs);
Then I don't know how to jump into the Python's function.
Is there a way to do that? I want to trace the full logic of the code, in addition to C++'s code.
Then I don't know how to jump into the Python's function.
Presumably you want to now step through Python code.
GDB doesn't know how to do that, but pdb can.
Assuming you can modify runLogic.py, add import pdb to the top of the file, and pdb.set_trace() in the process() function.
This will let you step through Python code. If you have breakpoints in C++, and Python code calls back into C++, these breakpoints will still be active, you should be able to debug both C++ and Python.
I have been working on a project where I want to remove the boost dependencies and replace it with the Python C API.
I spent some time understanding the Python C API and I saw this
catch (error_already_set const &)
I read the boost docs but it explains where it is used. But I want to know why it is needed and how can I achieve the same functionality using the native Python C api.
Boost throws error_already_set when a Python error has occurred. So if you see code like this:
try {
bp::exec(bp::str("garbage code is garbage"));
} catch (const bp::error_already_set&) {
// your code here to examine Python traceback etc.
}
you'll replace it with:
your_ptr<PyObject> res = PyRun_String("garbage code is garbage");
if (!res) {
// your code here to examine Python traceback etc.
}
In other words, wherever you see catch(error_already_set), there you will likely want to do some error handling using whatever PyObject* or other value is involved to recognize when an error has occurred (and therefore you can examine the traceback, or convert it into a C++ exception).
I have a shared library (DLL) in C++ with some C-style API which I use from Python. There is a function which takes a C callback as an argument. As far as I understood, Python extension module (a separate DLL) is required to do that. This module shall pass a native "proxy" callback to API, then call Python object from that callback. I try do use it as follows (it fails):
from ctypes import CDLL
from test_extension import set_python_callback
def callback():
pass
if __name__ == '__main__':
library = CDLL('test_dll.shared-library')
set_python_callback(library, callback)
library.trigger_callback()
I'm using Windows 8 x64 environment with Python 2.7.6 (32 bit) and MinGW 32-bit compiler (mingw32). Python extension is built by distutils. API library can be changed if necessary, however, Python-specific code has to be kept in a separate library.
Here is my code, reduced. All error-checking was removed, however, when performed it showed no errors either from Python API or from Windows API.
API library:
/* typedef void (*callback_t)(); */
static callback_t s_callback = nullptr;
void DLL_PUBLIC set_callback(callback_t callback) {
s_callback = callback;
}
void DLL_PUBLIC trigger_callback() {
s_callback(); // <--------------------------(1)
}
Python extension module function set_python_callback(library, callback) takes a ctypes.CDLL object and Python callable. It then extracts native DLL handle from the former:
PyObject* handle_object = PyObject_GetAttrString(library, "_handle");
const HMODULE handle = static_cast<const HMODULE>(
PyLong_AsVoidPtr(handle_object));
const native_set_callback_t native_api_routine =
reinterpret_cast<const native_set_callback_t>(
GetProcAddress(handle, "set_callback"));
native_api_routine(common_callback);
C function common_callback() is implemented as follows:
extern "C" void DLL_PUBLIC common_callback() {
// <--------------------------------------- (2)
PyObject *arguments = Py_BuildValue("()");
PyObject *callback_result = PyEval_CallObject(s_callback, arguments);
Py_DECREF(arguments);
Py_XDECREF(callback_result);
}
The error message says:
Invoking callback from DLL... Traceback (most recent call last):
File "script.py", line 14, in <module>
library.trigger_callback()
WindowsError: exception: access violation reading 0x00000028
Using debug print, I traced the error down to (1). Any code at (2) point doesn't execute. The strange thing is, changing the way common_callback() call Python code (e. g. passing Py_None instead of empty tuple) changes the address in the error message.
Maybe this is somehow related to the calling convention, but I have no idea where it's wrong exactly or how to fix it.
Solved the issue by registering the thread for GIL as described in documentation.
Please note that original API DLL, Python extension DLL, and the script were executed in single-threaded mode and no additional thread had been created, either Python-managed, or not. However, there had been a synchronization issue for sure, because using Py_AddPendingCall() also worked. (Using is is discouraged; I found it analyzing signals module sources, it lead to the solution above.)
This statement of mine was incorrect:
Using debug print, I traced the error down to (1). Any code at (2) point doesn't execute.
I was mislead by the fact that some of Python API functions still worked (e. g. Py_BuildValue or Py_Initialize), but most didn't (e. g. PySys_WriteStdout, PyObject_Call, etc).
I have been trying to get the python/c library to like my mingW compiler. The python online doncumentation; http://docs.python.org/c-api/intro.html#include-files only mentions that I need to import the python.h file. I grabbed it from the installation directory (as is required on the windows platform), and tested it by compiling the script:
#include "Python.h". This compiled fine. Next, I tried out the snippet of code shown a bit lower on the python/c API page:
PyObject *t;
t = PyTuple_New(3);
PyTuple_SetItem(t, 0, PyInt_FromLong(1L));
PyTuple_SetItem(t, 1, PyInt_FromLong(2L));
PyTuple_SetItem(t, 2, PyString_FromString("three"));
For some reason, the compiler would compile the code if I'd remove the last 4 lines (so that only the pyObject variable definition would be left), yet calling the actual constructor of the tuple returned errors.
I am probably missing something completely obvious here, given I am very new to C, but does anyone know what it is?
I've done some crafty Googling, and if you are getting errors at the linker stage (the error messages might have hex strings or references to ld), you may need to make sure the Python library that ships with the Windows version has been converted to a format that GCC (MinGW) can read; see here, among other sites. Also ensure that GCC can find and is using the library file if needs be, using -L/my/directory and -lpython26 (substituting appropriately for your path and Python version).
If the errors are at the compilation stage (if line numbers are given in the messages), make sure that you don't need to add any other directories to the include search path. Python might (I've not used its C API) include other header files in Python.h which are stored in some other directory. If this is the case, use the -I/my/directory/ flag to GCC to tell it to search there as well.
Exact (copied-and-pasted) error messages would help, though.
Warning: The text below does not answer the question!
Did you put the code inside a function? Try putting it in main, like so:
int main(int argc, char *argv[]) {
PyObject *t;
t = PyTuple_New(3);
PyTuple_SetItem(t, 0, PyInt_FromLong(1L));
PyTuple_SetItem(t, 1, PyInt_FromLong(2L));
PyTuple_SetItem(t, 2, PyString_FromString("three"));
return 0;
}
This code will be run on execution of the program. You can then use whatever other methods are provided to examine the contents of the tuple. If it isn't to be run separately as an executable program, then stick it in a differently-named method; I assume you have another way to invoke the function.
The PyObject *t; definition is valid outside the function as a global variable definition, as well as inside a function, declaring it as a local variable. The other four lines are function calls, which must be inside another function.
The above code on its own does not a program make. Are you trying to write a C extension to Python? If so, look at some more complete documentation here.
I have made some progress since I asked my question, and I thought I would just share it in case someone else is having similar problems.
These were the errors I got:
In function `main':
undefined reference to `_imp__PyTuple_New'
undefined reference to `_imp__PyInt_FromLong'
undefined reference to `_imp__PyTuple_SetItem'
undefined reference to `_imp__PyInt_FromLong'
undefined reference to `_imp__PyTuple_SetItem'
undefined reference to `_imp__PyString_FromString'
undefined reference to `_imp__PyTuple_SetItem'
The errors I got were the result of missing libraries from the mingW compiler. So only including the header file in the source ode is not enough, there is also a special file required (.lib, .o, .a, ..) that needs to be included for compilation. It is possible to use the -l[path] flag on the mingW command line, but I found that codeBlocks ( http://www.codeblocks.org/ ) is the most convenient to use here. After creating a project, and going to Project>Build options.., you can specify the location of the library file under the linker settings tab. When you are done, build the project, and it will hopefully work.
I hope anyone struggling with similar problems have help from this :)