I am building a C extension module for Python 3.x. I would like to access the functionality of the hex builtin in the Python layer. That is, I would like to convert (form my C code) a PyObject* which is of type 'PyLong_Type' (that is, a plain python int) into a PyObject* which is of type PyUnicode_Type and represents the hexadecimal coding of the int I started with.
This seems like it should be easy, but none of the functions in the integer section of the API guide seem to do it; neither do any of the functions in the string section. Note in particular that PyUnicode_FromFormat doesn't do what I need:
you can't use it with the %S or %R format specifier (you'll get a decimal representation)
you can't use it with the %x format specifier (you'd have to convert the PyObject* to a C int first, which isn't a safe thing to do, since the Python integer might be too large to fit.
This is the implementation of hex:
static PyObject *
builtin_hex(PyObject *self, PyObject *v)
{
return PyNumber_ToBase(v, 16);
}
It's static, but the implementation suggests an easy way to get the same functionality:
return PyNumber_ToBase(your_number, 16);
PyNumber_ToBase also exists in 2.6 and 2.7, though hex doesn't use it in the 2.x line.
Answering my own question, one way to do it is something like:
PyObject *fmt_string = PyUnicode_FromString("{0:x}");
if (!fmt_string)
return NULL;
PyObject *hex_rep_string = PyObject_CallMethodObjArgs(fmt_string, "format", int_obj);
Py_DECREF(fmt_string);
if (!hex_rep_string)
{
return NULL;
}
but it seems there must be a better/more canonical way...
Related
I see a problem from one Stackoverflow question. It is described below:
I have a C++ function which returns a raw float pointer, and another C++ function which accepts a raw float pointer as an argument. Something like:
float* ptr = something;
float* get_ptr(void) { return ptr; }
void use_ptr(float* ptr) { do_work(ptr); }
I want to be able to pass around pointers using Python. Something like this:
import my_native_functions as native
ptr = native.get_ptr()
native.use_ptr(ptr)
I am using pybind11 to create my native python module but I don't know how to create the bindings for the get_ptr() function. If I just do the following:
PYBIND11_MODULE(my_native_functions, m)
{
m.def("get_ptr", &get_ptr);
m.def("use_ptr", &use_ptr);
}
the get_ptr() function returns a Python Float object. I guess this makes sense because there are no pointer types in python. However, because this is now a simple Float, when I call the use_ptr() function and iterate over the pointer in C/C++, only the first element of the array is correct. The rest are garbage. To fix this, in C++, I have to cast my pointer to/from std::size_t. By doing this, everything works just fine.
However, I would like to ask: Is there a "right way" of achieving the above without the casting to/from std::size_t with pybind11?
The above is the whole problem description that comes from AstrOne's question:Returning and passing around raw POD pointers (arrays) with Python, C++, and pybind11.
My question is "To fix this, in C++, I have to cast my pointer to/from std::size_t", how should this cast be done? Can here give me a example or more detailed explanation? I don't quite understand.
Replace the float* type with std::uintptr_t. See below
#include <cstdint>
float* ptr = something;
std::uintptr_t get_ptr() { return reinterpret_cast<std::uintptr_t>(ptr); }
void use_ptr(std::uintptr_t ptr) { do_work(reinterpret_cast<float*>(ptr)); }
No pointer types will be exposed to pybind11 now. What is doing here is just cast a pointer type to a integral type, which has no difference in binary representation.
Aside: std::size_t would be Ok in most platforms. As per cppref:
On many platforms (an exception is systems with segmented addressing) std::size_t can safely store the value of any non-member pointer, in which case it is synonymous with std::uintptr_t.
This question already has answers here:
Why do some built-in Python functions only have pass?
(2 answers)
Closed 4 years ago.
in many codes, i see classes with functions in them that they just used pass phrase with some comment upon them.
like this native builtin function from python:
def copyright(*args, **kwargs): # real signature unknown
"""
interactive prompt objects for printing the license text, a list of
contributors and the copyright notice.
"""
pass
i know pass does nothing, and its kind of apathetic and null phrase, but why programmers use such functions ?
and also there are some functions with return "" like:
def bin(number): # real signature unknown; restored from __doc__
"""
bin(number) -> string
Return the binary representation of an integer.
>>> bin(2796202)
'0b1010101010101010101010'
"""
return ""
why programmers use such things ?
Your IDE is lying to you. Those functions don't actually look like that; your IDE has made up a bunch of fake source code with almost no resemblance to the real thing. That's why it says things like # real signature unknown. I don't know why they thought this was a good idea.
The real code looks completely different. For example, here's the real bin (Python 2.7 version):
static PyObject *
builtin_bin(PyObject *self, PyObject *v)
{
return PyNumber_ToBase(v, 2);
}
PyDoc_STRVAR(bin_doc,
"bin(number) -> string\n\
\n\
Return the binary representation of an integer or long integer.");
It's written in C, and it's implemented as a simple wrapper around the C function PyNumber_ToBase:
PyObject *
PyNumber_ToBase(PyObject *n, int base)
{
PyObject *res = NULL;
PyObject *index = PyNumber_Index(n);
if (!index)
return NULL;
if (PyLong_Check(index))
res = _PyLong_Format(index, base, 0, 1);
else if (PyInt_Check(index))
res = _PyInt_Format((PyIntObject*)index, base, 1);
else
/* It should not be possible to get here, as
PyNumber_Index already has a check for the same
condition */
PyErr_SetString(PyExc_ValueError, "PyNumber_ToBase: index not "
"int or long");
Py_DECREF(index);
return res;
}
its a TBD (to be done) thing
you know you will need this function, you know what to give it and you know what it returns, but you arent going to write it right now, so you make a "prototype"
sometimes packages will ship with these functions because they expect you to inherit them and overwrite them
I would like to pass as argument of a function in my C module an array of uint8_t's.
I couldn't find a method to directly parse this array, so I'm parsing it to a PyObject_t and then iterating as a PyTuple_t object. This way, I need to cast each element PyObject_t of this tuple to uint8_t.
How can I do that, once that there is no PyInt_FromUINT8_t function or anything like it?
You can usually just get away with B using unsigned char. According to Parsing Arguments you should just be able to do:
uint8_t b;
if (!PyArg_ParseTuple("b", &b)) {
return NULL;
}
If not directly using arguments (e.g. you are dealing with a PyObject, simply use one of the PyInt_*, PyLong_* or PyNumber_* functions (https://docs.python.org/3/c-api/number.html?highlight=pynumber#c.PyNumber_AsSsize_t).
Converting from a uin8_t to a PyObject is simple as well, you can use PyInt_FromLong or PyLong_FromLong
I use SWIG for generating wrappers. Therefore I need a function which looks like
%inline %{
// Serializes into a string
void* SerCmd(Class *v, int *length, char *str)
{
QByteArray ba;
QDataStream out(&ba, QIODevice::WriteOnly);
out << *v;
*length = ba.size();
str = new char[ba.size()];
memcpy(str, ba.constData(), ba.size());
return str;
}
%}
This function is called from python then but who is deleting the memory I allocate with new? Is python doing that for me or how can that be achieved?
Thanks!
If this doesn't answer your question, I will remove it. But according to the SWIG information found here:
http://www.swig.org/Doc1.3/Library.html#Library_stl_cpp_library
a std::string can be used instead of manually allocating memory. Given this information, this more than likely can be used.
%inline %{
// Serializes into a string
void SerCmd(Class *v, int *length, std::string& str)
{
QByteArray ba;
QDataStream out(&ba, QIODevice::WriteOnly);
out << *v;
*length = ba.size();
str.clear();
str.append(ba.constData(), ba.size());
}
%}
Since you noted that the std::string can contain NULLs, then the proper way to handle it is to use the string::append() function.
http://en.cppreference.com/w/cpp/string/basic_string/append
Please note item 4) in the link above (null characters are perfectly ok). Note that std::string does not determine its size by a null character, unlike C-strings.
Now, to get to this data, use the string::data() function, along with the string::size() function to tell you how much data you have in the string.
I don't know SWIG either, but since you asked, "Is python doing that for me or how can that be achieved?"
Python garbage collection deletes stuff when it is no longer in scope i.e. no longer has anything pointing to it. However, it cannot delete stuff it is not aware of. These docs may help.
Here is the doc on how memory management works:
https://docs.python.org/2/c-api/memory.html
And here are the docs to the gc module to help give you a more control of the process.
https://docs.python.org/2/library/gc.html
I was wondering what is the correct way to wrap an array of strings in C to a Python list using SWIG.
The array is inside a struct :
typedef struct {
char** my_array;
char* some_string;
}Foo;
SWIG automatically wraps some_string to a python string.
What should I put in the SWIG interface file so that I can access my_array in Python as a regular Python string list ['string1', 'string2' ] ?
I have used typemap as sugested :
%typemap(python,out) char** {
int len,i;
len = 0;
while ($1[len]) len++;
$result = PyList_New(len);
for (i = 0; i < len; i++) {
PyList_SetItem($result,i,PyString_FromString($1[i]));
}
}
But that still didn't work. In Python, the my_array variable appears as SwigPyObject: _20afba0100000000_p_p_char.
I wonder if that is because the char** is inside a struct? Maybe I need to inform SWIG that?
Any ideas?
I don't think there is a option to handle this conversion automatically in SWIG. You need use Typemap feature of SWIG and write type converter manually. Here you can find a conversion from Python list to char** http://www.swig.org/Doc1.3/Python.html#Python_nn59 so half of job is done. What you need to do right now is to check rest of documentation of Typemap and write converter from char** to Python list.
I am not an expert on this but I think:
%typemap(python,out) char** {
applies to a function that returns char **. Your char ** is inside a structure.. have a look at the code generated by swig to confirm the map got applied or not.
You might have to use something like:
%typemap(python,out) struct Foo {
To have a map that works on a structure Foo that gets returned.
Background: I used the same typemap definition as you used, but then for a char ** successfully.
I am sorry for being slightly off-topic, but if it is an option for you I would strongly recommend using ctypes instead of swig. Here is a related question I asked previously in ctypes context: Passing a list of strings to from python/ctypes to C function expecting char **