Develop Reference

how is every object related to pyObject when c does not have Inheritance

I have been going through source code of python. It looks like every object is derived from PyObject. But, in C, there is no concept of object oriented programming. So, how exactly is this implemented without inheritance?

Your assertion that C has no concept of object-oriented programming is wrong. C doesn't explicitly have OOP, and it wasn't built with it in mind, but you can certainly do OOP things with C without too much effort. This comes from leveraging the fact that C doesn't actually really care what a struct's internal memory layout looks like. If you have two structs:
struct A {
int field1;
int field2;
double field3;
};
struct B {
A fieldA
int field4;
float field5;
};
then that essentially lets B behave as a subclass of A. After all, the first part of B's memory layout is exactly the same as A's memory layout. If you pass it around as a void pointer, then you can typecast away and C doesn't really care:
void doSomething(void *obj) {
int field2value = ((A*) obj).field2;
float field5value = ((B*) obj).field5;
printf("field2: %d\nfield5: %f", field2value, field5value);
}
You tell C what type you think the void pointer is supposed to be, and it makes that happen. And you get unexpected behavior if you guess wrong, or you get a segfault if the type you think it's supposed to be is larger than the type it actually is. You can use this to clumsily implement inheritance:
void constructA(void* obj) {
a = (A*) obj
a.field1 = 4;
a.field2 = 2;
a.field3 = 3.14;
}
void constructB(void *obj) {
constructA(obj);
b = (B*) obj;
b.field4 = 7;
b.field5 = 6.28;
}
int main() {
B *myObj = malloc(sizeof(B));
constructB(myObj);
free(myObj);
}
If one of A's variables is a function pointer, then that's fine. That function pointer gets passed down alongside the rest. You can call it from anywhere, after all. You can supplant its functionality in a "subclass", and then still call the original version later on, if you don't actually override its spot in memory - or if you do, you could have your replacement manually call the function it was pointing to.
A lot of advanced C code uses a similar pattern for replicating the idea of inheritance (or, alternatively, just uses C++, which optimizes this whole arrangement and abstracts it away so that programmers can work with a more intuitive syntax).
But even then, that's missing the point. One of my favorite things about python is, at the deepest level, its consistency - everything is an object, and all objects are basically hashmaps with names pointing to a references. Python's idea of duck typing works not because the underlying C code has any idea of inheritance, but because the code just looks for an attribute with the right name, and if it finds one, it uses it.
Subclasses in python, then, are the same as above - a new python object that initializes itself from the top down, adding more and more fields to the hashmap as it gets closer to the bottom.

What makes the Object Oriented programming paradigm is the relation between "classes" as templates for a data set and functions that will operate on this data set. And, the inheritance mechanism which is a relation from a class to ancestor classes.
These relations, however, do not depend on a particular language Syntax - just that they are present in anyway.
So, nothing stops one from doing "object orientation" in C, and in fact, organized libraries, even without an OO framework, end up with an organization related to OO.
It happens that the Python object system is entirely defined in pure C, with objects having a __class__ slot that points to its class with a C pointer - only when "viwed" from Python the full represenation of the class is resented. Classes in their turn having a __mro__ and __bases__ slots that point to the different arrangements of superclasses (the pointers this time are for containers that will be seen from Python as sequences).
So, when coding in C using the definitions and API of the Python runtime, one can use OOP just in the same way as coding in Python - and in fact use Python objects that are interoperable with the Python language. (The cython project will even transpile a superset of the Python language to C and provide transparent ways f writing native code with Python syntax)
There are other frameworks available to C that provide different OOP systems, that are equaly conformant, for example, glib - which defines "gobject" and is the base for all GTK+ and GNOME applications.

Create a PyObject with attached functions and return to Python

I wonder how I can create a PyObject in C++ and then return it to Python.
Sadly the documentation is not very explicit about it.
There is no PyObject_Create so I wonder whether allocating sizeof(PyObject) via PyObject_Malloc and initializing the struct is sufficient.
For now I only need an object with functions attached.

Do you really want a (1) PyObject, as in what Python calls object, or (2) an object of some subtype? That you "need an object with functions attached" seems to indicate you want either methods or attributes. That needs (2) in any case. I'm no expert on the C API, but generally you'd define your own PyTypeObject, then create an instance of that via PyObject_New (refcount and type field are initialized, other fields you might add are not).

python ctypes sending pointer to structure as parameter to native library

I am trying to write a wrapper to a native library in Linux. Problem is this:
definition in c:
int mymethod(mystruct* ptr)
in python:
_lib.mymethod.argtypes = (ctypes.POINTER(mystruct),)
_lib.mymethod.restype = ctypes.c_int
s = mystruct()
_lib.mymethod(ctypes.byref(s))
# raises: expected LP_mystruct instance instead of pointer to mystruct
_lib.mymethod(ctypes.pointer(s))
# raises expected LP_mystruct instance instead of LP_mystruct
errors. How to pass a structure as a pointer to a native method ?
Thanks.
Mete

The problem is that the higher level "POINTER" from ctypes is, in Python, a different object than "a generic pointer" (ctypes.CArgObject by ctypes.byref)which is returned or a single number representing a memory address (which is what is returned by ctype's adrresof) - you can either annotate your function to receive a `ctypes.c_voidp and call it with _lib.mymethod(ctypes.addressof(a)) instead -
Or if you want to work on the stronged-typed side to avoid errors that would crash Python (a type error raises a Python exception instead - a wrong parameter passed to a C unction would cause a segmentation fault on the Python interpreter itself), you have to create a variable to hold the new "type" which is a POINTER to your structure - and then create an instance of this type with the address of your structure:
mystruct_pointer = ctypes.POINTER(mystruct)
_lib.mymethod.argtypes = (mystruct_pointer,)
_lib.mymethod.restype = ctypes.c_int
s = mystruct()
_lib.mymethod(mystruct_pointer.from_address(ctypes.addressof(s)))

(I know that this is an old question, but I think the accepted answer is an unnecessary workaround, so I want to leave this here for posterity.)
Actually ctypes should explicitly support using byref() to pass a pointer like that:
ctypes exports the byref() function which is used to pass parameters by reference. The same effect can be achieved with the pointer() function, although pointer() does a lot more work since it constructs a real pointer object, so it is faster to use byref() if you don’t need the pointer object in Python itself.
The likely cause of this is that you have defined your struct in more than one place (e.g. in different modules) - if the argtypes assignment sees one definition and the function call sees the other, this confusing error arises. In other words, ctypes tries to match two mystruct types that are (probably) identical in contents, and have the exact same name, but they are not the same type. As long as the base struct type is a single type object, it doesn't matter if you construct a pointer to it using pointer(), byref() or POINTER()() - ctypes will detect that the underlying (pointed-to) type is the same.
To verify if this is the case, try assert(_lib.mymethod.argtypes[0]._type_ == type(s)) right before calling the external function.

Boost.Python function pointers as class constructor argument

I have a C++ class that requires a function pointer in it's constructor (float(*myfunction)(vector<float>*))
I've already exposed some function pointers to Python.
The ideal way to use this class is something like this:
import mymodule
mymodule.some_class(mymodule.some_function)
So I tell Boost about this class like so:
class_<SomeClass>("some_class", init<float(*)(vector<float>*)>);
But I get:
error: no matching function for call to 'register_shared_ptr1(Sample (*)(std::vector<double, std::allocator<double> >*))'
when I try to compile it.
So, does anyone have any ideas on how I can fix the error without losing the flexibility gained from function pointers (ie no falling back to strings that indicate which function to call)?
Also, the main point of writing this code in C++ is for speed. So it would be nice if I was still able to keep that benefit (the function pointer gets assigned to a member variable during initialization and will get called over a million times later on).

OK, so this is a fairly difficult question to answer in general. The root cause of your problem is that there really is no python type which is exactly equivalent to a C function pointer. Python functions are sort-of close, but their interface doesn't match for a few reasons.
Firstly, I want to mention the technique for wrapping a constructor from here:
http://wiki.python.org/moin/boost.python/HowTo#namedconstructors.2BAC8factories.28asPythoninitializers.29. This lets you write an __init__ function for your object that doesn't directly correspond to an actual C++ constructor. Note also, that you might have to specify boost::python::no_init in the boost::python::class_ construction, and then def a real __init__ function later, if your object isn't default-constructible.
Back to the question:
Is there only a small set of functions that you'll usually want to pass in? In that case, you could just declare a special enum (or specialized class), make an overload of your constructor that accepts the enum, and use that to look up the real function pointer. You can't directly call the functions yourself from python using this approach, but it's not that bad, and the performance will be the same as using real function pointers.
If you want to provide a general approach that will work for any python callable, things get more complex. You'll have to add a constructor to your C++ object that accepts a general functor, e.g. using boost::function or std::tr1::function. You could replace the existing constructor if you wanted, because function pointers will convert to this type correctly.
So, assuming you've added a boost::function constructor to SomeClass, you should add these functions to your python wrapping code:
struct WrapPythonCallable
{
typedef float * result_type;
explicit WrapPythonCallable(const boost::python::object & wrapped)
: wrapped_(wrapped)
{ }
float * operator()(vector<float>* arg) const
{
//Do whatever you need to do to convert into a
//boost::python::object here
boost::python::object arg_as_python_object = /* ... */;
//Call out to python with the object - note that wrapped_
//is callable using an operator() overload, and returns
//a boost::python::object.
//Also, the call can throw boost::python::error_already_set -
//you might want to handle that here.
boost::python::object result_object = wrapped_(arg_as_python_object);
//Do whatever you need to do to extract a float * from result_object,
//maybe using boost::python::extract
float * result = /* ... */;
return result;
}
boost::python::object wrapped_;
};
//This function is the "constructor wrapper" that you'll add to SomeClass.
//Change the return type to match the holder type for SomeClass, like if it's
//held using a shared_ptr.
std::auto_ptr<SomeClass> CreateSomeClassFromPython(
const boost::python::object & callable)
{
return std::auto_ptr<SomeClass>(
new SomeClass(WrapPythonCallable(callable)));
}
//Later, when telling Boost.Python about SomeClass:
class_<SomeClass>("some_class", no_init)
.def("__init__", make_constructor(&CreateSomeClassFromPython));
I've left out details on how to convert pointers to and from python - that's obviously something that you'll have to work out, because there are object lifetime issues there.
If you need to call the function pointers that you'll pass in to this function from Python, then you'll need to def these functions using Boost.Python at some point. This second approach will work fine with these def'd functions, but calling them will be slow, because objects will be unnecessarily converted to and from Python every time they're called.
To fix this, you can modify CreateSomeClassFromPython to recognize known or common function objects, and replace them with their real function pointers. You can compare python objects' identity in C++ using object1.ptr() == object2.ptr(), equivalent to id(object1) == id(object2) in python.
Finally, you can of course combine the general approach with the enum approach. Be aware when doing this, that boost::python's overloading rules are different from C++'s, and this can bite you when dealing with functions like CreateSomeClassFromPython. Boost.Python tests functions in the order that they are def'd to see if the runtime arguments can be converted to the C++ argument types. So, CreateSomeClassFromPython will prevent single-argument constructors def'd later than it from being used, because its argument matches any python object. Be sure to put it after other single-argument __init__ functions.
If you find yourself doing this sort of thing a lot, then you might want to look at the general boost::function wrapping technique (mentioned on the same page with the named constructor technique): http://wiki.python.org/moin/boost.python/HowTo?action=AttachFile&do=view&target=py_boost_function.hpp.

Python and C coupling

I tried loading C shared library .so in Python using ctypes.CDLL class (Linux). Here is the link to which tells what I did. As I see the documentation it says CDLL class assumes that function returns int types. I've a doubt here what if I need to return variable of type other than the int type from a function in C?.
And to what extent we can use C functions in Python i mean what are the limits/restrictions on using C shared libraries and functions
Thanks in Advance

By default, it assumes int, but you can set restype to any of the supported types to override that. E.g., from the docs:
strchr.restype = c_char_p
This means that strchr returns a pointer to a char, which corresponds to a Python string (or None, for a NULL pointer).