Where is object.__init__ located in the cpython repository?
I searched for __init__ in Objects/object.c, but it gives no results.
It appears that all the immutable data types use object.__init__, so I would like to know the implementation of it.
Objects/object.c is where (most of) the object protocol is implemented, not where object is implemented.
object is implemented along with type in Objects/typeobject.c, and its __init__ method is object_init in that file.
(Note that the very similar-sounding PyObject_Init function is actually completely unrelated to object.__init__. PyObject_Init is a generic helper function that performs type pointer and refcount initialization for a newly-allocated object struct.)
Related
I am investigating how isinstance() func works for CPython 2.7
Now I have an example with two Python files: lib1.py lib2.py
In lib1.py:
from a.b import lib2
def func_h():
ob = lib2.X()
print(isinstance(ob, lib2.X))
In lib2.py:
class X(object):
a = 1
The results print: True
Then I dig into CPython source code to
https://github.com/python/cpython/blob/ad65d09fd02512b2ccf500f6c11063f705c9cd28/Objects/abstract.c#L2945
where CPython did this check:
if (Py_TYPE(inst) == (PyTypeObject *)cls)
return 1;
where Py_TYPE() is macro
#define Py_TYPE(ob) (((PyObject*)(ob))->ob_type)
Does anyone know the clues how CPython inits or sets the ob_type during the program starting?
Normally, this happens in PyObject_Init (or PyObject_InitVar, which I won't mention again, but there are equivalent variations across the board), or the PyObject_INIT macro (which does the same thing in a faster way, but one that isn't guaranteed to be binary-compatible with other interpreter builds on the same platform). The docs for PyObject_Init say:
Initialize a newly-allocated object op with its type and initial reference. Returns the initialized object. If type indicates that the object participates in the cyclic garbage detector, it is added to the detector’s set of observed objects. Other fields of the object are not affected.
You can see the source in object.c:
PyObject *
PyObject_Init(PyObject *op, PyTypeObject *tp)
{
if (op == NULL)
return PyErr_NoMemory();
/* Any changes should be reflected in PyObject_INIT (objimpl.h) */
Py_TYPE(op) = tp;
_Py_NewReference(op);
return op;
}
For more details, see the comments in objimpl.h.
When an object is constructed from Python (or via the high-level C API):
The type's __new__ method or tp_new slot gets called.
This usually inherits from or supers to object_new, which calls PyType_GenericNew.
… or it delegates to some other constructor (which ultimately gets you back here)
… or returns some already existing object
… but if not, it must call tp_alloc manually.
PyType_GenericNew calls the type's tp_alloc slot (there's no Python special method for this).
This usually inherits from or supers to PyType_GenericAlloc, which calls the PyObject_INIT macro.
… but if not, tp_alloc must call one of the PyObject_Init-family functions or macros, or do the same thing itself.
Code in C extension modules, and internal interpreter code may:
Use the same high-level API
… or call PyObject_New, which allocates the object and calls PyObject_Init on it, and casts the result pointer
… or just call PyObject_Init directly (when it knows the type it's constructing doesn't customize tp_new, tp_alloc, or tp_init)
… or construct objects manually, but at some point it must call one of the PyObject_Init family directly or indirectly, or do the same thing itself, just as with custom tp_alloc
… or allocate constant objects statically rather than on the heap, like PyNone and many builtin and extension type objects, in which case the type (which also has to be a static constant, of course) is just specified in the struct initializer.
Piggybacking on this question, say i have a container for a weakreference:
import weakref
class Foo(object):
a = lambda *_: None
def __init__(self, a):
self.a = weakref.ref(a, self._remove)
def _remove(self, *args):
self.__del__(self)
class Bar(object):
pass
>>> bar = Bar()
>>> foo = Foo(bar)
>>> del bar
>>> foo
<__main__.Foo object at 0x...>
I thought of storing the Foo instance in a static WeakKeyDictionary container, with the a attribute as a key, and using weakref.proxy of the instance everywhere--but that seems...inefficient. What's the best way to make it so that the Foo instance deletes itself when its reference to a dies?
You can't. I just spent some time digging through the Python source and ctypes documentation to ironically show how one might really delete (aka Py_DECREF until deallocated) an object until I gave up. The point is, you don't really want to do this. Python manages its own memory for a reason. Sure, it gives you access to things like weak references, but in no case will Python break a strong reference.
What you are proposing is to have an object reach into the environments of every bit of code loaded into the Python interpreter to rip out any references to itself. weakref has to rip out references too, but it only has to remove the references from the weakref object; it doesn't have to touch the object holding a reference to the weakref. To remove a reference in the way you propose would be at least invasive and most likely impossible.
To see why it would be impossible, consider how one might write a Python module in C that defines a type. Each instance of the object is going to hold some PyObject pointers to things it cares about. Some of these might be exposed to Python through properties, while others might remain internal. Suppose one of these internal references referenced one of your Foo objects. For it to 'delete' itself, it would have to reach into our C type and NULL out the reference. But to Python code, the C struct defining the object is opaque. If you dug into it with ctypes, you could inspect the bytes, but who's to know whether some sequence of bytes is a pointer to your object or an int that just happens to have the same value as the address of your object? You can't, at least without knowing implementation details of that type. And you can't handle every case, because someone can add another case just by importing another module written in C. You can't anticipate everything.
So what can you do? If you're deadset on doing something like this, you can mimic weakref's interface. Basically, make a new class that holds a reference to your class; to avoid ambiguity, I'll call this a fakeref. When it's called, it returns the instance of your class. Your class holds weak references1 to all of its fakerefs. Whenever your Foo class wants to delete itself, loop over the fakerefs, Noneing out the references to the Foo. Voilà; your class can 'delete' itself as desired and all of the fakerefs will now return None. But just as with weakrefs, storing the result of a call will make it a strong reference again, and your class will not be able to delete itself in the manner you desire.
All this said, I don't think you've presented a good enough case for why this is necessary. All you've said is that "there's no reason for it to stay in memory". Well, there is: it needs to be there for the objects that reference it. If, at some point in time, it becomes useless, then your objects shouldn't be holding a reference to it. When the objects referencing it don't care about it any more, they should remove those references. Then Python will clean it up with no further intervention on your part.
1 If you don't want to rely on weak references, your fakeref can implement __del__ and remove itself from your Foo instance it holds a reference to (if not None).
I wonder how I can create a PyObject in C++ and then return it to Python.
Sadly the documentation is not very explicit about it.
There is no PyObject_Create so I wonder whether allocating sizeof(PyObject) via PyObject_Malloc and initializing the struct is sufficient.
For now I only need an object with functions attached.
Do you really want a (1) PyObject, as in what Python calls object, or (2) an object of some subtype? That you "need an object with functions attached" seems to indicate you want either methods or attributes. That needs (2) in any case. I'm no expert on the C API, but generally you'd define your own PyTypeObject, then create an instance of that via PyObject_New (refcount and type field are initialized, other fields you might add are not).
I am trying to write a wrapper to a native library in Linux. Problem is this:
definition in c:
int mymethod(mystruct* ptr)
in python:
_lib.mymethod.argtypes = (ctypes.POINTER(mystruct),)
_lib.mymethod.restype = ctypes.c_int
s = mystruct()
_lib.mymethod(ctypes.byref(s))
# raises: expected LP_mystruct instance instead of pointer to mystruct
_lib.mymethod(ctypes.pointer(s))
# raises expected LP_mystruct instance instead of LP_mystruct
errors. How to pass a structure as a pointer to a native method ?
Thanks.
Mete
The problem is that the higher level "POINTER" from ctypes is, in Python, a different object than "a generic pointer" (ctypes.CArgObject by ctypes.byref)which is returned or a single number representing a memory address (which is what is returned by ctype's adrresof) - you can either annotate your function to receive a `ctypes.c_voidp and call it with _lib.mymethod(ctypes.addressof(a)) instead -
Or if you want to work on the stronged-typed side to avoid errors that would crash Python (a type error raises a Python exception instead - a wrong parameter passed to a C unction would cause a segmentation fault on the Python interpreter itself), you have to create a variable to hold the new "type" which is a POINTER to your structure - and then create an instance of this type with the address of your structure:
mystruct_pointer = ctypes.POINTER(mystruct)
_lib.mymethod.argtypes = (mystruct_pointer,)
_lib.mymethod.restype = ctypes.c_int
s = mystruct()
_lib.mymethod(mystruct_pointer.from_address(ctypes.addressof(s)))
(I know that this is an old question, but I think the accepted answer is an unnecessary workaround, so I want to leave this here for posterity.)
Actually ctypes should explicitly support using byref() to pass a pointer like that:
ctypes exports the byref() function which is used to pass parameters by reference. The same effect can be achieved with the pointer() function, although pointer() does a lot more work since it constructs a real pointer object, so it is faster to use byref() if you don’t need the pointer object in Python itself.
The likely cause of this is that you have defined your struct in more than one place (e.g. in different modules) - if the argtypes assignment sees one definition and the function call sees the other, this confusing error arises. In other words, ctypes tries to match two mystruct types that are (probably) identical in contents, and have the exact same name, but they are not the same type. As long as the base struct type is a single type object, it doesn't matter if you construct a pointer to it using pointer(), byref() or POINTER()() - ctypes will detect that the underlying (pointed-to) type is the same.
To verify if this is the case, try assert(_lib.mymethod.argtypes[0]._type_ == type(s)) right before calling the external function.
I am writing a binding system that exposes classes and functions to python in a slightly unusual way.
Normally one would create a python type and provide a list of functions that represent the methods of that type, and then allow python to use its generic tp_getattro function to select the right one.
For reasons I wont go into here, I can't do it this way, and must provide my own tp_getattro function, that selects methods from elsewhere and returns my own 'bound method' wrapper. This works fine, but means that a types methods are not listed in its dictionary (so dir(MyType()) doesn't show anything interesting).
The problem is that I cannot seem to get __add__ methods working. see the following sample:
>>> from mymod import Vec3
>>> v=Vec3()
>>> v.__add__
<Bound Method of a mymod Class object at 0xb754e080>
>>> v.__add__(v)
<mymod.Vec3 object at 0xb751d710>
>>> v+v
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'mymod.Vec3' and 'mymod.Vec3'
As you can see, Vec3 has an __add__ method which can be called, but python's + refuses to use it.
How can I get python to use it? How does the + operator actually work in python, and what method does it use to see if you can add two arbitrary objects?
Thanks.
(P.S. I am aware of other systems such as Boost.Python and SWIG which do this automatically, and I have good reason for not using them, however wonderful they may be.)
Do you have an nb_add in your type's number methods structure (pointed by field tp_as_number of your type object)?