I am using the Python 3 C API to interface with some Python code.
I have instantiated a Python object from C++ using PyObject_Call and I have been able to modify some of its attributes using PyObject_SetAttrString. One the attributes I would like to modify is a callback function, how can I instantiate a Python object from one of my C++ functions?
What you need to do is to create a Python object which is callable, and when it is called, it ends up calling a function implemented in C (or C++).
I would implement a new class, as specified in Python Type Objects docs. It may implement any kind of constructor etc. you may need, but most importantly for this purpose, it should implement the tp_call, which will make instances of this class callable (as if there was a __call__ method implemented in Python).
The implementation of tp_call is in C/C++ and you can create instances of this class, which are Python objects and can be passed to any Python code.
See also: Defining Extension Types: Tutorial
Related
This question already has answers here:
Python (and Python C API): __new__ versus __init__ [duplicate]
(6 answers)
Closed 2 years ago.
I recently started following Python after studying Java. I'm confused with the way of python interpreter's object construction.
Compared to Java when we construct an object when simply provide our arguments, for Python that is the case too.
But I can't think why the __init()__ method requires a self parameter when we define it in our class.
I read this question and I got that the methods require a self parameter because python calls a method in the format ClassA.methodA(ObjectA, arg1, arg2).
But I really don't get why the __init()__ method require this.
Is it because the way that Python generate an object differs from the way that Java generates an object.
I really appreciate if someone can explain it to me.
Why must ‘self’ be used explicitly in method definitions and calls?
The idea was borrowed from Modula-3. It turns out to be very useful, for a variety of reasons.
First, it’s more obvious that you are using a method or instance attribute instead of a local variable. Reading self.x or self.meth() makes it absolutely clear that an instance variable or method is used even if you don’t know the class definition by heart. In C++, you can sort of tell by the lack of a local variable declaration (assuming globals are rare or easily recognizable) – but in Python, there are no local variable declarations, so you’d have to look up the class definition to be sure. Some C++ and Java coding standards call for instance attributes to have an m_ prefix, so this explicitness is still useful in those languages, too.
Second, it means that no special syntax is necessary if you want to explicitly reference or call the method from a particular class. In C++, if you want to use a method from a base class which is overridden in a derived class, you have to use the :: operator – in Python you can write baseclass.methodname(self, ). This is particularly useful for __init__() methods, and in general in cases where a derived class method wants to extend the base class method of the same name and thus has to call the base class method somehow.
Finally, for instance variables it solves a syntactic problem with assignment: since local variables in Python are (by definition!) those variables to which a value is assigned in a function body (and that aren’t explicitly declared global), there has to be some way to tell the interpreter that an assignment was meant to assign to an instance variable instead of to a local variable, and it should preferably be syntactic (for efficiency reasons). C++ does this through declarations, but Python doesn’t have declarations and it would be a pity having to introduce them just for this purpose. Using the explicit self.var solves this nicely. Similarly, for using instance variables, having to write self.var means that references to unqualified names inside a method don’t have to search the instance’s directories. To put it another way, local variables and instance variables live in two different namespaces, and you need to tell Python which namespace to use.
reference: https://docs.python.org/3/faq/design.html#why-must-self-be-used-explicitly-in-method-definitions-and-calls
I have worked with Python for about 4 years and have recently started learning C++. In C++ you create a constructor method for each class I I was wondering if it is correct to think that this is equivalent to the __init__(self) function in Python? Are there any notable differences? Same question for a C++ destructor method vs. Python _exit__(self)
Yes, Python's __init__ is analogous to C++'s constructor. Both are typically where non-static data members are initialized. In both languages, these functions take the in-creation object as the first argument, explicit and by convention named self in Python and implicit and by language named this in C++. In both languages, these functions can return nothing. One notable difference between the languages is that in Python base-class __init__ must be called explicitly from an inherited class __init__ and in C++ it is implicit and automatic. C++ also has ways to declare data member initializers outside the body of the constructor, both by member initializer lists and non-static data member initializers. C++ will also generate a default constructor for you in some circumstances.
Python's __new__ is analogous to C++'s class-level operator new. Both are static class functions which must return a value for the creation to proceed. In C++, that something is a pointer to memory and in Python it is an uninitialized value of the class type being created.
Python's __del__ has no direct analogue in C++. It is an object finalizer, which exist also in other garbage collected languages like Java. It is not called at a lexically predetermined time, but the runtime calls it when it is time to deallocate the object.
__exit__ plays a role similar to C++'s destructor, in that it can provide for deterministic cleanup and a lexically predetermined point. In C++, this tends to be done through the C++ destructor of an RAII type. In Python, the same object can have __enter__ and __exit__ called multiple times. In C++, that would be accomplished with the constructor and destructor of a separate RAII resource holding type. For example, in Python given an instance lock of a mutual exclusion lock type, one can say with lock: to introduce a critical section. In C++, we create an instance of a different type taking the lock as a parameter std::lock_guard g{lock} to accomplish the same thing. The Python __enter__ and __exit__ calls map to the constructor and destructor of the C++ RAII type.
The best you can say is that __init__ and a C++ constructor are called at roughly the same point in the lifetime of a new object, and that __del__ and a C++ destructor are also called near the end of the lifetime of an object. The semantics, however, are markedly different, and the execution model of each language makes further comparison more difficult.
Suffice it to say that __init__ is used to initialize an object after it has been created. __del__ is like a destructor that may be called at some unspecified point in time after the last reference to an object goes away, and __exit__ is more like a callback invoked at the end of a with statement, whether or not the object's reference count reaches zero.
I was wondering if it is correct to think that this is equivalent to
the init(self) function in Python?
No. Just by looking at the structure of the statement you can understand. Indeed, self is a reference to the instance. Therefore, the instance must be constructed before __init__ is called.
See this for more information (__new__ is actually what you're looking for)
Same question for a C++ destructor method vs. Python _exit__(self)
No. __exit__ only exit the Runtime context related to the object. In this case, what you are really looking for is __del__.
See this, which clearly state:
Called when the instance is about to be destroyed. This is also called
a destructor.
I am creating a method object from string using "exec()". when I do dir(), I am able to see that the methods have been created in the current module. I need the method objects directly from exec() since there are lot of methods with same names which are currently getting replaced.
As I understand it, a client (the core program) needs to have a common type to allow a plugin, another object, etc. to be passed successfully to a client. I saw this answer on SO here,
What is dependency injection?
In Java, passing by constructor using an prescript Interface makes sense. From the SO question mentioned,
public SomeClass (MyClass myObject) {
this.myObject = myObject;
}
As I understand it, MyClass is a type defined by an Interface. myObject implements that, is required to in fact, thus allowing me to pass myObject to the constructor.
So how does Dependency Injection work in duck typing language? Python has no Interfaces. Is Python's DI implementation the same as Java or other statically typed languages, or a "workaround" type DI for scripting languages?
The need for an interface is just a detail of Java. It's the thing that lets you define a function that can accept an instance of any of several otherwise-unrelated types.
Since every Python function can accept an instance of any type, there is no need for anything comparable.
Of course, if you pass in an object that doesn't have the required capability then you'll get an exception at some point. Python has what is called "implicit interfaces" -- the interface required by the function is whatever operations it performs on the object in the expectation of them working.
I have a compatibility library that uses SWIG to access a C++ library. I would find it useful to be able to create a SWIG-wrapped Python object inside this layer (as opposed to accepting the C++ object as an argument or returning one). I.e. I want the PyObject* that points to the SWIG-wrapped C++ object.
I discovered that the SWIG_NewPointerObj function does exactly this. The SWIG-generated xx_wrap.cpp file uses this function, but it's also made available in the header emitted by swig -python -external-runtime swigpyrun.h
HOWEVER, I cannot find any reference to what the last argument to this function is. It appears that it specifies the ownership of the object, but there is no documentation that says what each of the options mean (or even what they all are).
It appears that the following are acceptable values:
0
SWIG_POINTER_OWN
SWIG_POINTER_NOSHADOW
SWIG_POINTER_NEW = OWN + NOSHADOW
SWIG_POINTER_DISOWN (I'm not sure if SWIG_NewPointerObj accepts this)
SWIG_POINTER_IMPLICIT_CONV (I'm not sure if SWIG_NewPointerObj accepts this)
I want to create an object that is used only in my wrapping layer. I want to create it out of my own pointer to the C++ object (so I can change the C++ object's value and have it be reflected in the Python object. I need it so it can be passed to a Python callback function. I want to keep this one instance throughout the life of the program so that I don't waste time creating/destroying identical objects for each callback. Which option is appropriate, and what do I Py_INCREF?
When you create new pointer objects with SWIG_NewPointerObj, you may pass the following flags:
SWIG_POINTER_OWN
SWIG_POINTER_NOSHADOW
If SWIG_POINTER_OWN is set, the destructor of the underlying C++ class will be called when the Python pointer is finalized. By default, the destructor will not be called. See Memory Management
For your use case, you don't need to set any flags at all.
From what I can see in the sources, if SWIG_POINTER_NOSHADOW is set, then a basic wrapped pointer is returned. You will not be able to access member variables in Python. All you'll have is an opaque pointer.
Reference: /usr/share/swig/2.0.7/python/pyrun.swg