I first learned polymorphism in c++, in c++ we had types for every variable. So we used polymorphism to get a single pointer which can point to different type objects, and we could use them very nice.
But I don't get polymorphism and abstract classes in python. Here every variable can be everything. It could be an iterator, a list, a singe variable or a function. Every thing. So what makes a programmer to use an abstract class or use polymorphism here?
In c++ we used inheritance in many ways. But in python, it is just used to use another classes method or attribute. Am I right? what's the matter?
You don't understand what polymorphism is (OO polymorphic dispatch I mean). Polymorphism is the ability to have objects of different types understanding the same message, so you can use those objects the same way without worrying about their concrete type.
C++ actually uses the same concept (class) to denote two slightly different semantics: the abstract type (interface) which is the set of messages an object of this type understand) and the concrete type (implementation) which defines how this type reacts to those messages.
Java clearly distinguishes between abstract type (interface) and concrete type (class).
Python, being dynamically typed, relies mostly on "duck typing" (if it walks like a duck and quack like duck, then it's a duck - or at least it's "kind-of-a-duck" enough). You'll often find terms like "file-like" or "dict-like" in Python docs, meaning "anything that has the same interface as a file (or dict)", and quite a few "interfaces" are (or at least have long been) more or less implicit.
The issue with those implicit interfaces is that they are seldom fully documented, and one sometimes have to get to a function source code to find out exactly what the object passed needs to support. That's one of the reasons why the abc module was introduced in python 2 and improved in python 3: as a way to better document those implicit interfaces by creating an abstract base type that clearly defines the interface.
Another reason for abstract base classes (whether using the abc module or not) is to provide a common base implementation for a set of concrete subclasses. This is specially useful for frameworks, ie Django's models.Model (ORM) or forms.Form (user input collection and validation) classes - in both cases, just defining the database or form fields is enough to have something working.
Inheritance in C++ suffers from the same issue as classes: it serves both as defining the interface and implementation. This adds to the confusion... Java had the good idea (IMHO) to have separate abstract type from implementation, but failed to go all the way and restrict typing to interfaces - you can use either classes or interfaces for type declaration, so it still doesn't make the distinction clear.
In Python, since we don't have static typing, inheritance is mostly about implementation reuse indeed. The abc module allows you to register totally unrelated classes (no inheritance relationship) as also being subtypes of a defined abstract base case, but the point here is mostly to document that your class implements the same interface (and that it's not an accident...).
Related
Can someone explain the difference between the terms protocol and interface in the context of Python programming?
I'm seeing references to the term "protocol" in things like the buffer protocol and PEP 544, but want to make sure that I understand what this term means, and when and where, you'd use it differently from the general idea of an "interface".
Before I attempt to answer this question, recall the definition of interfaces:
An interface contains definitions for a group of related functionalities that a non-abstract class or a struct must implement.
Source: Microsoft Docs
Interfaces are used in statically typed languages to describe that two independent objects "implement the same behaviour". The interfaces are formally declared in code and enforced by the compiler (hence the must in the definition of interfaces above). They are one way of telling the type system that two objects can theoretically be substituted for each other (and are therefore related in a way). The other way is inheritance. If they cannot, the compiler throws an error.
Opposing to that, dynamically typed languages like Python do not require mechanisms like interfaces or inheritance to check if two objects are related. They use duck typing where the search for the appropriate function/method of an object is deduced at runtime. If found, it is executed - if not, an error is thrown. Therefore, interfaces are not required. Instead, there are so called "special methods" that can be implemented by classes to give instances certain "features", e.g. they can be hashed by implementing the __eq__ and __hash__ methods. These informal interfaces are NOT enforced by the compiler and only exist in the documentation.
To give an example for these informal interfaces, just imagine stumbling across some piece of code that implements a custom class that behaves like a list. Even though nowhere in code is this class related to any abstract sequence class, you know that it is used to produce sequence-like objects because it implements the __len__ and __getitem__ special methods.
I view protocols as much less strict version of interfaces in that they are not enforced and not all of them have to be implemented by a class. If you just want the class to be iterable, you can pick and implement the special methods that you have to implement and leave the rest of them untouched.
That being said, you can emulate interface-like behavior by using abstract base classes (ABCs).
Are there any conventions on how to implement services in Django? Coming from a Java background, we create services for business logic and we "inject" them wherever we need them.
Not sure if I'm using python/django the wrong way, but I need to connect to a 3rd party API, so I'm using an api_service.py file to do that. The question is, I want to define this service as a class, and in Java, I can inject this class wherever I need it and it acts more or less like a singleton. Is there something like this I can use with Django or should I build the service as a singleton and get the instance somewhere or even have just separate functions and no classes?
TL;DR It's hard to tell without more details but chances are you only need a mere module with a couple plain functions or at most just a couple simple classes.
Longest answer:
Python is not Java. You can of course (technically I mean) use Java-ish designs, but this is usually not the best thing to do.
Your description of the problem to solve is a bit too vague to come with a concrete answer, but we can at least give you a few hints and pointers (no pun intended):
1/ Everything is an object
In python, everything (well, everything you can find on the RHS of an assignment that is) is an object, including modules, classes, functions and methods.
One of the consequences is that you don't need any complex framework for dependency injection - you just pass the desired object (module, class, function, method, whatever) as argument and you're done.
Another consequence is that you don't necessarily need classes for everything - a plain function or module can be just enough.
A typical use case is the strategy pattern, which, in Python, is most often implemented using a mere callback function (or any other callable FWIW).
2/ a python module is a singleton.
As stated above, at runtime a python module is an object (of type module) whose attributes are the names defined at the module's top-level.
Except for some (pathological) corner cases, a python module is only imported once for a given process and is garanteed to be unique. Combined with the fact that python's "global" scope is really only "module-level" global, this make modules proper singletons, so this design pattern is actually already builtin.
3/ a python class is (almost) a singleton
Python classes are objects too (instance of type type, directly or indirectly), and python has classmethods (methods that act on the class itself instead of acting on the current instance) and class-level attributes (attributes that belong to the class object itself, not to it's instances), so if you write a class that only has classmethods and class attributes, you technically have a singleton - and you can use this class either directly or thru instances without any difference since classmethods can be called on instances too.
The main difference here wrt/ "modules as singletons" is that with classes you can use inheritance...
4/ python has callables
Python has the concept of "callable" objects. A "callable" is an object whose class implements the __call__() operator), and each such object can be called as if it was a function.
This means that you can not only use functions as objects but also use objects as functions - IOW, the "functor" pattern is builtin. This makes it very easy to "capture" some context in one part of the code and use this context for computations in another part.
5/ a python class is a factory
Python has no new keyword. Pythonc classes are callables, and instanciation is done by just calling the class.
This means that you can actually use a class or function the same way to get an instance, so the "factory" pattern is also builtin.
6/ python has computed attributes
and beside the most obvious application (replacing a public attribute by a pair of getter/setter without breaking client code), this - combined with other features like callables etc - can prove to be very powerful. As a matter of fact, that's how functions defined in a class become methods
7/ Python is dynamic
Python's objects are (usually) dict-based (there are exceptions but those are few and mostly low-level C-coded classes), which means you can dynamically add / replace (and even remove) attributes and methods (since methods are attributes) on a per-instance or per-class basis.
While this is not a feature you want to use without reasons, it's still a very powerful one as it allows to dynamically customize an object (remember that classes are objects too), allowing for more complex objects and classes creation schemes than what you can do in a static language.
But Python's dynamic nature goes even further - you can use class decorators and/or metaclasses to taylor the creation of a class object (you may want to have a look at Django models source code for a concrete example), or even just dynamically create a new class using it's metaclass and a dict of functions and other class-level attributes.
Here again, this can really make seemingly complex issues a breeze to solve (and avoid a lot of boilerplate code).
Actually, Python exposes and lets you hook into most of it's inners (object model, attribute resolution rules, import mechanism etc), so once you understand the whole design and how everything fits together you really have the hand on most aspects of your code at runtime.
Python is not Java
Now I understand that all of this looks a bit like a vendor's catalog, but the point is highlight how Python differs from Java and why canonical Java solutions - or (at least) canonical Java implementations of those solutions - usually don't port well to the Python world. It's not that they don't work at all, just that Python usually has more straightforward (and much simpler IMHO) ways to implement common (and less common) design patterns.
wrt/ your concrete use case, you will have to post a much more detailed description, but "connecting to a 3rd part API" (I assume a REST api ?) from a Django project is so trivial that it really doesn't warrant much design considerations by itself.
In Python you can write the same as Java program structure. You don't need to be so strongly typed but you can. I'm using types when creating common classes and libraries that are used across multiple scripts.
Here you can read about Python typing
You can do the same here in Python. Define your class in package (folder) called services
Then if you want singleton you can do like that:
class Service(object):
instance = None
def __new__(cls):
if cls.instance is not None:
return cls.instance
else:
inst = cls.instance = super(Service, cls).__new__()
return inst
And now you import it wherever you want in the rest of the code
from services import Service
Service().do_action()
Adding to the answer given by bruno desthuilliers and TreantBG.
There are certain questions that you can ask about the requirements.
For example one question could be, does the api being called change with different type of objects ?
If the api doesn't change, you will probably be okay with keeping it as a method in some file or class.
If it does change, such that you are calling API 1 for some scenario, API 2 for some and so on and so forth, you will likely be better off with moving/abstracting this logic out to some class (from a better code organisation point of view).
PS: Python allows you to be as flexible as you want when it comes to code organisation. It's really upto you to decide on how you want to organise the code.
This is a programming language concept question, e.g. similar to the level of Programming Language Pragmatics, by Scott.
In Python, the classes of some kinds of objects are defined in terms of having some methods with special names, for example,
a descriptors' class is defined as a class which has a method named __get__, __set__, or __delete__().
an iterators' class is defined as a class which has a method named __next__.
Questions:
What is the language feature in Python called in programming language design? Is it duck typing?
How does the language feature work underneath?
In C++, C#, and Java, is it correct that a descriptor's class and an
iterator's class would have been defined as subclasses of some
particular classes? (similarly to C# interface IDisposable)
In Python,
Can descriptors' classes be defined as subclasses of some particular class?
Can iterators' classes be defined as subclasses of some particular class?
What is the language feature in Python called in programming language design? Is it duck typing?
"Any object with a member with a specific name (or signature), can work here" is duck typing. I don't think there is a more specific term for "any object with a member with a specific name (or signature), can work for this language feature", if that's what you were asking.
How does the language feature work underneath?
I don't understand the question. If a language feature means that it calls a method with a specific name, it calls a method with that name. That's it.
In C++, C#, and Java, is it correct that a descriptor's class and an iterator's class would have been defined as subclasses of some particular classes?
I'm not aware of anything similar to descriptor in any of these languages and I don't think it makes sense to speculate on how it would look if it did exist.
As for iterators, each of these languages has a foreach loop, so you can look at that:
In C++, the range-based for loop works on any type that has instance members begin and end or for which the begin and end functions exist. The returned type has to support the ++, != and * operators.
In C#, the foreach loop works on any type that has instance method GetEnumerator(), which returns a type with a MoveNext() method and a Current property. There is also the IEnumerable<T> interface, which describes the same shape. Enumerable types commonly implement this interface, but they're not required to do so.
In Java, the enhanced for loop works on any type that implements Iterable.
So, there are no subclasses anywhere (C# and Java differentiate between implementing an interface and deriving from a base class). Java requires you to implement an interface. C# uses duck typing, but also optionally allows you to implement an interface. C++ uses duck typing, there is no interface or base class at all.
Note that, depending on the language, the decision whether to use duck typing for a certain language feature might be complicated. As an extreme example, one feature of C# (collection initializers) requires implementing of a specific interface (IEnumerable) and also the presence of a method with a specific name (Add). So this feature is partially duck typed.
The term you are looking for is The Python Data Model (there's more to this than just dunder-methods and -attributes, but they are an integral part of it).
This is an aspect of duck typing.
Python, as a dynamically-typed language, cares less about the actual types of objects than about their behaviour. As the saying goes, if it quacks like a duck, then it's probably a duck; in the case of your descriptor, Python just wants to know it defines the special methods, and if it does then it accepts that it is a descriptor.
I'm learning about metaclasses in Python. I think it is a very powerful technique, and I'm looking for good uses for them. I'd like some feedback of good useful real-world examples of using metaclasses. I'm not looking for example code on how to write a metaclass (there are plenty examples of useless metaclasses out there), but real examples where you have applied the technique and it was really the appropriate solution. The rule is: no theoretical possibilities, but metaclasses at work in a real application.
I'll start with the one example I know:
Django models, for declarative programming, where the base class Model uses a metaclass to fill the model objects of useful ORM functionality from the attribute definitions.
Looking forward to your contributions.
In Python 2.6 and 3.1, the Python standard library provides an abc.ABCMeta, a meta-class for Abstract Base Classes ("ABCs"). Classes that use the meta-class can use #abstractmethod and #abstractproperty to define abstract methods and properties. The meta-class will ensure that derived classes override the abstract methods and properties.
Also, classes that implement the ABC without actually inheriting from it can register as implementing the interface, so that issubclass and isinstance will work.
For example, the collections module defines the Sequence ABC. It also calls Sequence.register(tuple) to register the built-in tuple type as a Sequence, even though tuple does not actually inherit from Sequence.
The Python implementation of Protocol Buffers uses metaclasses to generate the Python bindings that represent your data format. From the tutorial:
The important line in each class is __metaclass__ = reflection.GeneratedProtocolMessageType. While the details of how Python metaclasses work is beyond the scope of this tutorial, you can think of them as like a template for creating classes. At load time, the GeneratedProtocolMessageType metaclass uses the specified descriptors to create all the Python methods you need to work with each message type and adds them to the relevant classes. You can then use the fully-populated classes in your code.
FormEncode validators and Turbogears / Tosca widgets.
You might also be interested in class decorators: they can be written with the latest releases, and cover many use cases that were previously handled with metaclasses.
SQLalchemy also uses them for declarative database models.
Sorry my answer isn't very different from your example, but if you're looking for example code, I found declarative to be pretty readable.
The only time I used a metaclass so far was to write a deprecation warning mechanism. It was something along the following lines - syntax may be very approximative, but code will illustrate my point more easily than a complicated sentence :
class New(object):
pass
class Old(object):
def __new__(self):
deprecation_warning("Old class is no more supported, use New class instead")
return New()
I was wondering if I have a couple of models which both include fields like "meta_keywords" or "slug" which have to do with the web page the model instance will be displayed on, whether it would be advisable to break those page metadata elements out into their own class, say PageMeta, and have my other models subclass those via multiple inheritance?
General advice for a lightly-specified question:
Nontrivial multiple inheritance in Python requires Advanced Techniques to deal with the metaclass/metatype conflict. Look over this recipe from the ActiveState archives and see if it looks like the kind of stuff you like:
Extract from linked recipe:
The simplest case where a metatype
conflict happens is the following.
Consider a class A with metaclass M_A
and a class B with an independent
metaclass M_B; suppose we derive C
from A and B. The question is: what is
the metaclass of C ? Is it M_A or M_B
?
The correct answer (see the book
"Putting metaclasses to work" for a
thoughtful discussion) is M_C, where
M_C is a metaclass that inherits from
M_A and M_B.
However, Python is not that magic, and
it does not automatically create M_C.
Instead, it raises a TypeError,
warning the programmer of the possible
confusion.
Consequently, I recommend limiting your use of multiple inheritance in Python to the following cases:
You must, because your problem domain requires you to combine two separately-maintained single-inheritance libraries.
You have achieved such fluency with metatype and metaclass that you can write recipe 204197 or its equivalent as easily and confidently as you can write a print statement.
Edit:
Here's Guido van Rossum in An Introduction to Python:
It is clear that indiscriminate use of
multiple inheritance is a maintenance
nightmare, given the reliance in
Python on conventions to avoid
accidental name conflicts.
Here he is again in PEP 253, which describes the ideas which were incorporated into Python, but not the implementation:
Metatypes determine various policies
for types, such as what
happens when a type is called, how dynamic types are (whether a
type's dict can be modified after it is created), what the
method resolution order is, how instance attributes are looked
up, and so on.
I'll argue that left-to-right depth-first is not the best
solution when you want to get the most use from multiple
inheritance.
I'll argue that with multiple inheritance, the metatype of the
subtype must be a descendant of the metatypes of all base types.
This does not mean you shouldn't use multiple inheritance; I'm just warning you so you won't be suprised one day to find yourself slapping your forehead and exclaiming "D'oh!
The metatype of one of my subtypes isn't a descendant of the metatypes of all its base types! Don't you hate when that happens?"