Develop Reference

Python terminology: interface vs. protocol

Can someone explain the difference between the terms protocol and interface in the context of Python programming?
I'm seeing references to the term "protocol" in things like the buffer protocol and PEP 544, but want to make sure that I understand what this term means, and when and where, you'd use it differently from the general idea of an "interface".

Before I attempt to answer this question, recall the definition of interfaces:
An interface contains definitions for a group of related functionalities that a non-abstract class or a struct must implement.
Source: Microsoft Docs
Interfaces are used in statically typed languages to describe that two independent objects "implement the same behaviour". The interfaces are formally declared in code and enforced by the compiler (hence the must in the definition of interfaces above). They are one way of telling the type system that two objects can theoretically be substituted for each other (and are therefore related in a way). The other way is inheritance. If they cannot, the compiler throws an error.
Opposing to that, dynamically typed languages like Python do not require mechanisms like interfaces or inheritance to check if two objects are related. They use duck typing where the search for the appropriate function/method of an object is deduced at runtime. If found, it is executed - if not, an error is thrown. Therefore, interfaces are not required. Instead, there are so called "special methods" that can be implemented by classes to give instances certain "features", e.g. they can be hashed by implementing the __eq__ and __hash__ methods. These informal interfaces are NOT enforced by the compiler and only exist in the documentation.
To give an example for these informal interfaces, just imagine stumbling across some piece of code that implements a custom class that behaves like a list. Even though nowhere in code is this class related to any abstract sequence class, you know that it is used to produce sequence-like objects because it implements the __len__ and __getitem__ special methods.
I view protocols as much less strict version of interfaces in that they are not enforced and not all of them have to be implemented by a class. If you just want the class to be iterable, you can pick and implement the special methods that you have to implement and leave the rest of them untouched.
That being said, you can emulate interface-like behavior by using abstract base classes (ABCs).

why we use polymorphism and abstract classes in python

I first learned polymorphism in c++, in c++ we had types for every variable. So we used polymorphism to get a single pointer which can point to different type objects, and we could use them very nice.
But I don't get polymorphism and abstract classes in python. Here every variable can be everything. It could be an iterator, a list, a singe variable or a function. Every thing. So what makes a programmer to use an abstract class or use polymorphism here?
In c++ we used inheritance in many ways. But in python, it is just used to use another classes method or attribute. Am I right? what's the matter?

You don't understand what polymorphism is (OO polymorphic dispatch I mean). Polymorphism is the ability to have objects of different types understanding the same message, so you can use those objects the same way without worrying about their concrete type.
C++ actually uses the same concept (class) to denote two slightly different semantics: the abstract type (interface) which is the set of messages an object of this type understand) and the concrete type (implementation) which defines how this type reacts to those messages.
Java clearly distinguishes between abstract type (interface) and concrete type (class).
Python, being dynamically typed, relies mostly on "duck typing" (if it walks like a duck and quack like duck, then it's a duck - or at least it's "kind-of-a-duck" enough). You'll often find terms like "file-like" or "dict-like" in Python docs, meaning "anything that has the same interface as a file (or dict)", and quite a few "interfaces" are (or at least have long been) more or less implicit.
The issue with those implicit interfaces is that they are seldom fully documented, and one sometimes have to get to a function source code to find out exactly what the object passed needs to support. That's one of the reasons why the abc module was introduced in python 2 and improved in python 3: as a way to better document those implicit interfaces by creating an abstract base type that clearly defines the interface.
Another reason for abstract base classes (whether using the abc module or not) is to provide a common base implementation for a set of concrete subclasses. This is specially useful for frameworks, ie Django's models.Model (ORM) or forms.Form (user input collection and validation) classes - in both cases, just defining the database or form fields is enough to have something working.
Inheritance in C++ suffers from the same issue as classes: it serves both as defining the interface and implementation. This adds to the confusion... Java had the good idea (IMHO) to have separate abstract type from implementation, but failed to go all the way and restrict typing to interfaces - you can use either classes or interfaces for type declaration, so it still doesn't make the distinction clear.
In Python, since we don't have static typing, inheritance is mostly about implementation reuse indeed. The abc module allows you to register totally unrelated classes (no inheritance relationship) as also being subtypes of a defined abstract base case, but the point here is mostly to document that your class implements the same interface (and that it's not an accident...).

Python/Django and services as classes

Are there any conventions on how to implement services in Django? Coming from a Java background, we create services for business logic and we "inject" them wherever we need them.
Not sure if I'm using python/django the wrong way, but I need to connect to a 3rd party API, so I'm using an api_service.py file to do that. The question is, I want to define this service as a class, and in Java, I can inject this class wherever I need it and it acts more or less like a singleton. Is there something like this I can use with Django or should I build the service as a singleton and get the instance somewhere or even have just separate functions and no classes?

TL;DR It's hard to tell without more details but chances are you only need a mere module with a couple plain functions or at most just a couple simple classes.
Longest answer:
Python is not Java. You can of course (technically I mean) use Java-ish designs, but this is usually not the best thing to do.
Your description of the problem to solve is a bit too vague to come with a concrete answer, but we can at least give you a few hints and pointers (no pun intended):
1/ Everything is an object
In python, everything (well, everything you can find on the RHS of an assignment that is) is an object, including modules, classes, functions and methods.
One of the consequences is that you don't need any complex framework for dependency injection - you just pass the desired object (module, class, function, method, whatever) as argument and you're done.
Another consequence is that you don't necessarily need classes for everything - a plain function or module can be just enough.
A typical use case is the strategy pattern, which, in Python, is most often implemented using a mere callback function (or any other callable FWIW).
2/ a python module is a singleton.
As stated above, at runtime a python module is an object (of type module) whose attributes are the names defined at the module's top-level.
Except for some (pathological) corner cases, a python module is only imported once for a given process and is garanteed to be unique. Combined with the fact that python's "global" scope is really only "module-level" global, this make modules proper singletons, so this design pattern is actually already builtin.
3/ a python class is (almost) a singleton
Python classes are objects too (instance of type type, directly or indirectly), and python has classmethods (methods that act on the class itself instead of acting on the current instance) and class-level attributes (attributes that belong to the class object itself, not to it's instances), so if you write a class that only has classmethods and class attributes, you technically have a singleton - and you can use this class either directly or thru instances without any difference since classmethods can be called on instances too.
The main difference here wrt/ "modules as singletons" is that with classes you can use inheritance...
4/ python has callables
Python has the concept of "callable" objects. A "callable" is an object whose class implements the __call__() operator), and each such object can be called as if it was a function.
This means that you can not only use functions as objects but also use objects as functions - IOW, the "functor" pattern is builtin. This makes it very easy to "capture" some context in one part of the code and use this context for computations in another part.
5/ a python class is a factory
Python has no new keyword. Pythonc classes are callables, and instanciation is done by just calling the class.
This means that you can actually use a class or function the same way to get an instance, so the "factory" pattern is also builtin.
6/ python has computed attributes
and beside the most obvious application (replacing a public attribute by a pair of getter/setter without breaking client code), this - combined with other features like callables etc - can prove to be very powerful. As a matter of fact, that's how functions defined in a class become methods
7/ Python is dynamic
Python's objects are (usually) dict-based (there are exceptions but those are few and mostly low-level C-coded classes), which means you can dynamically add / replace (and even remove) attributes and methods (since methods are attributes) on a per-instance or per-class basis.
While this is not a feature you want to use without reasons, it's still a very powerful one as it allows to dynamically customize an object (remember that classes are objects too), allowing for more complex objects and classes creation schemes than what you can do in a static language.
But Python's dynamic nature goes even further - you can use class decorators and/or metaclasses to taylor the creation of a class object (you may want to have a look at Django models source code for a concrete example), or even just dynamically create a new class using it's metaclass and a dict of functions and other class-level attributes.
Here again, this can really make seemingly complex issues a breeze to solve (and avoid a lot of boilerplate code).
Actually, Python exposes and lets you hook into most of it's inners (object model, attribute resolution rules, import mechanism etc), so once you understand the whole design and how everything fits together you really have the hand on most aspects of your code at runtime.
Python is not Java
Now I understand that all of this looks a bit like a vendor's catalog, but the point is highlight how Python differs from Java and why canonical Java solutions - or (at least) canonical Java implementations of those solutions - usually don't port well to the Python world. It's not that they don't work at all, just that Python usually has more straightforward (and much simpler IMHO) ways to implement common (and less common) design patterns.
wrt/ your concrete use case, you will have to post a much more detailed description, but "connecting to a 3rd part API" (I assume a REST api ?) from a Django project is so trivial that it really doesn't warrant much design considerations by itself.

In Python you can write the same as Java program structure. You don't need to be so strongly typed but you can. I'm using types when creating common classes and libraries that are used across multiple scripts.
Here you can read about Python typing
You can do the same here in Python. Define your class in package (folder) called services
Then if you want singleton you can do like that:
class Service(object):
instance = None
def __new__(cls):
if cls.instance is not None:
return cls.instance
else:
inst = cls.instance = super(Service, cls).__new__()
return inst
And now you import it wherever you want in the rest of the code
from services import Service
Service().do_action()

Adding to the answer given by bruno desthuilliers and TreantBG.
There are certain questions that you can ask about the requirements.
For example one question could be, does the api being called change with different type of objects ?
If the api doesn't change, you will probably be okay with keeping it as a method in some file or class.
If it does change, such that you are calling API 1 for some scenario, API 2 for some and so on and so forth, you will likely be better off with moving/abstracting this logic out to some class (from a better code organisation point of view).
PS: Python allows you to be as flexible as you want when it comes to code organisation. It's really upto you to decide on how you want to organise the code.

Python data attributes can be used by client

I was going through the Python docs and couldn't understand the line:
Data attributes may be referenced by methods as well as by ordinary
users (“clients”) of an object In other words, classes are not usable
to implement pure abstract data types. In fact, nothing in Python
makes it possible to enforce data hiding — it is all based upon
convention. (On the other hand, the Python implementation, written in
C, can completely hide implementation details and control access to an
object if necessary; this can be used by extensions to Python written
in C.)

What's the Python version for “Code against an interface, not an object”?

Inspired by a great question (and bunch of great answers) from here.
Does the statement "Code against an interface, not an object" have any significance in Python?
I'm looking for answers like the ones in the Original Question but with Python snippets and thoughts.

"Code against an interface, not an object" doesn't make literal sense in Python because the language doesn't have an interface feature. The rough Python equivalent is "use duck typing." If you want to see if an object is a duck, in other words, you should check to see whether it has a quack() method, or better yet try to quack() and provide appropriate error handling, not test to see if it is an instance of Duck.
Common duck types in Python are files (well, really, file-like objects), mappings (dict-like objects), callables (function-like objects), sequences (list-like objects), and iterables (things you can iterate over, which can be containers or generators).
As an example, Python features that want a file will generally be happy to accept an object that implements the methods of file it needs; it needn't be derived from the file class. To use an object as standard out, for example, the main thing it is going to need is a write() method (and maybe flush() and close(), which needn't actually do anything). Similarly, a callable is any object that has a __call__() method; it needn't be derived from the function type (in fact, you can't derive from the function type).
You should take a similar approach. Check for the methods and attributes you need for what you're going to do with an object. Better yet, document what you expect and assume that whoever is calling your code is not a total doofus. (If they give you an object you can't use, they will certainly figure that out quickly enough from the errors they get.) Test for specific types only when necessary. It is necessary at times, which is why Python gives you type(), isinstance(), and issubclass(), but be careful with them.
Python's duck typing is equivalent to "code against an interface, not an object" in the sense that you're advised not to make your code too reliant on an object's type, but rather to see whether it has the interface you need. The difference is that in Python, "interface" just means an informal bundle of attributes and methods of an object that provide a certain behavior, rather than a language construct specifically named interface.
You can formalize Python "interfaces" to some extent using the abc module, which allows you to declare that a given class is a subclass of a given "abstract base class" (interface) using any criteria you desire, such as "it has attributes color, tail_length, and quack, and quack is callable." But this is still much less strict than static languages having an interface feature.

To understand interfaces in Python you have to understand duck-typing. From the very Python glossary:
duck-typing: A programming style which does not look at an object’s type to determine if it has the right interface; instead, the method or attribute is simply called or used (“If it looks like a duck and quacks like a duck, it must be a duck.”) By emphasizing interfaces rather than specific types, well-designed code improves its flexibility by allowing polymorphic substitution. Duck-typing avoids tests using type() or isinstance(). (Note, however, that duck-typing can be complemented with abstract base classes.) Instead, it typically employs hasattr() tests or EAFP programming.
Python encourages coding for interfaces, only they are not enforced but by convention. Concepts like iterables, callables or the file interface are very pervasive in Python - as well the builtins that rely on interfaces like map, filter or reduce.

An interface means you expect certain methods to be present and standardised across objects; that is the point of an interface or abstract base class, or whatever implementation you wish to consider.
For example (Java), one might have an interface for symmetric encryption like so:
public interface cipher
{
public void encrypt(byte[] block, byte[] key);
public void decrypt(byte[] block, byte[] key);
}
Then you can implement it:
public class aes128 implements cipher
{
public void encrypt(byte[] block, byte[] key)
{
//...
}
public void decrypt(byte[] block, byte[] key)
{
//...
}
}
It is then possible to declare an object like so:
cipher c;
What have we done here? Well, we've created this object c whose type must match that of the interface. c can refer to anything that matches this interface, so the next stage would be:
c = new aes128();
You can now call methods you expect a cipher to have.
That's java. Now here's what you do in python:
class aes128(Object):
def __init__(self):
pass
def encrypt(self, block, key):
# here I am going to pass, but you really
# should check what you were passed, it could be
# anything. Don't forget, if you're a frog not a duck
# not to quack!
pass
When you want to use this, and you're not sure that the object you've been passed is, just try to use it:
c = aes128()
try:
c.encrypt(someinput, someoutput)
except:
print "eh? No encryption method?!"
Here, you're relying on c.encrypt's implementation to raise if it can't handle what it has been passed, if the method exists. Of course, if c is a string type and therefore not of the right type you require, it will also throw automatically, and you will catch (hopefully).
In short, one form of programming is typed such that you have to obey the interface rules, the other is saying you don't even need to write them down, you simply trust that if it didn't error, it worked.
I hope that shows you the practical difference between the two.

What's the Python version for “Code against an interface, not an object.”?
The proper quote is "program against an interface, not an implementation". The principle holds the same way in Python that it did in Smalltalk, the language it in which it originated.
Does the statement "Code against an interface, not an object." have any significance in Python?
Yes. It has the same significance in Python as it has in the language that this quote originated from (Smalltalk), and in every other language.