Develop Reference

Are "Iterable", "Iterator" examples of types in python?

This is a question about how python's (dynamic) type system works. I have read articles online saying that to define a class to be "an iterable", we need to define a __iter__ function for it. We don't in fact have to explicitly state that that class "is an iterable". I would have guessed based on experience with other languages that I'd have to write something like
class Foo extends Iterable:
def __iter__(self):
return self
When I test the type of Foo and one of its instances I get:
print(type(Foo))
print(type(Foo()))
print(isinstance(Foo(), collections.abc.Iterable))
Output:
<class 'type'>
<class '__main__.Foo'>
True
My question is: What is the status of a concept like "Iterable" in python's (dynamic) type system? Should I think of it as having anything to do with types at all?

A type is what python calls any object that's been defined via a class statement. Using Java as a reference point, type is akin to java.lang.Class - the class that represents classes.
"Iterable" is more akin to an interface than a class - whereas a class/type defines both an internal state and certain methods, an interface only defines those methods. Python doesn't formalize this like other languages, but the principle is used in most of python's "hidden" methods (the ones that have two underscores on both sides). If a particular hidden method is defined for a class (for example, __iter__()), then that class is considered to be iterable*.
In your example, you use isinstance() to prove your point. The python documentation actually has a page on collections.abc, which goes into detail about their behavior:
This module provides abstract base classes that can be used to test whether a class provides a particular interface; for example, whether it is hashable or whether it is a mapping.
(bold added for emphasis).
And it even mentions Iterators specifically:
class collections.abc.Iterable
ABC for classes that provide the __iter__() method.
Checking isinstance(obj, Iterable) detects classes that are registered as Iterable or that have an __iter__() method, but it does not detect classes that iterate with the __getitem__() method. The only reliable way to determine whether an object is iterable is to call iter(obj).
Python allows for ABC classes essentially hijack the builtin isinstance() call via defining the __instancecheck__() metaclass method, which is why isinstance(Foo(), Iterable) is able to return True despite Foo not inheriting from Iterable.
*while the inputs and outputs for these functions aren't as strictly defined as in static-typed languages like Java, the built-in methods that call them have very specific expectations that, in effect, result in the same thing. For example, I once ran into a problem trying to override __len__() on an object to return a float instead of an int, because the built-in len() threw an error when I tried to use it on that object.

Where is the descriptor protocol for python functions described in the language reference?

I stumbled upon this nice trick to dynamically assign a bound method to a class instance in Python:
class X: pass
def f(self): pass
x = X()
x.f = f.__get__(x)
What I want to know is where this behavior is specified in the reference. Here's the closest I've found:
PEP 590
Descriptor HowTo Guide
I'd like to know if this behavior is in fact specified in the language reference somewhere.
It seems like an important enough use case to be guaranteed by the documentation (i.e. it's not clear if what appears in a HowTo demonstrates a guaranteed language feature or makes use of an implementation detail, and I'd like to think that, in principle, all guaranteed functionality can be deduced from the spec without referring to PEPs).

You're probably looking for this bit`:
object.__get__(self, instance, owner=None)
Called to get the attribute of the owner class (class attribute access) or of an instance of that class (instance attribute access). The optional owner argument is the owner class.
You're essentially calling function.__get__, whose rather simple implementation (in CPython anyway) is here; it basically calls PyMethod_New, which basically just binds a function with a self.

Deferred assignment of Python meta‑class parameters (kind of abstract meta‑class instance)

Say a meta‑class A expects parameters. A class B of this meta‑class, will have to assign these parameters. Say this class B is to be a base class, and that's its imediat derived classes which are expected to assign these parameters. Is this feasible if this makes sense? If ever this does not make sense, so why?
With the hope it will help to understand the question, here is an extract of the concrete case (which does not pretend to be marvelous), where only the relevant parts are shown (not strictly valid Python, due to place‑holders). Please note this example is valid with Python 3, and I'm not sure it is with the prior Python version (a point I'm adding after a comment).
class Type(type)
def __new__(…, elements)
…
class ArrowType(Type):
def __new__(…, domain, codomain)
# This build a suitable `elements`
# parameter, from its own arguments,
# for the call to `Type.__new__`
…
class Function(
object,
metaclass=ArrowType,
domain=…,
codomain=…)
…
Say the Function class is expected to be a base class, but it's not concrete. It's abstract with regard to its domain and codomain parameters, intended to ArrowType. Say there is no defaults justified, and providing defaults in the vein of None or () or who‑know what, is not considered good (looks like a hack to me).
I tried to make Function abstract, but failed to get a solution from this, as the parameters still needs to be instantiated at the point of Function's definition. Or may be I did it erroneously?
Well, I could use a function:
def function(domain, codomain):
class Function(
object,
metaclass=ArrowType,
domain=domain,
codomain=codomain)
…
return Function
This works, but I don't like it, for multiple reasons: 1) Function is “hidden” in a function body, 2) this is not anymore a single base class (and neither really abstract), as it ends into as many base classes as there are classes derived from it (function being invoked for each), 3) the class‑names the instances gets as displayed by type(…), are ugly.
In few words, what's expected, is to have Function to be abstract, with respect to its domain and codomain parameters, which would be assigned something in its derived only classes.

Python builtin functions aren't really functions, right?

Was just thinking about Python's dict "function" and starting to realize that dict isn't really a function at all. For example, if we do dir(dict), we get all sorts of methods that aren't include in the usual namespace of an user defined function. Extending that thought, its similar to dir(list) and dir(len). They aren't function, but really types. But then I'm confused about the documentation page, http://docs.python.org/2/library/functions.html, which clearly says functions. (I guess it should really just says builtin callables)
So what gives? (Starting to seem that making the distinction of classes and functions is trivial)

It's a callable, as are classes in general. Calling dict() is effectively to call the dict constructor. It is like when you define your own class (C, say) and you call C() to instantiate it.

One way that dict is special, compared to, say, sum, is that though both are callable, and both are implemented in C (in cpython, anyway), dict is a type; that is, isinstance(dict, type) == True. This means that you can use dict as the base class for other types, you can write:
class MyDictSubclass(dict):
pass
but not
class MySumSubclass(sum):
pass
This can be useful to make classes that behave almost like a builtin object, but with some enhancements. For instance, you can define a subclass of tuple that implements + as vector addition instead of concatenation:
class Vector(tuple):
def __add__(self, other):
return Vector(x + y for x, y in zip(self, other))
Which brings up another interesting point. type is also implemented in C. It's also callable. Like dict (and unlike sum) it's an instance of type; isinstance(type, type) == True. Because of this weird, seemingly impossible cycle, type can be used to make new classes of classes, (called metaclasses). You can write:
class MyTypeSubclass(type):
pass
class MyClass(object):
__metaclass__ = MyTypeSubclass
or, in Python 3:
class MyClass(metaclass=MyTypeSubclass):
pass
Which give the interesting result that isinstance(MyClass, MyTypeSubclass) == True. How this is useful is a bit beyond the scope of this answer, though.

dict() is a constructor for a dict instance. When you do dir(dict) you're looking at the attributes of class dict. When you write a = dict() you're setting a to a new instance of type dict.
I'm assuming here that dict() is what you're referring to as the "dict function". Or are you calling an indexed instance of dict, e.g. a['my_key'] a function?

Note that calling dir on the constructor dict.__init__
dir(dict.__init__)
gives you what you would expect, including the same stuff as you'd get for any other function. Since a call to the dict() constructor results in a call to dict.__init__(instance), that explains where those function attributes went. (Of course there's a little extra behind-the-scenes work in any constructor, but that's the same for dicts as for any object.)

Python metaclasses

I've been hacking classes in Python like this:
def hack(f,aClass) :
class MyClass(aClass) :
def f(self) :
f()
return MyClass
A = hack(afunc,A)
Which looks pretty clean to me. It takes a class, A, creates a new class derived from it that has an extra method, calling f, and then reassigns the new class to A.
How does this differ from metaclass hacking in Python? What are the advantages of using a metaclass over this?

The definition of a class in Python is an instance of type (or an instance of a subclass of type). In other words, the class definition itself is an object. With metaclasses, you have the ability to control the type instance that becomes the class definition.
When a metaclass is invoked, you have the ability to completely re-write the class definition. You have access to all the proposed attributes of the class, its ancestors, etc. More than just injecting a method or removing a method, you can radically alter the inheritance tree, the type, and pretty much any other aspect. You can also chain metaclasses together for a very dynamic and totally convoluted experience.
I suppose the real benefit, though is that the class's type remains the class's type. In your example, typing:
a_inst = A()
type(a_inst)
will show that it is an instance of MyClass. Yes, isinstance(a_inst, aClass) would return True, but you've introduced a subclass, rather than a dynamically re-defined class. The distinction there is probably the key.
As rjh points out, the anonymous inner class also has performance and extensibility implications. A metaclass is processed only once, and the moment that the class is defined, and never again. Users of your API can also extend your metaclass because it is not enclosed within a function, so you gain a certain degree of extensibility.
This slightly old article actually has a good explanation that compares exactly the "function decoration" approach you used in the example with metaclasses, and shows the history of the Python metaclass evolution in that context: http://www.ibm.com/developerworks/linux/library/l-pymeta.html

You can use the type callable as well.
def hack(f, aClass):
newfunc = lambda self: f()
return type('MyClass', (aClass,), {'f': newfunc})
I find using type the easiest way to get into the metaclass world.

A metaclass is the class of a class. IMO, the bloke here covered it quite serviceably, including some use-cases. See Stack Overflow question "MetaClass", "new", "cls" and "super" - what is the mechanism exactly?.