Due to the wierdness of the particular module I am working with, I am wondering if there is a general way for one class to inherit the properties of another class that is locally created in a method. For example, something the following:
def DefForm():
class F(object):
foo = bar
return F
class MyClass(DefForm):
pass
m = myClass()
m.foo
>>> 'bar'
Inherit your class from the return value of the function not the function itself:
class MyClass(DefForm()):
...
Classes in Python are plain objects and when you're inheriting from one, you're also just inheriting from an object.
For instance, this also works:
class Foo(Bar if x == 3 else Baz):
...
In fact, I can't think of a situation where this wouldn't work. Even this is perfectly valid:
try:
...
except (FooExc if x == 3 else BarExc):
...
So not only are classes objects in Python, they are also treated as objects in all situations.
On a more general note, generating classes is called metaprogramming; it's a common practice in many languages (both dynamic as well as compiled and statically typed) and there's generally nothing "weird" about code that does this, as long as sanity and readability are maintained. It's as normal as creating and returning functions from other functions (which is extremely prevalent in Functional Programming), or in fact returning any object.
Try this:
def DefForm():
class F(object):
foo = bar
return F
FClass = DeftForm()
class MyClass(FClass):
pass
m = myClass()
m.foo
Related
I'm surprised that the following code runs without error.
# ABC
class Foo(object):
__metaclass__ = ABCMeta
a = 1
def __init__(self, b, c):
self.b = b
self.c = c
def get_scaled_a(self):
return self.a / Bar1.a # why can I access Bar1.a?
#abstractmethod
def class_type(self):
pass
# Derived class 1
class Bar1(Foo):
a = 100
def class_type(self):
return 'bar1'
# Derived class 2
class Bar2(Foo):
a = 10
def class_type(self):
return 'bar2'
my_bar2_inst = Bar2(0, 0)
print(my_bar2_inst.get_scaled_a())
# 0.1
# Why can I access Bar.a?
Because Python assumes that developers are mature human beings. Instead of the interpreter checking whether you have access to a certain attribute, in Python you usually have access to all attributes. It is up to you to be mature enough and don't break anything.
There is however a convention that attributes starting with a lowercase, like _foo, __bar and __qux__ are considered private. It means it is usually a bad idea to access those yourself. But there is no mechanism in place to prevent you from accessing them: the variable name more or less asks that you would be so kind not to access it. In case you absolutely need it, it is your responsibility.
Now the a of Bar is a member of the Bar class, not of a Bar instance. So in some other languages, it would be considered to be "static". That's another reason why you can access it.
When you run get_scaled_a, the Bar1 class has already been defined, and so has its a attribute. Remember that classes are objects too, not just class instances. So when Bar1 is defined, you can access certain attributes without creating an instance of it.
The fact that it is a subclass doesn't play a role at all.
You can access the child class attribute for the same reason you can do this:
>>> class myobj: pass
>>> def f(obj):
print(obj.a)
>>> obj1, obj2 = myobj(), myobj()
>>> obj1.a, obj2.a = 1, 2
>>> f(obj1)
1
>>> f(obj2)
2
Classes are just objects. Metaclasses are little more than class-object-factories.
Here's another illustration. Say I have a Reuben maker:
>>> Reuben = type('Reuben', (), {'mayo':False}) # <-- shortcut for making a class
Now say I have some kind of worthless sandwich making factory that puts mayo on everything (gross):
>>> def MakeMeASandwich(sandwich_type):
sandwich_type.mayo = True
return sandwich_type()
Would you expect that to work? If you said yes, you're right:
>>> s = MakeMeASandwich(Reuben)
# reuben with mayo!
Why did you expect that to work? Probably because there is no reason the function shouldn't be able to access mayo. It's there. It's not hidden. So of course it can get to it.
Again: a metaclass is little more than a class making factory. It is very much the same as any other factory (though they do have some nifty extra bells and whistles that you probably don't need).
It's possible to use type in Python to create a new class object, as you probably know:
A = type('A', (object,), {})
a = A() # create an instance of A
What I'm curious about is whether there's any problem with creating different class objects with the same name, eg, following on from the above:
B = type('A', (object,), {})
In other words, is there an issue with this second class object, B, having the same name as our first class object, A?
The motivation for this is that I'd like to get a clean copy of a class to apply different decorators to without using the inheritance approach described in this question.
So I'd like to define a class normally, eg:
class Fruit(object):
pass
and then make a fresh copy of it to play with:
def copy_class(cls):
return type(cls.__name__, cls.__bases__, dict(cls.__dict__))
FreshFruit = copy_class(fruit)
In my testing, things I do with FreshFruit are properly decoupled from things I do to Fruit.
However, I'm unsure whether I should also be mangling the name in copy_class in order to avoid unexpected problems.
In particular, one concern I have is that this could cause the class to be replaced in the module's dictionary, such that future imports (eg, from module import Fruit return the copied class).
There is no reason why you can't have 2 classes with the same __name__ in the same module if you want to and have a good reason to do so.
e.g. In your example from module import Fruit -- python doesn't care at all about the __name__ of the class. It looks in the module's globals for Fruit and imports what it finds there.
Note that, in general, this approach isn't great if you're using super (although the same can be said for class decorators ...):
class A(Base):
def foo(self):
super(A, self).foo()
B = copy_class(A)
In this case, when B.foo is called, it will end up calling super(A, self) which could lead to funky behaviour in a number of circumstances. . .
I was just reading the Python documentation about classes; it says, in Python "classes themselves are objects". How is that different from classes in C#, Java, Ruby, or Smalltalk?
What advantages and disadvantages does this type of classes have compared with those other languages?
In Python, classes are objects in the sense that you can assign them to variables, pass them to functions, etc. just like any other objects. For example
>>> t = type(10)
>>> t
<type 'int'>
>>> len(t.__dict__)
55
>>> t() # construct an int
0
>>> t(10)
10
Java has Class objects which provide some information about a class, but you can't use them in place of explicit class names. They aren't really classes, just class information structures.
Class C = x.getClass();
new C(); // won't work
Declaring a class is simply declaring a variable:
class foo(object):
def bar(self): pass
print foo # <class '__main__.foo'>
They can be assigned and stored like any variable:
class foo(object):
pass
class bar(object):
pass
baz = bar # simple variable assignment
items = [foo, bar]
my_foo = items[0]() # creates a foo
for x in (foo, bar): # create one of each type
print x()
and passed around as a variable:
class foo(object):
def __init__(self):
print "created foo"
def func(f):
f()
func(foo)
They can be created by functions, including the base class list:
def func(base_class, var):
class cls(base_class):
def g(self):
print var
return cls
class my_base(object):
def f(self): print "hello"
new_class = func(my_base, 10)
obj = new_class()
obj.f() # hello
obj.g() # 10
By contrast, while classes in Java have objects representing them, eg. String.class, the class name itself--String--isn't an object and can't be manipulated as one. That's inherent to statically-typed languages.
In C# and Java the classes are not objects. They are types, in the sense in which those languages are statically typed. True you can get an object representing a specific class - but that's not the same as the class itself.
In python what looks like a class is actually an object too.
It's exlpained here much better than I can ever do :)
The main difference is that they mean you can easily manipulate the class as an object. The same facility is available in Java, where you can use the methods of Class to get at information about the class of an object. In languages like Python, Ruby, and Smalltalk, the more dynamic nature of the language lets you "open" the class and change it, which is sometimes called "monkey patching".
Personally I don't think the differences are all that much of a big deal, but I'm sure we can get a good religious war started about it.
Classes are objects in that they are manipulable in Python code just like any object. Others have shown how you can pass them around to functions, allowing them to be operated upon like any object. Here is how you might do this:
class Foo(object):
pass
f = Foo()
f.a = "a" # assigns attribute on instance f
Foo.b = "b" # assigns attribute on class Foo, and thus on all instances including f
print f.a, f.b
Second, like all objects, classes are instantiated at runtime. That is, a class definition is code that is executed rather than a structure that is compiled before anything runs. This means a class can "bake in" things that are only known when the program is run, such as environment variables or user input. These are evaluated once when the class is declared and then become a part of the class. This is different from compiled languages like C# which require this sort of behavior to be implemented differently.
Finally, classes, like any object, are built from classes. Just as an object is built from a class, so is a class built from a special kind of class called a metaclass. You can write your own metaclasses to change how classes are defined.
Another advantage of classes being objects is that objects can change their class at runtime:
>>> class MyClass(object):
... def foo(self):
... print "Yo There! I'm a MyCLass-Object!"
...
>>> class YourClass(object):
... def foo(self):
... print "Guess what?! I'm a YourClass-Object!"
...
>>> o = MyClass()
>>> o.foo()
Yo There! I'm a MyCLass-Object!
>>> o.__class__ = YourClass
>>> o.foo()
Guess what?! I'm a YourClass-Object!
Objects have a special attribute __class__ that points to the class of which they are an instance. This is possible only because classes are objects themself, and therefore can be bound to an attribute like __class__.
As this question has a Smalltalk tag, this answer is from a Smalltalk perspective. In Object-Oriented programming, things get done through message-passing. You send a message to an object, if the object understands that message, it executes the corresponding method and returns a value. But how is the object created in the first place? If special syntax is introduced for creating objects that will break the simple syntax based on message passing. This is what happens in languages like Java:
p = new Point(10, 20); // Creates a new Point object with the help of a special keyword - new.
p.draw(); // Sends the message `draw` to the Point object.
As it is evident from the above code, the language has two ways to get things done - one imperative and the other Object Oriented. In contrast, Smalltalk has a consistent syntax based only on messaging:
p := Point new: 10 y: 20.
p draw.
Here new is a message send to a singleton object called Point which is an instance of a Metaclass. In addition to giving the language a consistent model of computation, metaclasses allow dynamic modification of classes. For instance, the following statement will add a new instance variable to the Point class without requiring a recompilation or VM restart:
Point addInstVarName: 'z'.
The best reading on this subject is The Art of the Metaobject Protocol.
I want to have compact class based python DSLs in the following form:
class MyClass(Static):
z = 3
def _init_(cls, x=0):
cls._x = x
def set_x(cls, x):
cls._x = x
def print_x_plus_z(cls):
print cls._x + cls.z
#property
def x(cls):
return cls._x
class MyOtherClass(MyClass):
z = 6
def _init_(cls):
MyClass._init_(cls, x=3)
I don't want to write MyClass() and MyOtherClass() afterwards. Just want to get this working with only class definitions.
MyClass.print_x_plus_z()
c = MyOtherClass
c.z = 5
c.print_x_plus_z()
assert MyOtherClass.z == 5, "instances don't share the same values!"
I used metaclasses and managed to get _init_, print_x and subclassing working properly, but properties don't work.
Could anyone suggest better alternative?
I'm using Python 2.4+
To give a class (as opposed to its instances) a property, you need to have that property object as an attribute of the class's metaclass (so you'll probably need to make a custom metaclass to avoid inflicting that property upon other classes with the same metaclass). Similarly for special methods such as __init__ -- if they're on the class they'd affect the instances (which you don't want to make) -- to have them affect the class, you need to have them on the (custom) metaclass. What are you trying to accomplish by programming everything "one metalevel up", i.e., never-instantiated class with custom metaclass rather than normal instances of a normal class? It just seems a slight amount of extra work for no returns;-).
Python's inner/nested classes confuse me. Is there something that can't be accomplished without them? If so, what is that thing?
Quoted from http://www.geekinterview.com/question_details/64739:
Advantages of inner class:
Logical grouping of classes: If a class is useful to only one other class then it is logical to embed it in that class and keep the two together. Nesting such "helper classes" makes their package more streamlined.
Increased encapsulation: Consider two top-level classes A and B where B needs access to members of A that would otherwise be declared private. By hiding class B within class A A's members can be declared private and B can access them. In addition B itself can be hidden from the outside world.
More readable, maintainable code: Nesting small classes within top-level classes places the code closer to where it is used.
The main advantage is organization. Anything that can be accomplished with inner classes can be accomplished without them.
Is there something that can't be accomplished without them?
No. They are absolutely equivalent to defining the class normally at top level, and then copying a reference to it into the outer class.
I don't think there's any special reason nested classes are ‘allowed’, other than it makes no particular sense to explicitly ‘disallow’ them either.
If you're looking for a class that exists within the lifecycle of the outer/owner object, and always has a reference to an instance of the outer class — inner classes as Java does it – then Python's nested classes are not that thing. But you can hack up something like that thing:
import weakref, new
class innerclass(object):
"""Descriptor for making inner classes.
Adds a property 'owner' to the inner class, pointing to the outer
owner instance.
"""
# Use a weakref dict to memoise previous results so that
# instance.Inner() always returns the same inner classobj.
#
def __init__(self, inner):
self.inner= inner
self.instances= weakref.WeakKeyDictionary()
# Not thread-safe - consider adding a lock.
#
def __get__(self, instance, _):
if instance is None:
return self.inner
if instance not in self.instances:
self.instances[instance]= new.classobj(
self.inner.__name__, (self.inner,), {'owner': instance}
)
return self.instances[instance]
# Using an inner class
#
class Outer(object):
#innerclass
class Inner(object):
def __repr__(self):
return '<%s.%s inner object of %r>' % (
self.owner.__class__.__name__,
self.__class__.__name__,
self.owner
)
>>> o1= Outer()
>>> o2= Outer()
>>> i1= o1.Inner()
>>> i1
<Outer.Inner inner object of <__main__.Outer object at 0x7fb2cd62de90>>
>>> isinstance(i1, Outer.Inner)
True
>>> isinstance(i1, o1.Inner)
True
>>> isinstance(i1, o2.Inner)
False
(This uses class decorators, which are new in Python 2.6 and 3.0. Otherwise you'd have to say “Inner= innerclass(Inner)” after the class definition.)
There's something you need to wrap your head around to be able to understand this. In most languages, class definitions are directives to the compiler. That is, the class is created before the program is ever run. In python, all statements are executable. That means that this statement:
class foo(object):
pass
is a statement that is executed at runtime just like this one:
x = y + z
This means that not only can you create classes within other classes, you can create classes anywhere you want to. Consider this code:
def foo():
class bar(object):
...
z = bar()
Thus, the idea of an "inner class" isn't really a language construct; it's a programmer construct. Guido has a very good summary of how this came about here. But essentially, the basic idea is this simplifies the language's grammar.
Nesting classes within classes:
Nested classes bloat the class definition making it harder to see whats going on.
Nested classes can create coupling that would make testing more difficult.
In Python you can put more than one class in a file/module, unlike Java, so the class still remains close to top level class and could even have the class name prefixed with an "_" to help signify that others shouldn't be using it.
The place where nested classes can prove useful is within functions
def some_func(a, b, c):
class SomeClass(a):
def some_method(self):
return b
SomeClass.__doc__ = c
return SomeClass
The class captures the values from the function allowing you to dynamically create a class like template metaprogramming in C++
I understand the arguments against nested classes, but there is a case for using them in some occasions. Imagine I'm creating a doubly-linked list class, and I need to create a node class for maintaing the nodes. I have two choices, create Node class inside the DoublyLinkedList class, or create the Node class outside the DoublyLinkedList class. I prefer the first choice in this case, because the Node class is only meaningful inside the DoublyLinkedList class. While there's no hiding/encapsulation benefit, there is a grouping benefit of being able to say the Node class is part of the DoublyLinkedList class.
Is there something that can't be accomplished without them? If so,
what is that thing?
There is something that cannot be easily done without: inheritance of related classes.
Here is a minimalist example with the related classes A and B:
class A(object):
class B(object):
def __init__(self, parent):
self.parent = parent
def make_B(self):
return self.B(self)
class AA(A): # Inheritance
class B(A.B): # Inheritance, same class name
pass
This code leads to a quite reasonable and predictable behaviour:
>>> type(A().make_B())
<class '__main__.A.B'>
>>> type(A().make_B().parent)
<class '__main__.A'>
>>> type(AA().make_B())
<class '__main__.AA.B'>
>>> type(AA().make_B().parent)
<class '__main__.AA'>
If B were a top-level class, you could not write self.B() in the method make_B but would simply write B(), and thus lose the dynamic binding to the adequate classes.
Note that in this construction, you should never refer to class A in the body of class B. This is the motivation for introducing the parent attribute in class B.
Of course, this dynamic binding can be recreated without inner class at the cost of a tedious and error-prone instrumentation of the classes.
1. Two functionally equivalent ways
The two ways shown before are functionally identical. However, there are some subtle differences, and there are situations when you would like to choose one over another.
Way 1: Nested class definition (="Nested class")
class MyOuter1:
class Inner:
def show(self, msg):
print(msg)
Way 2: With module level Inner class attached to Outer class(="Referenced inner class")
class _InnerClass:
def show(self, msg):
print(msg)
class MyOuter2:
Inner = _InnerClass
Underscore is used to follow PEP8 "internal interfaces (packages, modules, classes, functions, attributes or other names) should -- be prefixed with a single leading underscore."
2. Similarities
Below code snippet demonstrates the functional similarities of the "Nested class" vs "Referenced inner class"; They would behave the same way in code checking for the type of an inner class instance. Needless to say, the m.inner.anymethod() would behave similarly with m1 and m2
m1 = MyOuter1()
m2 = MyOuter2()
innercls1 = getattr(m1, 'Inner', None)
innercls2 = getattr(m2, 'Inner', None)
isinstance(innercls1(), MyOuter1.Inner)
# True
isinstance(innercls2(), MyOuter2.Inner)
# True
type(innercls1()) == mypackage.outer1.MyOuter1.Inner
# True (when part of mypackage)
type(innercls2()) == mypackage.outer2.MyOuter2.Inner
# True (when part of mypackage)
3. Differences
The differences of "Nested class" and "Referenced inner class" are listed below. They are not big, but sometimes you would like to choose one or the other based on these.
3.1 Code Encapsulation
With "Nested classes" it is possible to encapsulate code better than with "Referenced inner class". A class in the module namespace is a global variable. The purpose of nested classes is to reduce clutter in the module and put the inner class inside the outer class.
While no-one* is using from packagename import *, low amount of module level variables can be nice for example when using an IDE with code completion / intellisense.
*Right?
3.2 Readability of code
Django documentation instructs to use inner class Meta for model metadata. It is a bit more clearer* to instruct the framework users to write a class Foo(models.Model) with inner class Meta;
class Ox(models.Model):
horn_length = models.IntegerField()
class Meta:
ordering = ["horn_length"]
verbose_name_plural = "oxen"
instead of "write a class _Meta, then write a class Foo(models.Model) with Meta = _Meta";
class _Meta:
ordering = ["horn_length"]
verbose_name_plural = "oxen"
class Ox(models.Model):
Meta = _Meta
horn_length = models.IntegerField()
With the "Nested class" approach the code can be read a nested bullet point list, but with the "Referenced inner class" method one has to scroll back up to see the definition of _Meta to see its "child items" (attributes).
The "Referenced inner class" method can be more readable if your code nesting level grows or the rows are long for some other reason.
* Of course, a matter of taste
3.3 Slightly different error messages
This is not a big deal, but just for completeness: When accessing non-existent attribute for the inner class, we see slighly different exceptions. Continuing the example given in Section 2:
innercls1.foo()
# AttributeError: type object 'Inner' has no attribute 'foo'
innercls2.foo()
# AttributeError: type object '_InnerClass' has no attribute 'foo'
This is because the types of the inner classes are
type(innercls1())
#mypackage.outer1.MyOuter1.Inner
type(innercls2())
#mypackage.outer2._InnerClass
The main use case I use this for is the prevent proliferation of small modules and to prevent namespace pollution when separate modules are not needed. If I am extending an existing class, but that existing class must reference another subclass that should always be coupled to it. For example, I may have a utils.py module that has many helper classes in it, that aren't necessarily coupled together, but I want to reinforce coupling for some of those helper classes. For example, when I implement https://stackoverflow.com/a/8274307/2718295
:utils.py:
import json, decimal
class Helper1(object):
pass
class Helper2(object):
pass
# Here is the notorious JSONEncoder extension to serialize Decimals to JSON floats
class DecimalJSONEncoder(json.JSONEncoder):
class _repr_decimal(float): # Because float.__repr__ cannot be monkey patched
def __init__(self, obj):
self._obj = obj
def __repr__(self):
return '{:f}'.format(self._obj)
def default(self, obj): # override JSONEncoder.default
if isinstance(obj, decimal.Decimal):
return self._repr_decimal(obj)
# else
super(self.__class__, self).default(obj)
# could also have inherited from object and used return json.JSONEncoder.default(self, obj)
Then we can:
>>> from utils import DecimalJSONEncoder
>>> import json, decimal
>>> json.dumps({'key1': decimal.Decimal('1.12345678901234'),
... 'key2':'strKey2Value'}, cls=DecimalJSONEncoder)
{"key2": "key2_value", "key_1": 1.12345678901234}
Of course, we could have eschewed inheriting json.JSONEnocder altogether and just override default():
:
import decimal, json
class Helper1(object):
pass
def json_encoder_decimal(obj):
class _repr_decimal(float):
...
if isinstance(obj, decimal.Decimal):
return _repr_decimal(obj)
return json.JSONEncoder(obj)
>>> json.dumps({'key1': decimal.Decimal('1.12345678901234')}, default=json_decimal_encoder)
'{"key1": 1.12345678901234}'
But sometimes just for convention, you want utils to be composed of classes for extensibility.
Here's another use-case: I want a factory for mutables in my OuterClass without having to invoke copy:
class OuterClass(object):
class DTemplate(dict):
def __init__(self):
self.update({'key1': [1,2,3],
'key2': {'subkey': [4,5,6]})
def __init__(self):
self.outerclass_dict = {
'outerkey1': self.DTemplate(),
'outerkey2': self.DTemplate()}
obj = OuterClass()
obj.outerclass_dict['outerkey1']['key2']['subkey'].append(4)
assert obj.outerclass_dict['outerkey2']['key2']['subkey'] == [4,5,6]
I prefer this pattern over the #staticmethod decorator you would otherwise use for a factory function.
I have used Python's inner classes to create deliberately buggy subclasses within unittest functions (i.e. inside def test_something():) in order to get closer to 100% test coverage (e.g. testing very rarely triggered logging statements by overriding some methods).
In retrospect it's similar to Ed's answer https://stackoverflow.com/a/722036/1101109
Such inner classes should go out of scope and be ready for garbage collection once all references to them have been removed. For instance, take the following inner.py file:
class A(object):
pass
def scope():
class Buggy(A):
"""Do tests or something"""
assert isinstance(Buggy(), A)
I get the following curious results under OSX Python 2.7.6:
>>> from inner import A, scope
>>> A.__subclasses__()
[]
>>> scope()
>>> A.__subclasses__()
[<class 'inner.Buggy'>]
>>> del A, scope
>>> from inner import A
>>> A.__subclasses__()
[<class 'inner.Buggy'>]
>>> del A
>>> import gc
>>> gc.collect()
0
>>> gc.collect() # Yes I needed to call the gc twice, seems reproducible
3
>>> from inner import A
>>> A.__subclasses__()
[]
Hint - Don't go on and try doing this with Django models, which seemed to keep other (cached?) references to my buggy classes.
So in general, I wouldn't recommend using inner classes for this kind of purpose unless you really do value that 100% test coverage and can't use other methods. Though I think it's nice to be aware that if you use the __subclasses__(), that it can sometimes get polluted by inner classes. Either way if you followed this far, I think we're pretty deep into Python at this point, private dunderscores and all.