I tried to write simple class factory using the "type" method.
I called myclass_factory twice and It returned identical namespaces
(case1 and case2). But the values of myclass attribute in that namespaces were different (!).
Actualy it just what I need but I can not undestnad why I got that result.
For my udestanding, since case1 and case2 are not objects but just same namespace (<class 'main.MyClass'>) they should refer to the same memory and should be case1.myclass = case2.myclass
Please explain how It could be so?
>>> def myclass_factory(myclass):
... return type('MyClass',(object,),{'myclass': myclass})
...
>>> class class1:
... pass
...
>>> class class2:
... pass
...
>>> case1 = myclass_factory(class1)
>>> case2 = myclass_factory(class2)
>>>
>>> case1
<class '__main__.MyClass'>
>>> case2
<class '__main__.MyClass'>
>>>
>>> case1.myclass
<class '__main__.class1'>
>>> case2.myclass
<class '__main__.class2'>
>>>
>>>
Actually case 1 and case 2 are objects since you instantiate them. They are type objects and thus ordinary python objects. So you create two different type objects.
also see this answer: Link to answer
Also see This description
This is essentially equivalent to the following "normal" code for defining classes:
def MyClass(object):
myclass = class1
case1 = MyClass
def MyClass(object):
myclass = class2
case2 = MyClass
print(case1.myclass)
print(case2.myclass)
Types/classes are first-class objects, and creating a new type with the same name has no effect on the previous class one with that name that was saved in the variable case1.
Related
My question is pretty general in principle. I have a class called Menu that has a list of items and one or more of those items can be either a string, or another instance of Menu. My code for that looks like this:
class Menu():
def __init__(self):
self.items = []
def add_item(self, item):
self.items.append(item)
def add_menu(self):
self.add_item(Menu())
As you can see I've used the actual name of the class Menu within one of it's functions. My question is if it's possible to do that without writing the actual name of the class, but rather by referring to what class it's defined in? For example, I've tried
self.add_item(super(self))
Which gives TypeError: super() argument 1 must be type, not Menu and also tried
self.add_item(super())
That runs without error, but the object it inserts is <super: <class 'Menu'>, <Menu object>>
I'm beginning to suspect i'm using the wrong tool for the job, and my question is what i'm doing wrong? and is referencing of the type I require even possible?
If it's relevant my python version is 3.5.3
Sure it is possible:
>>> class A:
... def create_instance(self):
... return type(self)()
...
>>> a1 = A()
>>> a2 = a1.add_self()
>>> a1
<__main__.A object at 0x1029c27f0>
>>> a2
<__main__.A object at 0x1029c28d0>
Note, of course, this is because:
>>> type(a1)
<class '__main__.A'>
>>> A
<class '__main__.A'>
>>> type(a1) is A
True
Alternatively, this may also be a use-case for classmethod:
>>> class A:
... #classmethod
... def make_instance(cls):
... return cls()
...
>>> a1 = A()
>>> a2 = a1.make_instance()
>>> a1
<__main__.A object at 0x1029c29b0>
>>> a2
<__main__.A object at 0x1029c27f0>
Now, it is perfectly reasonable for instances of a class to return new instances of the same class, but whether or not it is advisable in your case I don't think I have enough information to give an opinion. But it is certainly possible.
Wrong abstraction: consider creating a class MenuItem for example - that represents single line menu entries, or sub menus. Or some other kind of menu entry that you can't think of today.
In other words: good OOP is about creating helpful abstractions. Menu items can have many different shapes, thus the better answer is not to fit in raw strings, but think up an inheritance hierarchy that supports you solving your problem.
Suppose I have a class with __slots__
class A:
__slots__ = ['x']
a = A()
a.x = 1 # works fine
a.y = 1 # AttributeError (as expected)
Now I am going to change __slots__ of A.
A.__slots__.append('y')
print(A.__slots__) # ['x', 'y']
b = A()
b.x = 1 # OK
b.y = 1 # AttributeError (why?)
b was created after __slots__ of A had changed, so Python, in principle, could allocate memory for b.y. Why it didn't?
How to properly modify __slots__ of a class, so that new instances have the modified attributes?
You cannot dynamically alter the __slots__ attribute after creating the class, no. That's because the value is used to create special descriptors for each slot. From the __slots__ documentation:
__slots__ are implemented at the class level by creating descriptors (Implementing Descriptors) for each variable name. As a result, class attributes cannot be used to set default values for instance variables defined by __slots__; otherwise, the class attribute would overwrite the descriptor assignment.
You can see the descriptors in the class __dict__:
>>> class A:
... __slots__ = ['x']
...
>>> A.__dict__
mappingproxy({'__module__': '__main__', '__doc__': None, 'x': <member 'x' of 'A' objects>, '__slots__': ['x']})
>>> A.__dict__['x']
<member 'x' of 'A' objects>
>>> a = A()
>>> A.__dict__['x'].__get__(a, A)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: x
>>> A.__dict__['x'].__set__(a, 'foobar')
>>> A.__dict__['x'].__get__(a, A)
'foobar'
>>> a.x
'foobar'
You cannot yourself create these additional descriptors. Even if you could, you cannot allocate more memory space for the extra slot references on the instances produced for this class, as that's information stored in the C struct for the class, and not in a manner accessible to Python code.
That's all because __slots__ is only an extension of the low-level handling of the elements that make up Python instances to Python code; the __dict__ and __weakref__ attributes on regular Python instances were always implemented as slots:
>>> class Regular: pass
...
>>> Regular.__dict__['__dict__']
<attribute '__dict__' of 'Regular' objects>
>>> Regular.__dict__['__weakref__']
<attribute '__weakref__' of 'Regular' objects>
>>> r = Regular()
>>> Regular.__dict__['__dict__'].__get__(r, Regular) is r.__dict__
True
All the Python developers did here was extend the system to add a few more of such slots using arbitrary names, with those names taken from the __slots__ attribute on the class being created, so that you can save memory; dictionaries take more memory than simple references to values in slots do. By specifying __slots__ you disable the __dict__ and __weakref__ slots, unless you explicitly include those in the __slots__ sequence.
The only way to extend slots then is to subclass; you can dynamically create a subclass with the type() function or by using a factory function:
def extra_slots_subclass(base, *slots):
class ExtraSlots(base):
__slots__ = slots
ExtraSlots.__name__ = base.__name__
return ExtraSlots
It appears to me a type turns __slots__ into a tuple as one of it's first orders of action. It then stores the tuple on the extended type object. Since beneath it all, the python is looking at a tuple, there is no way to mutate it. Indeed, I'm not even sure you can access it unless you pass a tuple in to the instance in the first place.
The fact that the original object that you set still remains as an attribute on the type is (perhaps) just a convenience for introspection.
You can't modify __slots__ and expect to have that show up somewhere (and really -- from a readability perspective, You probably don't really want to do that anyway, right?)...
Of course, you can always subclass to extend the slots:
>>> class C(A):
... __slots__ = ['z']
...
>>> c = C()
>>> c.x = 1
>>> c.z = 1
You cannot modify the __slots__ attribute after class creation. This is because it would leade to strange behaviour.
Imagine the following.
class A:
__slots__ = ["x"]
a = A()
A.__slots__.append("y")
a.y = None
What should happen in this scenario? No space was originally allocated for a second slot, but according to the slots attribute, a should be able have space for y.
__slots__ is not about protecting what names can and cannot be accessed. Rather __slots__ is about reducing the memory footprint of an object. By attempting to modify __slots__ you would defeat the optimisations that __slots__ is meant to achieve.
How __slots__ reduces memory footprint
Normally, an object's attributes are stored in a dict, which requires a fair bit of memory itself. If you are creating millions of objects then the space required by these dicts becomes prohibitive. __slots__ informs the python machinery that makes the class object that there will only be so many attributes refered to by instances of this class and what the names of the attributes will be. Therefore, the class can make an optimisation by storing the attributes directly on the instance rather than in a dict. It places the memory for the (pointers to the) attributes directly on the object, rather than creating a new dict for the object.
Putting answers to this and related question together, I want to make an accent on a solution to this problem:
You can kind of modify __slots__ by creating a subclass with the same name and then replacing parent class with its child. Note that you can do this for classes declared and used in any module, not just yours!
Consider the following module which declares some classes:
module.py:
class A(object):
# some class a user should import
__slots__ = ('x', 'b')
def __init__(self):
self.b = B()
class B(object):
# let's suppose we can't use it directly,
# it's returned as a part of another class
__slots__ = ('z',)
Here's how you can add attributes to these classes:
>>> import module
>>> from module import A
>>>
>>> # for classes imported into your module:
>>> A = type('A', (A,), {'__slots__': ('foo',)})
>>> # for classes which will be instantiated by the `module` itself:
>>> module.B = type('B', (module.B,), {'__slots__': ('bar',)})
>>>
>>> a = A()
>>> a.x = 1
>>> a.foo = 2
>>>
>>> b = a.b
>>> b.z = 3
>>> b.bar = 4
>>>
But what if you receive class instances from some third-party module using the module?
module_3rd_party.py:
from module import A
def get_instance():
return A()
No problem, it will also work! The only difference is that you may need to patch them before you import third-party module (in case it imports classes from the module):
>>> import module
>>>
>>> module.A = type('A', (module.A,), {'__slots__': ('foo',)})
>>> module.B = type('B', (module.B,), {'__slots__': ('bar',)})
>>>
>>> # note that we import `module_3rd_party` AFTER we patch the `module`
>>> from module_3rd_party import get_instance
>>>
>>> a = get_instance()
>>> a.x = 1
>>> a.foo = 2
>>>
>>> b = a.b
>>> b.z = 3
>>> b.bar = 4
>>>
It works because Python imports modules only once and then shares them between all other modules, so the changes you make to modules affect all code running along yours.
Suppose I have a class with __slots__
class A:
__slots__ = ['x']
a = A()
a.x = 1 # works fine
a.y = 1 # AttributeError (as expected)
Now I am going to change __slots__ of A.
A.__slots__.append('y')
print(A.__slots__) # ['x', 'y']
b = A()
b.x = 1 # OK
b.y = 1 # AttributeError (why?)
b was created after __slots__ of A had changed, so Python, in principle, could allocate memory for b.y. Why it didn't?
How to properly modify __slots__ of a class, so that new instances have the modified attributes?
You cannot dynamically alter the __slots__ attribute after creating the class, no. That's because the value is used to create special descriptors for each slot. From the __slots__ documentation:
__slots__ are implemented at the class level by creating descriptors (Implementing Descriptors) for each variable name. As a result, class attributes cannot be used to set default values for instance variables defined by __slots__; otherwise, the class attribute would overwrite the descriptor assignment.
You can see the descriptors in the class __dict__:
>>> class A:
... __slots__ = ['x']
...
>>> A.__dict__
mappingproxy({'__module__': '__main__', '__doc__': None, 'x': <member 'x' of 'A' objects>, '__slots__': ['x']})
>>> A.__dict__['x']
<member 'x' of 'A' objects>
>>> a = A()
>>> A.__dict__['x'].__get__(a, A)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: x
>>> A.__dict__['x'].__set__(a, 'foobar')
>>> A.__dict__['x'].__get__(a, A)
'foobar'
>>> a.x
'foobar'
You cannot yourself create these additional descriptors. Even if you could, you cannot allocate more memory space for the extra slot references on the instances produced for this class, as that's information stored in the C struct for the class, and not in a manner accessible to Python code.
That's all because __slots__ is only an extension of the low-level handling of the elements that make up Python instances to Python code; the __dict__ and __weakref__ attributes on regular Python instances were always implemented as slots:
>>> class Regular: pass
...
>>> Regular.__dict__['__dict__']
<attribute '__dict__' of 'Regular' objects>
>>> Regular.__dict__['__weakref__']
<attribute '__weakref__' of 'Regular' objects>
>>> r = Regular()
>>> Regular.__dict__['__dict__'].__get__(r, Regular) is r.__dict__
True
All the Python developers did here was extend the system to add a few more of such slots using arbitrary names, with those names taken from the __slots__ attribute on the class being created, so that you can save memory; dictionaries take more memory than simple references to values in slots do. By specifying __slots__ you disable the __dict__ and __weakref__ slots, unless you explicitly include those in the __slots__ sequence.
The only way to extend slots then is to subclass; you can dynamically create a subclass with the type() function or by using a factory function:
def extra_slots_subclass(base, *slots):
class ExtraSlots(base):
__slots__ = slots
ExtraSlots.__name__ = base.__name__
return ExtraSlots
It appears to me a type turns __slots__ into a tuple as one of it's first orders of action. It then stores the tuple on the extended type object. Since beneath it all, the python is looking at a tuple, there is no way to mutate it. Indeed, I'm not even sure you can access it unless you pass a tuple in to the instance in the first place.
The fact that the original object that you set still remains as an attribute on the type is (perhaps) just a convenience for introspection.
You can't modify __slots__ and expect to have that show up somewhere (and really -- from a readability perspective, You probably don't really want to do that anyway, right?)...
Of course, you can always subclass to extend the slots:
>>> class C(A):
... __slots__ = ['z']
...
>>> c = C()
>>> c.x = 1
>>> c.z = 1
You cannot modify the __slots__ attribute after class creation. This is because it would leade to strange behaviour.
Imagine the following.
class A:
__slots__ = ["x"]
a = A()
A.__slots__.append("y")
a.y = None
What should happen in this scenario? No space was originally allocated for a second slot, but according to the slots attribute, a should be able have space for y.
__slots__ is not about protecting what names can and cannot be accessed. Rather __slots__ is about reducing the memory footprint of an object. By attempting to modify __slots__ you would defeat the optimisations that __slots__ is meant to achieve.
How __slots__ reduces memory footprint
Normally, an object's attributes are stored in a dict, which requires a fair bit of memory itself. If you are creating millions of objects then the space required by these dicts becomes prohibitive. __slots__ informs the python machinery that makes the class object that there will only be so many attributes refered to by instances of this class and what the names of the attributes will be. Therefore, the class can make an optimisation by storing the attributes directly on the instance rather than in a dict. It places the memory for the (pointers to the) attributes directly on the object, rather than creating a new dict for the object.
Putting answers to this and related question together, I want to make an accent on a solution to this problem:
You can kind of modify __slots__ by creating a subclass with the same name and then replacing parent class with its child. Note that you can do this for classes declared and used in any module, not just yours!
Consider the following module which declares some classes:
module.py:
class A(object):
# some class a user should import
__slots__ = ('x', 'b')
def __init__(self):
self.b = B()
class B(object):
# let's suppose we can't use it directly,
# it's returned as a part of another class
__slots__ = ('z',)
Here's how you can add attributes to these classes:
>>> import module
>>> from module import A
>>>
>>> # for classes imported into your module:
>>> A = type('A', (A,), {'__slots__': ('foo',)})
>>> # for classes which will be instantiated by the `module` itself:
>>> module.B = type('B', (module.B,), {'__slots__': ('bar',)})
>>>
>>> a = A()
>>> a.x = 1
>>> a.foo = 2
>>>
>>> b = a.b
>>> b.z = 3
>>> b.bar = 4
>>>
But what if you receive class instances from some third-party module using the module?
module_3rd_party.py:
from module import A
def get_instance():
return A()
No problem, it will also work! The only difference is that you may need to patch them before you import third-party module (in case it imports classes from the module):
>>> import module
>>>
>>> module.A = type('A', (module.A,), {'__slots__': ('foo',)})
>>> module.B = type('B', (module.B,), {'__slots__': ('bar',)})
>>>
>>> # note that we import `module_3rd_party` AFTER we patch the `module`
>>> from module_3rd_party import get_instance
>>>
>>> a = get_instance()
>>> a.x = 1
>>> a.foo = 2
>>>
>>> b = a.b
>>> b.z = 3
>>> b.bar = 4
>>>
It works because Python imports modules only once and then shares them between all other modules, so the changes you make to modules affect all code running along yours.
I am learning a tutorial on python.It is explaining how functions are first class objects in Python.
def foo():
pass
print(foo.__class__)
print(issubclass(foo.__class__,object))
The output that I get for the above code is
<type 'function'>
True
This program is supposed to demonstrate that functions are first class objects in python? My questions are as follows.
How does the above code prove that functions are fist class objects?
What are the attributes of a first class object?
what does function.__class__ signify? It returns a tuple <type,function> which doesn't mean much?
Here's what Guido says about first class objects in his blog:
One of my goals for Python was to make it so that all objects were "first class." By this, I meant that I wanted all objects that could be named in the language (e.g., integers, strings, functions, classes, modules, methods, etc.) to have equal status. That is, they can be assigned to variables, placed in lists, stored in dictionaries, passed as arguments, and so forth.
The whole blog post is worth reading.
In the example you posted, the tutorial may be making the point that first class objects are generally descendents of the "object" class.
First-class simply means that functions can be treated as a value -- that is you can assign them to variables, return them from functions, as well as pass them in as a parameter. That is you can do code like:
>>> def say_hi():
print "hi"
>>> def say_bye():
print "bye"
>>> f = say_hi
>>> f()
hi
>>> f = say_bye
>>> f()
bye
This is useful as you can now assign functions to variables like any ordinary variable:
>>> for f in (say_hi, say_bye):
f()
hi
bye
Or write higher order functions (that take functions as parameters):
>>> def call_func_n_times(f, n):
for i in range(n):
f()
>>> call_func_n_times(say_hi, 3)
hi
hi
hi
>>> call_func_n_times(say_bye, 2)
bye
bye
About __class__ in python tells what type of object you have. E.g., if you define an list object in python: a = [1,2,3], then a.__class__ will be <type 'list'>. If you have a datetime (from datetime import datetime and then d = datetime.now(), then the type of d instance will be <type 'datetime.datetime'>. They were just showing that in python a function is not a brand new concept. It's just an ordinary object of <type 'function'>.
You proved that functions are first class objects because you were allowed to pass foo as an argument to a method.
The attributes of first class objects was nicely summarised in this post: https://stackoverflow.com/a/245208/3248346
Depending on the language, this can
imply:
being expressible as an anonymous literal value
being storable in variables
being storable in data structures
having an intrinsic identity (independent of any given name)
being comparable for equality with other entities
being passable as a parameter to a procedure/function
being returnable as the result of a procedure/function
being constructible at runtime
being printable
being readable
being transmissible among distributed processes
being storable outside running processes
Regarding your third question, <type 'function'> isn't a tuple. Python's tuple notation is (a,b), not angle brackets.
foo.__class__ returns a class object, that is, an object which represents the class to which foo belongs; class objects happen to produce descriptive strings in the interpreter, in this case telling you that the class of foo is the type called 'function'. (Classes and types are basically the same in modern Python.)
It doesn't mean a whole lot other than that, like any other object, functions have a type:
>>> x = 1
>>> x.__class__
<type 'int'>
>>> y = "bar"
>>> y.__class__
<type 'str'>
>>> def foo(): pass
...
>>> foo.__class__
<type 'function'>
Regarding your comment to #I.K.s answer, f_at_2() in the following would be the method.
def f_at_2(f):
return f(2)
def foo(n):
return n ** n
def bar(n):
return n * n
def baz(n):
return n / 2
funcs = [foo, bar, baz]
for f in funcs:
print f.func_name, f_at_2(f)
...
>>>
foo 4
bar 4
baz 1
>>>
A method is a function of/in a class, but the concept also applies to a function (outside of a class). The functions (as objects) are contained in a data structure and passed to another object.
I have a utility method that behaves like this
def my_patch_method(self):
pass
def patch_my_lib():
from mylib import MyClass
MyClass.target_method = my_patch_method
return MyClass()
This test fails:
self.assertEqual(my_patch_method, patch_my_lib().target_method)
Whereas this one works:
self.assertEqual(my_patch_method.__name__, patch_my_lib().target_method.__name__)
As the patch method does not have the same name, this is still acceptable proof that patch_my_lib() is doing what it's payed for but why doesn't the first work as I would expect ? And, is there a way to "fix" it ?
The reason your first test fails is that once you monkey-patch the function into your class, it isn't really the same object any more.
>>> def foo(self): pass
...
>>> class Foo: pass
...
>>> Foo.bar = foo
>>> type(Foo.bar)
<type 'instancemethod'>
>>> type(foo)
<type 'function'>
>>>
>>> Foo.bar is foo
False
>>> Foo.bar == foo
False
In fact, the original function and the new method have different types. Instead, have your first test check this condition:
>>> Foo.bar.im_func is foo
True
So maybe this: self.assertIs(my_patch_method, patch_my_lib().target_method.im_func)
Try:
self.assertEqual(my_patch_method, patch_my_lib().target_method.im_func)
You are returning an instance from patch_my_lib, so comparing the function to the bound method
Something like this should pass
self.assertEqual(my_patch_method, patch_my_lib().target_method.im_func)
But it's probably better to check that the behaviour you are patching is working
MyClass.target_method = my_patch_method sets your function as a class function for MyClass, but you return a instance of that class with return MyClass().