Let's say I have a class like this:
class Test(object):
prop = property(lambda self: "property")
The descriptor takes priority whenever I try to access Test().prop. So that will return 'property'. If I want to access the object's instance storage, I can do:
x = Test()
x.__dict__["prop"] = 12
print(x.__dict__["prop"])
However if I change my class to:
class Test(object):
__slots__ = ("prop",)
prop = property(lambda self: "property")
How do I do the same, and access the internal storage of x, to write 12 and read it back, since x.__dict__ no longer exist?
I am fairly new with Python, but I understand the Python philosophy is to give complete control, so why is an implementation detail preventing me from doing that?
Isn't Python missing a built-in function that could read from an instance internal storage, something like:
instance_vars(x)["prop"] = 12
print(instance_vars(x)["prop"])
which would work like vars, except it also works with __slots__, and with built-in types that don't have a __dict__?
Short answer, You can't
The problem is that slots are themselves implemented in terms of descriptors. Given:
class Test(object):
__slots__ = ("prop",)
t = Test()
the phrase:
t.prop
Is translated, approximately to:
Test.prop.__get__(t, Test)
where Test.prop is a <type 'member_descriptor'> crafted by the run-time specifically to load prop values out of Test instances from their reserved space.
If you add another descriptor to the class body definition, it masks out the member_descriptor that would let you get to the slotted attribute; there's no way to ask for it, it's just not there anymore. It's effectively like saying:
class Test(object):
#property
def prop(self):
return self.__dict__['prop']
#property
def prop(self):
return "property"
You've defined it twice. there's no way to "get at" the first prop definition.
but:
Long answer, you can't in a general way. You can
You can still abuse the python type system to get at it using another class definition. You can change the type of a python object, so long as it has the exact same class layout, which roughly means that it has all of the same slots:
>>> class Test1(object):
... __slots__ = ["prop"]
... prop = property(lambda self: "property")
...
>>> class Test2(object):
... __slots__ = ["prop"]
...
>>> t = Test1()
>>> t.prop
'property'
>>> t.__class__ = Test2
>>> t.prop = 5
>>> t.prop
5
>>> t.__class__ = Test1
>>> t.prop
'property'
But there's no general way to introspect an instance to work out its class layout; you just have to know from context. You could look at it's __slots__ class attribute, but that won't tell you about the slots provided in the superclass (if any) nor will it give you any hint if that attribute has changed for some reason after the class was defined.
I don't quite understand what and why you want to do this, but does this help you?
>>> class Test(object):
__slots__ = ("prop",)
prop = property(lambda self: "property")
>>> a = Test()
>>> b = Test()
>>> a.prop
'property'
>>> tmp = Test.prop
>>> Test.prop = 23
>>> a.prop
23
>>> Test.prop = tmp; del tmp
>>> b.prop
'property'
of course, you cannot overwrite the property on a per-instance basis, that's the whole point of slotted descriptors.
Note that subclasses of a class with __slots__ do have a __dict__ unless you manually define __slots__, so you can do:
>>> class Test2(Test):pass
>>> t = Test2()
>>> t.prop
'property'
>>> t.__dict__['prop'] = 5
>>> t.__dict__['prop']
5
>>> Test2.prop
<property object at 0x00000000032C4278>
but still:
>>> t.prop
'property'
and that's not because of __slots__, it's the way descriptors work.
your __dict__ is bypassed on attribute lookup, you are just abusing it as data structure that happens to be there for storing a state.
it is equivalent to do this:
>>> class Test(object):
__slots__ = ("prop", "state")
prop = property(lambda self: "property")
state = {"prop": prop}
>>> t.prop
'property'
>>> t.state["prop"] = 5
>>> t.state["prop"]
5
>>> t.prop
'property'
If you really ever want to do something like that, and you REALL REALLY need something like that, you can always override __getattribute__ and __setattribute__, it's just as stupid... This is just to prove it to you:
class Test(object):
__slots__ = ("prop",)
prop = property(lambda self: "property")
__internal__ = {}
def __getattribute__(self, k):
if k == "__dict__":
return self.__internal__
else:
try:
return object.__getattribute__(self, k)
except AttributeError, e:
try:
return self.__internal__[k]
except KeyError:
raise e
def __setattribute__(self, k, v):
self.__internal__[k] = v
object.__setattribute__(self, k, v)
t = Test()
print t.prop
t.__dict__["prop"] = "test"
print "from dict", t.__dict__["prop"]
print "from getattr", t.prop
import traceback
# These won't work: raise AttributeError
try:
t.prop2 = "something"
except AttributeError:
print "see? I told you!"
traceback.print_exc()
try:
print t.prop2
except AttributeError:
print "Haha! Again!"
traceback.print_exc()
(Tried it on Python 2.7)
It's exactly what you expect I guess. Don't do this, it's useless.
Related
This is my first question here and also my first project in Python.
I'm trying to store instances of a class called Ip500Device:
class Ip500Device(object):
list = []
def __init__(self, shortMac, mac, status, deviceType):
self.__shortMac =shortMac
self.__mac=mac
self.__status=status
self.__deviceType=deviceType
self.__nbOfObjects=0
Ip500Device.list.append(self)
def __getattribute__(self, att):
if att=='hello':
return 0
This first test is just a 'hello', but after that I want to get all the attributes.
From an other class, I'm creating devices object and adding them to a list:
self.__ip500DevicesLst.append(Ip500Device.Ip500Device(lst[0],lst[1],lst[2],lst[3]))
for abcd in self.__ip500DevicesLst:
print abcd.__getattribute__('hello')
But when I try to print, the program returns this message:
TypeError: 'NoneType' object is not callable
I don't understand really well how to store class instances in Python.
print abcd.__getattribute__('hello')
abcd.__getattribute__ is not the __getattribute__ method. When you try to evaluate abcd.__getattribute__, you're actually calling
type(abcd).__getattribute__(abcd, '__getattribute__')
which returns None, which you then try to call as if it were a method.
The error happens because __getattribute__ is called for all attributes, and you have defined it to return None for everything other than "hello". Since __getattribute__ is itself an attribute, when you try to call it you will get a TypeError.
This problem can be fixed by calling the base-class method for unhandled attributes:
>>> class Ip500Device(object):
... def __getattribute__(self, att):
... print('getattribute: %r' % att)
... if att == 'hello':
... return 0
... return super(Ip500Device, self).__getattribute__(att)
...
>>> abcd = Ip500Device()
>>> abcd.__getattribute__('hello')
getattribute: '__getattribute__'
getattribute: 'hello'
0
However, it is better to define __getattr__, since that is only called for attributes which don't already exist:
>>> class Ip500Device(object):
... def __getattr__(self, att):
... print('getattr: %r' % att)
... if att == 'hello':
... return 0
... raise AttributeError(att)
...
>>> abcd = Ip500Device()
>>> abcd.hello
getattr: 'hello'
0
>>> abcd.foo = 10
>>> abcd.foo
10
Finally, note that if all you want to do is access attributes by name, you can use the built-in getattr function:
>>> class Ip500Device(object): pass
...
>>> abcd = Ip500Device()
>>> abcd.foo = 10
>>> getattr(abcd, 'foo')
10
I know the questions about: copy properties, or dynamic creation of properties has already been posted and also been answered (here, here and here). You could also find an excellent description, how the property function works here.
But I think, that my question is a bit more specific. I do not only want to copy the property from one class to another. No, I also want the specific getter, setter and deleter functions to be copied to the destination class. After a whole day of searching for an answer, I decided to create an new post for this question.
So let me get a bit more in detail. A have an attribute class which is more a class group and stores property-classes:
class AttrContainer():
class a():
ATTR=1
#property
def a(self):
return self.ATTR
#a.setter
def a(self, n):
self.ATTR = n + 3.021
class b():
ATTR=None
#property
def b(self):
return "Something"
class c():
ATTR=None
#property
def c(self):
return 3
#c.setter
def c(self, n):
self.ATTR = n - 8.5201
As you can see, I have different getter, setter (not in the example: deleter) definitions of each property.
I want to use those properties with my item "wrapper" objects. But not all of item objects needs all properties, thats why I want to copy them dynamically into my wrapper classes.
So, this is how my item "wrapper" classes looks like:
class Item01Object():
properties = ["a","c"]
ATTR = None
#[...]
class Item02Object():
properties = ["b","c"]
ATTR = None
#[...]
#[...]
Because I can't set the properties dynamically while the item class will be instanced, I have to set them before I instance the class:
def SetProperties( ItemObject ):
for propName, cls in AttrContainer.__dict__.iteritems():
if propName in ItemObject.properties:
prop = cls.__dict__[propName]
fget = prop.fget if prop.fget else None
fset = prop.fset if prop.fset else None
fdel = prop.fdel if prop.fdel else None
ItemObject.__dict__[propName] = property(fget,fset,fdel)
return ItemObject()
In the end, i would instance my ItemObjects like this:
item = SetProperties(Item01Object)
I would expect, that this will work...
>>> print item
<__builtin__.Item01Object instance at 0x0000000003270F88>
>>> print item.a
None
This is result is right, because I do not update my property ATTR..
Lets change the property:
>>> item.a = 20
>>> print item.a
20
But this result is wrong, it should be 23.021 and NOT 20 . It looks like my properties do not using the setter functions from its classes.
Why? What do I wrong in my code?
Edit: Sorry, I forgot to remove the inherited object of the ItemObject classes.. Now the code works.
For properties with setters and deleters to work properly, your classes need to inherit from object: Why does #foo.setter in Python not work for me?
You can just copy the property object itself over to the new class. It'll hold references to the getter, setter and deleter functions and there is no need to copy those across.
For new-style classes, your code is not working; you cannot assign to a class __dict__ attribute:
>>> item = SetProperties(Item01Object)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in SetProperties
TypeError: 'dictproxy' object does not support item assignment
Use setattr() instead to set attributes on new-style classes:
def SetProperties( ItemObject ):
for propName, cls in AttrContainer.__dict__.iteritems():
if propName in ItemObject.properties:
setattr(ItemObject, propName, cls.__dict__[propName])
return ItemObject()
Note that the property object is copied across wholesale.
Demo:
>>> class Item01Object(object):
... properties = ["a","c"]
... ATTR = None
...
>>> def SetProperties( ItemObject ):
... for propName, cls in AttrContainer.__dict__.iteritems():
... if propName in ItemObject.properties:
... setattr(ItemObject, propName, cls.__dict__[propName])
... return ItemObject()
...
>>> item = SetProperties(Item01Object)
>>> item
<__main__.Item01Object object at 0x108205850>
>>> item.a
>>> item.a = 20
>>> item.a
23.021
You only have to copy across property objects to the target class once though; that your function returns an instance implies you are planning to use it for all instances created.
I'd make it a decorator instead:
def set_properties(cls):
for name, propcls in vars(AttrContainer).iteritems():
if name in cls.properties:
setattr(cls, name, vars(propcls)[name])
return cls
then use this on each of your Item*Object classes:
#set_properties
class Item01Object(object):
properties = ["a","c"]
ATTR = None
#set_properties
class Item02Object(object):
properties = ["b","c"]
ATTR = None
Demo:
>>> def set_properties(cls):
... for name, propcls in vars(AttrContainer).iteritems():
... if name in cls.properties:
... setattr(cls, name, vars(propcls)[name])
... return cls
...
>>> #set_properties
... class Item01Object(object):
... properties = ["a","c"]
... ATTR = None
...
>>> #set_properties
... class Item02Object(object):
... properties = ["b","c"]
... ATTR = None
...
>>> item01 = Item01Object()
>>> item01.c = 20
>>> item01.c
3
>>> item02 = Item02Object()
>>> item02.b = 42
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: can't set attribute
>>> item02.b
'Something'
So I don't come from a computer science background and I am having trouble googling/SO searching on the right terms to answer this question. If I have a Python class with a class variable objects like so:
class MyClass(object):
objects = None
pass
MyClass.objects = 'test'
print MyClass.objects # outputs 'test'
a = MyClass()
print a.objects # also outputs 'test'
both the class and instances of the class will have access to the objects variable. I understand that I can change the instance value like so:
a.objects = 'bar'
print a.objects # outputs 'bar'
print MyClass.objects # outputs 'test'
but is it possible to have a class variable in Python that is accessible to users of the class (i.e. not just from within the class) but not accessible to the instances of that class? I think this is called a private member or static member in other languages?
Python is designed to allow instances of a class to access that class's attributes through the instance.
This only goes one level deep, so you can use a metaclass:
class T(type):
x = 5
class A(object):
__metaclass__ = T
Note that the metaclass syntax is different in Python 3. This works:
>>> A.x
5
>>> A().x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'A' object has no attribute 'x'
It doesn't prevent you setting the attribute on instances of the class, though; to prevent that you'd have to play with __setattr__ magic method:
class A(object):
x = 1
def __getattribute__(self, name):
if name == 'x':
raise AttributeError
return super(A, self).__getattribute__(name)
def __setattr__(self, name, value):
if name == 'x':
raise AttributeError
return super(A, self).__setattr__(name, value)
def __delattr__(self, name):
if name == 'x':
raise AttributeError
return super(A, self).__delattr__(name)
The simplest way of achieving it is to use a descriptor. Descriptors are the thing meant for giving a higher level of control over attribute access. For example:
class ClassOnly(object):
def __init__(self, name, value):
self.name = name
self.value = value
def __get__(self, inst, cls):
if inst is not None:
msg = 'Cannot access class attribute {} from an instance'.format(self.name)
raise AttributeError(msg)
return self.value
class A(object):
objects = ClassOnly('objects', [])
Used as:
In [11]: a = A()
In [12]: a.objects
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-12-24afc67fd0ba> in <module>()
----> 1 a.objects
<ipython-input-9-db6510cd313b> in __get__(self, inst, cls)
5 def __get__(self, inst, cls):
6 if inst is not None:
----> 7 raise AttributeError('Cannot access class attribute {} from an instance'.format(self.name))
8 return self.value
AttributeError: Cannot access class attribute objects from an instance
In [13]: A.objects
Out[13]: []
If you want there to be a "single source of truth" for objects, you could make it a mutable type:
class MyClass(object):
objects = []
With immutable types, the fact that each instance starts out with the same reference from MyClass is irrelevant, as the first time that attribute is changed for the instance, it becomes "disconnected" from the class's value.
However, if the attribute is mutable, changing it in an instance changes it for the class and all other instances of the class:
>>> MyClass.objects.append(1)
>>> MyClass.objects
[1]
>>> a = MyClass()
>>> a.objects
[1]
>>> a.objects.append(2)
>>> a.objects
[1, 2]
>>> MyClass.objects
[1, 2]
In Python, nothing is really "private", so you can't really prevent the instances from accessing or altering objects (in that case, is it an appropriate class attribute?), but it is conventional to prepend names with an underscore if you don't ordinarily want them to be accessed directly: _objects.
One way to actually protect objects from instance access would be to override __getattribute__:
def __getattribute__(self, name):
if name == "objects":
raise AttributeError("Do not access 'objects' though MyClass instances.")
return super(MyClass, self).__getattribute__(name)
>>> MyClass.objects
[1]
>>> a.objects
...
AttributeError: Do not access 'objects' though MyClass instances.
No, you can't (EDIT: you can't in a way that is completely unaccessible, like in Java or C++).
You can do this, if you like:
class MyClass(object):
objects = None
pass
MyClass_objects = 'test'
print MyClass_objects # outputs 'test'
a = MyClass()
print a.objects # outputs 'None'
or this:
in your_module.py:
objects = 'test'
class MyClass(object):
objects = None
pass
in yourapp.py:
import your_module
print your_module.objects # outputs 'test'
a = your_module.MyClass()
print a.objects # outputs 'None'
the reason is:
When you create an instance of some class there is nothing to prevent
you from poking around inside and using various internal, private
methods that are (a) necessary for the class to function, BUT (b) not
intended for direct use/access.
Nothing is really private in python. No class or class instance can
keep you away from all what's inside (this makes introspection
possible and powerful). Python trusts you. It says "hey, if you want
to go poking around in dark places, I'm gonna trust that you've got a
good reason and you're not making trouble."
Karl Fast
I understood python descriptor but I have a little confusion about this..
if you have a class descriptor as follows
class Descriptor(object):
def __get__(self, instance, owner):
print 'getting'
return self.value
def __set__(self, instance, value):
print 'setting'
self.value = value
def __delete__(self, instance):
print 'deleting'
del self.value
and a class whose attributes we want to manage is something like this..
class Test(object):
name = Descriptor()
def __init__(self, name):
print 'init test'
self.name = name
when I create object of class Test and do something it gives me answer like this...
t = Test('abc')
init test
setting
>>> t.name
getting
'abc'
>>> del t.name
deleting
>>> t
<__main__.Test object at 0x013FCCD0>
>>> t.name
getting
Now I want to have a class Test1 something like this..
class Test1(object):
def __init__(self, value):
print 'init test1'
self.name = Descriptor()
self. value = value
and if I create object of Test1 and try to access attribute of instance of Test1, I get output something like this..
t1 = Test1(12)
t1.name
>>> getting
>>> 12
>>> t1.name = 30
>>> setting
Q 1) my question is that is this name attribute declared in init of Test1, is bound to instance of Test1 or not... because when I try to get attribute dictionary of t1, it return empty dict...
t1.__dict__
>>> {}
same for class Test's instance t
t.__dict__
>>> {}
When I add a new attribute to any of these instances, like this...
t.some = 'some'
>>> t1.some = 'some'
and again if I try to access attribute dictionary it gives me only which I have added just now.. now all instance attribute
t.__dict__
>>> {'some': 'some'}
>>> t1.__dict__
>>> {'some': 'some'}
Q 2) So what is the difference between instance attributes defined in init (like variable name and value in class Descriptor and Test) and attributes defined after instance creation (like variable t.some).
Q 3) How class Test is different than class Test1.
In Test1 your Descriptor isn't really used as a descriptor, it's just a normal attribute called name, that happens to have some the special methods. But that doensn't really make it a descriptor yet.
If you read the docs about how descriptors are invoked, youll see the mechanism that is used to invoke the descriptors methods. In your case this would mean t.name woud be roughly equivalent to:
type(t).__dict__['name'].__get__(t, type(t))
and t1.name:
type(t1).__dict__['name'].__get__(t1, type(t1))
name is looked up in the __dict__ of the class, not of the instance, so that's where the difference is, Test1.__dict__ doesn't have a descriptor called name:
>>> Test.__dict__['name']
<__main__.Descriptor object at 0x7f637a57bc90>
>>> Test1.__dict__['name']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'name'
What you also should consider, is that your descriptor sets the value attribute on itself, that means all instances of Test will share the same value:
>>> t1 = Test(1)
init test
setting
>>> t2 = Test(2)
init test
setting
>>> t1.name
getting
2
>>> t2.name
getting
2
>>> t1.name = 0
setting
>>> t2.name
getting
0
I think that what yo acutally want to do is to set value on instance instead of self, that would get you the expected behaviour in Test.
I have a two part question.
>>> class One(object):
... pass
...
>>> class Two(object):
... pass
...
>>> def digest(constr):
... c = apply(constr)
... print c.__class__.__name__
... print constr.__class__.__name__
...
>>> digest(Two)
Two
type
How would one create object 'Two'? Neither constr() or c() work; and it seems that apply turns it into a type.
What happens when you pass a class rather and an instance into a method?
Classes are high level objects, so you can simply pass them like this:
def createMyClass ( myClass ):
obj = myClass()
return obj
class A ( object ):
pass
>>> x = createMyClass( A )
>>> type( x )
<class '__main__.A'>
How would one create object 'Two'?
Neither constr() or c() work; and it
seems that apply turns it into a
type.
The above comment was made in regards to this code:
>>> def digest(constr):
... c = apply(constr)
... print c.__class__.__name__
... print constr.__class__.__name__
apply (deprecated: see #pyfunc's answer) certainly does not turn the class Two into a type: It already is one.
>>> class Two(object): pass
...
>>> type(Two)
<type 'type'>
Classes are first class objects: they're instances of type. This makes sense if you look at the next example.
>>> two = Two()
>>> type(two)
<class '__main__.Two'>
You can see that a class very clearly functions as a type because it can be returned from type. Here's another example.
>>> Three = type('Three', (Two, ), {'foo': 'bar'})
>>> type(Three)
<type 'type'>
>>> three = Three()
>>> type(three)
<class '__main__.Three'>
You can see that type is a class that can be instantiated. Its constructor takes three arguments: the name of the class, a tuple of base classes and a dictionary containing the class attributes. It returns a new type aka class.
As to your final question,
What happens when you pass a class
rather and an instance into a method?
You're going to have to be more specific. Classes are just instances of type and so are first class objects. Asking what happens if I pass a class into a method is like asking what happens if I pass an integer into a method: It depends entirely on what the method is expecting.
Just another one example:
def InstanceFactory(classname):
cls = globals()[classname]
return cls()
class A(object):
def start(self):
print "a.start"
class B(object):
def start(self):
print "b.start"
InstanceFactory("A").start()
InstanceFactory("B").start()
If the class belongs to another module:
def InstanceFactory(modulename, classname):
if '.' in modulename:
raise ValueError, "can't handle dotted modules yet"
mod = __import__(modulename)
cls = getattr(mod, classname]
return cls()
I am confused though. Wasn't apply() deprecated since 2.3
http://www.python.org/dev/peps/pep-0290/
We don't need this any more.
apply(f, args, kwds) --> f(*args, **kwds)
Others have been moved / considered deprecated in modern usage:
buffer()
coerce()
and intern()
Simply use : Classname() to create an object.