I use #property to ensure that changes to an objects instance variables are wrapped by methods where I need to.
What about when an instance has an variable that logically should not be changed? Eg, if I'm making a class for a Process, each Process instance should have a PID attribute that will frequently be accessed but should not be changed.
What's the most Pythonic way to handle someone attempting to modify that instance variable?
Simply trust the user not to try and change
something they shouldn't?
Use property but raise an
exception if the instance variable is
changed?
Something else?
Prepend name of the variable with __, and create read-only property, Python will take care of exceptions, and variable itself will be protected from accidental overwrite.
class foo(object):
def __init__(self, bar):
self.__bar = bar
#property
def bar(self):
return self.__bar
f = foo('bar')
f.bar # => bar
f.bar = 'baz' # AttributeError; would have to use f._foo__bar
Simply trusting the user is not necessarily a bad thing; if you are just writing a quick Python program to be used once and thrown away, you might very well just trust that the user not alter the pid field.
IMHO the most Pythonic way to enforce the read-only field is to use a property that raises an exception on an attempt to set the field.
So, IMHO you have good instincts about this stuff.
Maybe you can override __setattr__? In the line of,
def __setattr__(self, name, value):
if self.__dict__.has_key(name):
raise TypeError, 'value is read only'
self.__dict__[name] = value
Simply use a property and a hidden attribute (prefixed with one underscore).
Simple properties are read-only!
>>> class Test (object):
... #property
... def bar(self):
... return self._bar
...
>>> t = Test()
>>> t._bar = 2
>>> t.bar
2
>>> t.bar = 2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: can't set attribute
Hiding with double underscore is not used to hide the implementation, but to make sure you don't collide with a subclass' attributes; consider a mixin for example, it has to be very careful!
If you just want to hide the attribute, use a single underscore. And as you see there is no extra magic to add -- if you don't define a set function, your property is just as read-only as the return value of a method.
Related
The snippet:
class Base:
def superclass_only(self):
return 'yes'
class Foo(Base):
pass
foo = Foo()
>>> foo.superclass_only()
yes
# Expection is to raise error
>>> foo.superclass_only()
traceback
...
AttributeError: 'Foo' object has no attribute 'superclass_only'
How can I do if I just want to define a superclass-only method?
TL;DR: prefix the method name with __ to trigger Python's name mangling mechanism.
answer:
You normally can't do that: it is not how inheritance is supposed to work.
If you need to "hide away" methods in the "subclasses", you should rething your approach.
One first thing is to use the naming convention to indicate the method is private, which in Python we do by adding a "_" prefix to the method name: that should be an indicator to users of your Foo class that the reserved method should be used only by whoever writes the code in Base and be let alone.
Another thing is to think if you would not be better with composition than with inheritance in this case: if your Base class knows to do things that Foo can't do on itself, can you really say that "Foo objects are also Base objects"? (which is what inheritance is about).
Maybe, the better design is:
class Base:
...
class Bar:
def method_foo_cant_do(...):
...
class Foo(Base):
def __init__(self, ...):
self.bar = Bar()
...
And finally, although not designed for that, and rather meant to avoid method-name clashes in complex hierarchies, Python has a "name mangling" mechanism, which will transparently change a method name to one including the class name as prefix. This will avoid casual use of the method in subclasses, and be an even stronger indicator that it should be used in "Base" along - but won't "prevent at all costs" that it be called.
The way to go is simply prefix the method with two underscores. At compilation time, Python translates the method to be f"_{class_name}__{method_name}", at the method declaration and in all references to it inside the class where it is declared. So Foo.__superclass_only will not reach Base.__superclass_only since the later has had its name mangled to Base._Base__superclass_only:
class Base:
def __superclass_only(self):
return 'yes'
class Foo(Base):
pass
And on the interactive interpreter:
In [3]: f= Foo()
In [4]: f.__superclass_only()
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-4-19c8185aa9ad> in <module>
----> 1 f.__superclass_only()
But it is still reachable by using the transformed name: f._Base__superclass_only() would work.
Another thing that Python allows is to customize the way attributes are retrieved for a given class: the somewhat search for attributes and methods in a class is performed by the __getattribute__ method in each class (do not mistake it with __getattr__ which is simpler and designed to be hit only when an attribute is not found).
Reimplementing __getattribute__ is error prone and would likely leave you worse than the way you started with, and given a foo object, one would stil be able to call the superclass_only by doing
Base.superclass_only(foo, ...) (i.e.:retrieving the method as an unbound method (function) from the Base class itself and passing in the foo instance manually to become the "self" argument), and mitigating this would require you to implement a correct __get_attribute__ on the metaclass - (and that would still be ultimately bypassable by one who could read the source code)
You can wrap the superclass-only method with a decorator function that validates the current instance's class name against the method's class name, which can be derived from the method's __qualname__ attribute:
def superclass_only(method):
def wrapper(self, *args, **kwargs):
if self.__class__.__name__ != method.__qualname__.split('.')[-2]:
raise NotImplementedError
return method(self, *args, **kwargs)
return wrapper
so that with:
class Base:
#superclass_only
def method(self):
return 'yes'
class Foo(Base):
pass
Calling Base().method() returns 'yes', while calling Foo().method() raises NotImplementedError.
Demo: https://replit.com/#blhsing/SpringgreenHonorableCharacters
In Python, if you call a method that doesn't exist it throws an AttributeError. Ex
>>> class A:
... def yo(self):
... print(1)
...
>>> a = A()
>>> a.yo()
1
>>> a.hello()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'A' object has no attribute 'hello'
In the below code, MagicMock class doesn't have a function named hello or no patch was created for a method hello. Still below code doesn't throw an AttributeError
>>> from unittest.mock import MagicMock
>>> obj = MagicMock()
>>> obj.hello()
<MagicMock name='mock.hello()' id='4408758568'>
How is MagicMock able to do this? How can I create a class that can perform an action when any method (which might not be defined) is called on it?
The Python datamodel documents a hook, __getattr__, which shall be called when attribute access fails to resolve in the usual ways. Mocks use it to return a new mock instance - i.e. mocks define unknown attributes as factories.
Reproducing mock's implementation in a simpler way, you would just turn __getattr__ and __call__ into factory functions:
class M:
def __call__(self):
return M()
def __getattr__(self, name):
return M()
Example usage:
>>> mock = M()
>>> mock.potato
<__main__.M at 0xdeadbeef>
>>> mock.potato()
<__main__.M at 0xcafef00d>
How is MagicMock able to do this?
This part is not specific to MagicMock, an ordinary Mock will do the same (the "magic" in the name is just referring to additional features allowing better mocking of magic methods). MagicMock inherits such behavior from one of the base classes:
>>> MagicMock.mro()
[unittest.mock.MagicMock,
unittest.mock.MagicMixin,
unittest.mock.Mock,
unittest.mock.CallableMixin,
unittest.mock.NonCallableMock, # <--- this one overrides __getattr__!
unittest.mock.Base,
object]
How can I create a class that can perform an action when any method (which might not be defined) is called on it?
It depends if you want to be in front of, or behind, a normal attribute access. If you want to get in front, you should define __getattribute__, it's called unconditionally to implement attribute accesses before searching the class/instance namespaces. However, if you want to take a lower precedence to normal attributes (i.e. those living in the object __dict__) and to descriptors, then you should define __getattr__ as previously discussed.
I don't actually know how specifically MagicMock works (I've never used it, but I've Heard Good Things), but this part of the behaviour can be replicated (along with probably multiple other possible solutions) by hijacking __getattr__ in a way that it returns a callable that creates a new mock instance when called:
class MM:
def __init__(self, name=None):
# store a name, TODO: random id, etc.
self.name = name
def __repr__(self):
# make it pretty
if self.name:
r = f'<MM name={self.name}>'
else:
r = f'<MM>'
return r
def __getattr__(self, attrname):
# we want a factory for a mock instance with a name corresponding to attrname
def magicattr():
return MM(name=f"'mock.{attrname}()'")
return magicattr
When executed, we see the following:
>>> MM()
<MM>
>>> MM().hello()
<MM name='mock.hello()'>
I didn't go overboard with defining an id and whatnot, but the basic trick can be seen on the above stripped-down example.
The way the above works is that accessing .hello or any other attribute goes through our custom __getattr__ which gives us a chance to generate a fake (mocked) method on the fly, with whatever properties we want to. As I understand one of the many benefits of MagicMock is exactly that we don't have to worry about AttributeErrors being thrown by default, it just works.
Is there a way to create a class which instances respond to arbitrary method calls?
I know there is a the special method __getattr__(self, attr) which would be called when someone is trying to access an attribute of an instance. I am searching for something similar that enables me to intercept method calls, too. The desired behavior would look something like this:
class A(object):
def __methodintercept__(self, method, *args, **kwargs): # is there a special method like this??
print(str(method))
>>> a = A()
>>> a.foomatic()
foomatic
EDIT
The other suggested questions do not address my case: I do not want to wrap another class or change the metaclass of a second class or similar. I just want to have a class that responds to arbitrary method calls.
Thanks to jonrshape I now know that __getattr__(self, attr) will also be called when a method is called in the same way as it would be when an attribute is accessed. But how do i distinguish in __getattr__ if attr comes from a method call or an attribute access and how to get the parameters of a potential method call?
This is something I came up with, which will behave exactly as if the method exists.
First let's establish one thing: You cannot distinguish in __getattr__ if attr comes from a function call or an "attribute access", because a class method is an attribute of your class. So someone can access that method even if they don't intend to call it, as in:
class Test:
def method(self):
print "Hi, I am method"
>> t = Test()
>> t.method # just access the method "as an attribute"
<bound method Test.method of <__main__.Test instance at 0x10a970c68>>
>> t.method() # actually call the method
Hi, I am method
Therefore, the closest thing I could think of is this behavior:
Create a class A, such that:
When we try to access an attribute / method, which already exists in that class, act normal and just return the requested attribute / method.
When we try to access something that doesn't exist in the class definition, treat it as a class method and have 1 global handler for all such methods.
I will first write the class definition and then show how accessing a method that doesn't exist behaves exactly like accessing one that exists, whether you are just accessing it, or actually calling it.
Class definition:
class A(object):
def __init__(self):
self.x = 1 # set some attribute
def __getattr__(self,attr):
try:
return super(A, self).__getattr__(attr)
except AttributeError:
return self.__get_global_handler(attr)
def __get_global_handler(self, name):
# Do anything that you need to do before simulating the method call
handler = self.__global_handler
handler.im_func.func_name = name # Change the method's name
return handler
def __global_handler(self, *args, **kwargs):
# Do something with these arguments
print "I am an imaginary method with name %s" % self.__global_handler.im_func.func_name
print "My arguments are: " + str(args)
print "My keyword arguments are: " + str(kwargs)
def real_method(self, *args, **kwargs):
print "I am a method that you actually defined"
print "My name is %s" % self.real_method.im_func.func_name
print "My arguments are: " + str(args)
print "My keyword arguments are: " + str(kwargs)
I added the method real_method just so I have something that actually exists in the class to compare its behavior with that of an 'imaginary method'
Here's the result:
>> a = A()
>> # First let's try simple access (no method call)
>> a.real_method # The method that is actually defined in the class
<bound method A.real_method of <test.A object at 0x10a9784d0>>
>> a.imaginary_method # Some method that is not defined
<bound method A.imaginary_method of <test.A object at 0x10a9784d0>>
>> # Now let's try to call each of these methods
>> a.real_method(1, 2, x=3, y=4)
I am a method that you actually defined
My name is real_method
My arguments are: (1, 2)
My keyword arguments are: {'y': 4, 'x': 3}
>> a.imaginary_method(1, 2, x=3, y=4)
I am an imaginary method with name imaginary_method
My arguments are: (1, 2)
My keyword arguments are: {'y': 4, 'x': 3}
>> # Now let's try to access the x attribute, just to make sure that 'regular' attribute access works fine as well
>> a.x
1
unittest.mock.Mock does this by default.
from unittest.mock import Mock
a = Mock()
a.arbitrary_method() # No error
a.arbitrary_method.called # True
a.new_method
a.new_method.called # False
a.new_method("some", "args")
a.new_method.called # True
a.new_method.assert_called_with("some", "args") # No error
a.new_method_assert_called_with("other", "args") # AssertionError
This is the solution I was looking for when coming across this question:
class Wrapper:
def __init__(self):
self._inner = [] # or whatever type you want to wrap
def foo(self, x):
print(x)
def __getattr__(self, attr):
if attr in self.__class__.__dict__:
return getattr(self, attr)
else:
return getattr(self._inner, attr)
t = Test()
t.foo('abc') # prints 'abc'
t.append('x') # appends 'x' to t._inner
Criticisms very welcome. I wanted to add methods to the Browser class in the Splinter package, but it only exposes a function to return an instance, not the class itself. This approach permitted pseudo-inheritance, which meant I could declaratively decouple DOM code from website-specific code. (A better approach in hindsight might have been to use Selenium directly.)
Method calls aren't any different from attribute access. __getattr__() or __getattribute__() is the way to respond to arbitrary attribute requests.
You cannot know if the access comes from "just retrieval" or "method call".
It works like this: first, attribute retrieval, then, call on the retrieved object (in Python, call is just another operator: anything can be called and will throw an exception if it isn't callable). One doesn't, and shouldn't, know about the other (well, you can analyze the code up the call stack, but that's totally not the thing to do here).
One of the reasons is - functions are first-class objects in Python, i.e. a function (or, rather, a reference to it) is no different from any other data type: I can get the reference, save it and pass it around. I.e. there's completely no difference between requesting a data field and a method.
Elaborate on what you need this for for us to suggest a better solution.
E.g., if you need the "method" to be able to be called with different signatures, *args and **kwargs is the way to go.
The follow will respond to all undefined method calls:
class Mock:
def __init__(self, *args, **kwargs):
pass
def __getattr__(self, attr):
def func(*args, **kwargs):
pass
return func
Or just use unittest.mock.Mock.
What's the correct idiom for this please?
I want to define an object containing properties which can (optionally) be initialized from a dict (the dict comes from JSON; it may be incomplete). Later on I may modify the properties via setters.
There are actually 13+ properties, and I want to be able to use default getters and setters, but that doesn't seem to work for this case:
But I don't want to have to write explicit descriptors for all of prop1... propn
Also, I'd like to move the default assignments out of __init__() and into the accessors... but then I'd need expicit descriptors.
What's the most elegant solution? (other than move all the setter calls out of __init__() and into a method/classmethod _make()?)
[DELETED COMMENT The code for badprop using default descriptor was due to comment by a previous SO user, who gave the impression it gives you a default setter. But it doesn't - the setter is undefined and it necessarily throws AttributeError.]
class DubiousPropertyExample(object):
def __init__(self,dct=None):
self.prop1 = 'some default'
self.prop2 = 'other default'
#self.badprop = 'This throws AttributeError: can\'t set attribute'
if dct is None: dct = dict() # or use defaultdict
for prop,val in dct.items():
self.__setattr__(prop,val)
# How do I do default property descriptors? this is wrong
##property
#def badprop(self): pass
# Explicit descriptors for all properties - yukk
#property
def prop1(self): return self._prop1
#prop1.setter
def prop1(self,value): self._prop1 = value
#property
def prop2(self): return self._prop2
#prop2.setter
def prop2(self,value): self._prop2 = value
dub = DubiousPropertyExample({'prop2':'crashandburn'})
print dub.__dict__
# {'_prop2': 'crashandburn', '_prop1': 'some default'}
If you run this with line 5 self.badprop = ... uncommented, it fails:
self.badprop = 'This throws AttributeError: can\'t set attribute'
AttributeError: can't set attribute
[As ever, I read the SO posts on descriptors, implicit descriptors, calling them from init]
I think you're slightly misunderstanding how properties work. There is no "default setter". It throws an AttributeError on setting badprop not because it doesn't yet know that badprop is a property rather than a normal attribute (if that were the case it would just set the attribute with no error, because that's now normal attributes behave), but because you haven't provided a setter for badprop, only a getter.
Have a look at this:
>>> class Foo(object):
#property
def foo(self):
return self._foo
def __init__(self):
self._foo = 1
>>> f = Foo()
>>> f.foo = 2
Traceback (most recent call last):
File "<pyshell#12>", line 1, in <module>
f.foo = 2
AttributeError: can't set attribute
You can't set such an attribute even from outside of __init__, after the instance is constructed. If you just use #property, then what you have is a read-only property (effectively a method call that looks like an attribute read).
If all you're doing in your getters and setters is redirecting read/write access to an attribute of the same name but with an underscore prepended, then by far the simplest thing to do is get rid of the properties altogether and just use normal attributes. Python isn't Java (and even in Java I'm not convinced of the virtue of private fields with the obvious public getter/setter anyway). An attribute that is directly accessible to the outside world is a perfectly reasonable part of your "public" interface. If you later discover that you need to run some code whenever an attribute is read/written you can make it a property then without changing your interface (this is actually what descriptors were originally intended for, not so that we could start writing Java style getters/setters for every single attribute).
If you're actually doing something in the properties other than changing the name of the attribute, and you do want your attributes to be readonly, then your best bet is probably to treat the initialisation in __init__ as directly setting the underlying data attributes with the underscore prepended. Then your class can be straightforwardly initialised without AttributeErrors, and thereafter the properties will do their thing as the attributes are read.
If you're actually doing something in the properties other than changing the name of the attribute, and you want your attributes to be readable and writable, then you'll need to actually specify what happens when you get/set them. If each attribute has independent custom behaviour, then there is no more efficient way to do this than explicitly providing a getter and a setter for each attribute.
If you're running exactly the same (or very similar) code in every single getter/setter (and it's not just adding an underscore to the real attribute name), and that's why you object to writing them all out (rightly so!), then you may be better served by implementing some of __getattr__, __getattribute__, and __setattr__. These allow you to redirect attribute reading/writing to the same code each time (with the name of the attribute as a parameter), rather than to two functions for each attribute (getting/setting).
It seems like the easiest way to go about this is to just implement __getattr__ and __setattr__ such that they will access any key in your parsed JSON dict, which you should set as an instance member. Alternatively, you could call update() on self.__dict__ with your parsed JSON, but that's not really the best way to go about things, as it means your input dict could potentially trample members of your instance.
As to your setters and getters, you should only be creating them if they actually do something special other than directly set or retrieve the value in question. Python isn't Java (or C++ or anything else), you shouldn't try to mimic the private/set/get paradigm that is common in those languages.
I simply put the dict in the local scope and get/set there my properties.
class test(object):
def __init__(self,**kwargs):
self.kwargs = kwargs
#self.value = 20 asign from init is possible
#property
def value(self):
if self.kwargs.get('value') == None:
self.kwargs.update(value=0)#default
return self.kwargs.get('value')
#value.setter
def value(self,v):
print(v) #do something with v
self.kwargs.update(value=v)
x = test()
print(x.value)
x.value = 10
x.value = 5
Output
0
10
5
the next is my code:
class foo:
def __init__(self):
self.a = "a"
def __getattr__(self,x,defalut):
if x in self:
return x
else:return defalut
a=foo()
print getattr(a,'b','sss')
i know the __getattr__ must be 2 argument,but i want to get a default attribute if the attribute is no being.
how can i get it, thanks
and
i found if defined __setattr__,my next code is also can't run
class foo:
def __init__(self):
self.a={}
def __setattr__(self,name,value):
self.a[name]=value
a=foo()#error ,why
hi alex,
i changed your example:
class foo(object):
def __init__(self):
self.a = {'a': 'boh'}
def __getattr__(self, x):
if x in self.a:
return self.a[x]
raise AttributeError
a=foo()
print getattr(a,'a','sss')
it print {'a': 'boh'},not 'boh'
i think it will print self.a not self.a['a'], This is obviously not want to see
why ,and Is there any way to avoid it
Your problem number one: you're defining an old-style class (we know you're on Python 2.something, even though you don't tell us, because you're using print as a keyword;-). In Python 2:
class foo:
means you're defining an old-style, aka legacy, class, whose behavior can be rather quirky at times. Never do that -- there's no good reason! The old-style classes exist only for compatibility with old legacy code that relies on their quirks (and were finally abolished in Python 3). Use new style classes instead:
class foo(object):
and then the check if x in self: will not cause a recursive __getattr__ call. It will however cause a failure anyway, because your class does not define a __contains__ method and therefore you cannot check if x is contained in an instance of that class.
If what you're trying to do is whether x is defined in the instance dict of self, don't bother: __getattr__ doesn't even get called in that case -- it's only called when the attribute is not otherwise found in self.
To support three-arguments calls to the getattr built-in, just raise AttributeError in your __getattr__ method if necessary (just as would happen if you had no __getattr__ method at all), and the built-in will do its job (it's the built-in's job to intercept such cases and return the default if provided). That's the reason one never ever calls special methods such as __getattr__ directly but rather uses built-ins and operators which internally call them -- the built-ins and operators provide substantial added value.
So to give an example which makes somewhat sense:
class foo(object):
def __init__(self):
self.blah = {'a': 'boh'}
def __getattr__(self, x):
if x in self.blah:
return self.blah[x]
raise AttributeError
a=foo()
print getattr(a,'b','sss')
This prints sss, as desired.
If you add a __setattr__ method, that one intercepts every attempt to set attributes on self -- including self.blah = whatever. So -- when you need to bypass the very __setattr__ you're defining -- you must use a different approach. For example:
class foo(object):
def __init__(self):
self.__dict__['blah'] = {}
def __setattr__(self, name, value):
self.blah[name] = value
def __getattr__(self, x):
if x in self.blah:
return self.blah[x]
raise AttributeError
a=foo()
print getattr(a,'b','sss')
This also prints sss. Instead of
self.__dict__['blah'] = {}
you could also use
object.__setattr__(self, 'blah', {})
Such "upcalls to the superclass's implementation" (which you could also obtain via the super built-in) are one of the rare exceptions to the rules "don't call special methods directly, call the built-in or use the operator instead" -- here, you want to specifically bypass the normal behavior, so the explicit special-method call is a possibility.
You are confusing the getattr built-in function, which retrieves some attribute binding of an object dynamically (by name), at runtime, and the __getattr__ method, which is invoked when you access some missing attribute of an object.
You can't ask
if x in self:
from within __getattr__, because the in operator will cause __getattr__ to be invoked, leading to infinite recursion.
If you simply want to have undefined attributes all be defined as some value, then
def __getattr__(self, ignored):
return "Bob Dobbs"