I'm just trying to grok how exactly Python handles this behind the scenes. So take this code snippet (from Effective Python by Brett Slatkin):
class Resistor(object):
def __init__(self, ohms):
self.ohms = ohms
self.voltage = 0
self.current = 0
class VoltageResistor(Resistor):
def __init__(self, ohms):
super().__init__(ohms)
self._voltage = 0
#property
def ohms(self):
return self._ohms
#ohms.setter
def ohms(self, ohms):
if ohms <= 0:
raise ValueError('{o} ohms must be > 0'.format(o=ohms))
self._ohms = ohms
#property
def voltage(self):
return self._voltage
#voltage.setter
def voltage(self, voltage):
self._voltage = voltage
self.current = self._voltage / self.ohms
VoltageResistor(-1) # fails
Running the super() call invokes the property check so that you can't instantiate with a zero or negative value. What is confusing me to me is that I would think that since the the __init__(ohms) call is being ran on the superclass, shouldn't it be in a different scope (the scope of the superclass) and thus exempt from invoking the #property check?
Scope doesn't come into play when working with object's attributes. Consider the following:
class A(object):
def __init__(self):
self.a = 1
def foo():
a = A()
a.a = 2
return a
def bar(a):
print(a.a)
bar(foo())
This example code will print 2. Note that within the scope of bar, there is no way to gain access to the scope of foo or even A.__init__. The class instance is carrying along all of it's attributes/properties with it (and a reference to it's class which has a reference to it's superclass, etc).
In your code, when you call VoltageResistor, an instance of VoltageResistor is created and passed to __init__ as self. When you call super.__init__(self), that VoltageResistor instance is passed along to Resistor.__init__. When it does self.ohms = ohms, python sees that self.ohms resolves to a property and you get the error. The tl;dr; here is that self is an instance of VoltageResistor and when working with attributes, the object on which the attributes are accessed is what is important, not the current scope).
To supplement the above excellent answer, just add the following line in the parent's constructor to get a better idea of what is going on:
class Resistor(object):
def __init__(self, ohms):
print (type(self).__name__)
self.ohms = ohms
It will print VoltageResistor and then throw a ValueError. The Python docs confirm this:
If c is an instance of C, c.x will invoke the getter, c.x = value will invoke the setter and del c.x the deleter.
ValueError Happens there because VoltageResistor.__init__ calls Resistor.__init__, which assigns self.ohms = -1. That assignment causes the #ohms.setter method from VoltageResistor to be called, and it immediately runs the validation code before object construction has completed.
Related
I am using python and have an object, that object has a method. I am looking for a simple way, to replace the entire object from within that function.
E.g
class a():
def b(self):
self = other_object
How can you do that?
Thanks
You use a proxy/facade object to hold a reference to the actual object, the self if you wish and that proxy (better term than Facade, but not changing my code now) is what the rest of your codebase sees. However, any attribute/method access is forwarded on to the actual object, which is swappable.
Code below should give you a rough idea. Note that you need to be careful about recursion around __the_instance, which is why I am assigning to __dict__ directly. Bit messy, since it's been a while I've written code that wraps getattr and setattr entirely.
class Facade:
def __init__(self, instance):
self.set_obj(instance)
def set_obj(self, instance):
self.__dict__["__theinstance"] = instance
def __getattr__(self, attrname):
if attrname == "__theinstance":
return self.__dict__["__theinstance"]
return getattr(self.__dict__["__theinstance"], attrname)
def __setattr__(self, attrname, value):
if attrname == "__theinstance":
self.set_obj(value)
return setattr(self.__dict__["__theinstance"], attrname, value)
class Test:
def __init__(self, name, cntr):
self.name = name
self.cntr = cntr
def __repr__(self):
return "%s[%s]" % (self.__class__.__name__, self.__dict__)
obj1 = Test("first object", 1)
obj2 = Test("second", 2)
obj2.message = "greetings"
def pretend_client_code(facade):
print(id(facade), facade.name, facade.cntr, getattr(facade, "value", None))
facade = Facade(obj1)
pretend_client_code(facade)
facade.set_obj(obj2)
pretend_client_code(facade)
facade.value = 3
pretend_client_code(facade)
facade.set_obj(obj1)
pretend_client_code(facade)
output:
4467187104 first object 1 None
4467187104 second 2 None
4467187104 second 2 3
4467187104 first object 1 None
So basically, the "client code" always sees the same facade object, but what it is actually accessing depends on what your equivalent of def b is has done.
Facade has a specific meaning in Design Patterns terminology and it may not be really applicable here, but close enough. Maybe Proxy would have been better.
Note that if you want to change the class on the same object, that is a different thing, done through assigning self.__class__ . For example, say an RPG game with an EnemyClass who gets swapped to DeadEnemyClass once killed: self.__class__ = DeadEnemyClass
You can't directly do that. What you can do is save it as an instance variable.
class A():
def __init__(self, instance=None):
self.instance = val or self
# yes, you can make it a property as well.
def set_val(self, obj):
self.instance = obj
def get_val(self):
return self.instance
It is unlikely that replacing the 'self' variable will accomplish
whatever you're trying to do, that couldn't just be accomplished by
storing the result of func(self) in a different variable. 'self' is
effectively a local variable only defined for the duration of the
method call, used to pass in the instance of the class which is being
operated upon. Replacing self will not actually replace references to
the original instance of the class held by other objects, nor will it
create a lasting reference to the new instance which was assigned to
it.
Original source: Is it safe to replace a self object by another object of the same type in a method?
I have a class factory method that is used to instantiate an object. With multiple objects are created through this method, I want to be able to compare the classes of the objects. When using isinstance, the comparison is False, as can be seen in the simple example below. Also running id(a.__class__) and id(b.__class__), gives different ids.
Is there a simple way of achieving this? I know that this does not exactly conform to duck-typing, however this is the easiest solution for the program I am writing.
def factory():
class MyClass(object):
def compare(self, other):
print('Comparison Result: {}'.format(isinstance(other, self.__class__)))
return MyClass()
a = factory()
b = factory()
print(a.compare(b))
The reason is that MyClass is created dynamically every time you run factory. If you print(id(MyClass)) inside factory you get different results:
>>> a = factory()
140465711359728
>>> b = factory()
140465712488632
This is because they are actually different classes, dynamically created and locally scoped at the time of the call.
One way to fix this is to return (or yield) multiple instances:
>>> def factory(n):
class MyClass(object):
def compare(self, other):
print('Comparison Result: {}'.format(isinstance(other, self.__class__)))
for i in range(n):
yield MyClass()
>>> a, b = factory(2)
>>> a.compare(b)
Comparison Result: True
is a possible implementation.
EDIT: If the instances are created dynamically, then the above solution is invalid. One way to do it is to create a superclass outside, then inside the factory function subclass from that superclass:
>>> class MyClass(object):
pass
>>> def factory():
class SubClass(MyClass):
def compare(self, other):
print('Comparison Result: {}'.format(isinstance(other, self.__class__)))
return SubClass()
However, this does not work because they are still different classes. So you need to change your comparison method to check against the first superclass:
isinstance(other, self.__class__.__mro__[1])
If your class definition is inside the factory function, than each instance of the class you create will be an instance of a separate class. That's because the class definition is a statement, that's executed just like any other assignment. The name and contents of the different classes will be the same, but their identities will be distinct.
I don't think there's any simple way to get around that without changing the structure of your code in some way. You've said that your actual factory function is a method of a class, which suggests that you might be able to move the class definition somewhere else so that it can be shared by multiple calls to the factory method. Depending on what information you expect the inner class to use from the outer class, you might define it at class level (so there'd be only one class definition used everywhere), or you could define it in another method, like __init__ (which would create a new inner class for every instance of the outer class).
Here's what that last approach might look like:
class Outer(object):
def __init__(self):
class Inner(object):
def compare(self, other):
print('Comparison Result: {}'.format(isinstance(other, self.__class__)))
self.Inner = Inner
def factory(self):
return self.Inner()
f = Outer()
a = f.factory()
b = f.factory()
print(a.compare(b)) # True
g = Outer() # create another instance of the outer class
c = g.factory()
print(a.compare(c)) # False
It's not entirely clear what you're asking. It seems to me you want a simpler version of the code you already posted. If that's incorrect, this answer is not relevant.
You can create classes dynamically by explicitly constructing a new instance of the type type.
def compare(self, other):
...
def factory():
return type("MyClass", (object,), { 'compare': compare }()
type takes three arguments: the name, the parents, and the predefined slots. So this will behave the same way as your previous code.
Working off the answer from #rassar, and adding some more detail to represent the actual implementation (e.g. the factory-method existing in a parent class), I have come up with a working example below.
From #rassar's answer, I realised that the class is dynamically created each time, and so defining it within the parent object (or even above that), means that it will be the same class definition each time it is called.
class Parent(object):
class MyClass(object):
def __init__(self, parent):
self.parent = parent
def compare(self, other):
print('Comparison Result: {}'.format(isinstance(other, self.__class__)))
def factory(self):
return self.MyClass(self)
a = Parent()
b = a.factory()
c = a.factory()
b.compare(c)
print(id(b.__class__))
print(id(c.__class__))
Say I have an class that looks like this:
class Test(object):
def __init__(self, a, b):
self.a = a
self.b = b
self.c = self.a + self.b
I would like the value of self.c to change whenever the value of attributes self.a or self.b changes for the same instance.
e.g.
test1 = Test(2,4)
print test1.c # prints 6
test1.a = 3
print test1.c # prints = 6
I know why it would still print 6, but is there a mechanism I could use to fire an update to self.c when self.a has changed. Or the only option I have is to have a method that returns me the value of self.c based on the current state of self.a and self.b
Yes, there is! It's called properties.
Read Only Properties
class Test(object):
def __init__(self,a,b):
self.a = a
self.b = b
#property
def c(self):
return self.a + self.b
With the above code, c is now a read-only property of the Test class.
Mutable Properties
You can also give a property a setter, which would make it read/write and allow you to set its value directly. It would look like this:
class Test(object):
def __init__(self, c = SomeDefaultValue):
self._c = SomeDefaultValue
#property
def c(self):
return self._c
#c.setter
def c(self,value):
self._c = value
However, in this case, it would not make sense to have a setter for self.c, since its value depends on self.a and self.b.
What does #property mean?
The #property bit is an example of something called a decorator. A decorator actually wraps the function (or class) it decorates into another function (the decorator function). After a function has been decorated, when it is called it is actually the decorator that is called with the function (and its arguments) as an argument. Usually (but not always!) the decorated function does something interesting, and then calls the original (decorated) function like it would normally. For example:
def my_decorator(thedecoratedfunction):
def wrapped(*allofthearguments):
print("This function has been decorated!") #something interesting
thedecoratedfunction(*allofthearguments) #calls the function as normal
return wrapped
#my_decorator
def myfunction(arg1, arg2):
pass
This is equivalent to:
def myfunction(arg1, arg2):
pass
myfunction = my_decorator(myfunction)
So this means in the class example above, instead of using the decorator you could also do this:
def c(self):
return self.a + self.b
c = property(c)
They are exactly the same thing. The #property is just syntactic sugar to replace calls for myobject.c with the property getter and setter (deleters are also an option).
Wait - How does that work?
You might be wondering why simply doing this once:
myfunction = my_decorator(myfunction)
...results in such a drastic change! So that, from now on, when calling:
myfunction(arg1, arg2)
...you are actually calling my_decorator(myfunction), with arg1, arg2 sent to the interior wrapped function inside of my_decorator. And not only that, but all of this happens even though you didn't even mention my_decorator or wrapped in your function call at all!
All of this works by virtue of something called a closure. When the function is passed into the decorator in this way (e.g., property(c)), the function's name is re-bound to the wrapped version of the function instead of the original function, and the original function's arguments are always passed to wrapped instead of the original function. This is simply the way that closures work, and there's nothing magical about it. Here is some more information about closures.
Descriptors
So to summarize so far: #property is just a way of wrapping the class method inside of the property() function so the wrapped class method is called instead of the original, unwrapped class method. But what is the property function? What does it do?
The property function adds something called a descriptor to the class. Put simply, a descriptor is an object class that can have separate get, set, and delete methods. When you do this:
#property
def c(self):
return self._c
...you are adding a descriptor to the Test class called c, and defining the get method (actually, __get__()) of the c descriptor as equal to the c(self) method.
When you do this:
#c.setter
def c(self,value):
self._c
...you are defining the set method (actually, __set__()) of the c descriptor as equal to the c(self,value) method.
Summary
An amazing amount of stuff is accomplished by simply adding #property to your def c(self) method! In practice, you probably don't need to understand all of this right away to begin using it. However, I recommend keeping in mind that when you use #property, you are using decorators, closures, and descriptors, and if you are at all serious about learning Python it would be well worth your time to investigate each of these topics on their own.
The simplest solution is to make c a read-only property:
class Test(object):
def __init__(self, a, b):
self.a = a
self.b = b
#property
def c(self):
return self.a + self.b
Now every time you access test_instance.c, it calls the property getter and calculates the appropriate value from the other attributes. In use:
>>> t = Test(2, 4)
>>> t.c
6
>>> t.a = 3
>>> t.c
7
Note that this means that you cannot set c directly:
>>> t.c = 6
Traceback (most recent call last):
File "<pyshell#16>", line 1, in <module>
t.c = 6
AttributeError: can't set attribute
I am using the python mock framework for testing (http://www.voidspace.org.uk/python/mock/) and I want to mock out a superclass and focus on testing the subclasses' added behavior.
(For those interested I have extended pymongo.collection.Collection and I want to only test my added behavior. I do not want to have to run mongodb as another process for testing purposes.)
For this discussion, A is the superclass and B is the subclass. Furthermore, I define direct and indirect superclass calls as shown below:
class A(object):
def method(self):
...
def another_method(self):
...
class B(A):
def direct_superclass_call(self):
...
A.method(self)
def indirect_superclass_call(self):
...
super(A, self).another_method()
Approach #1
Define a mock class for A called MockA and use mock.patch to substitute it for the test at runtime. This handles direct superclass calls. Then manipulate B.__bases__ to handle indirect superclass calls. (see below)
The issue that arises is that I have to write MockA and in some cases (as in the case for pymongo.collection.Collection) this can involve a lot of work to unravel all of the internal calls to mock out.
Approach #2
The desired approach is to somehow use a mock.Mock() class to handle calls on the the mock just in time, as well as defined return_value or side_effect in place in the test. In this manner, I have to do less work by avoiding the definition of MockA.
The issue that I am having is that I cannot figure out how to alter B.__bases__ so that an instance of mock.Mock() can be put in place as a superclass (I must need to somehow do some direct binding here). Thus far I have determined, that super() examines the MRO and then calls the first class that defines the method in question. I cannot figure out how to get a superclass to handle the check to it and succeed if it comes across a mock class. __getattr__ does not seem to be used in this case. I want super to to think that the method is defined at this point and then use the mock.Mock() functionality as usual.
How does super() discover what attributes are defined within the class in the MRO sequence? And is there a way for me to interject here and to somehow get it to utilize a mock.Mock() on the fly?
import mock
class A(object):
def __init__(self, value):
self.value = value
def get_value_direct(self):
return self.value
def get_value_indirect(self):
return self.value
class B(A):
def __init__(self, value):
A.__init__(self, value)
def get_value_direct(self):
return A.get_value_direct(self)
def get_value_indirect(self):
return super(B, self).get_value_indirect()
# approach 1 - use a defined MockA
class MockA(object):
def __init__(self, value):
pass
def get_value_direct(self):
return 0
def get_value_indirect(self):
return 0
B.__bases__ = (MockA, ) # - mock superclass
with mock.patch('__main__.A', MockA):
b2 = B(7)
print '\nApproach 1'
print 'expected result = 0'
print 'direct =', b2.get_value_direct()
print 'indirect =', b2.get_value_indirect()
B.__bases__ = (A, ) # - original superclass
# approach 2 - use mock module to mock out superclass
# what does XXX need to be below to use mock.Mock()?
#B.__bases__ = (XXX, )
with mock.patch('__main__.A') as mymock:
b3 = B(7)
mymock.get_value_direct.return_value = 0
mymock.get_value_indirect.return_value = 0
print '\nApproach 2'
print 'expected result = 0'
print 'direct =', b3.get_value_direct()
print 'indirect =', b3.get_value_indirect() # FAILS HERE as the old superclass is called
#B.__bases__ = (A, ) # - original superclass
is there a way for me to interject here and to somehow get it to utilize a mock.Mock() on the fly?
There may be better approaches, but you can always write your own super() and inject it into the module that contains the class you're mocking. Have it return whatever it should based on what's calling it.
You can either just define super() in the current namespace (in which case the redefinition only applies to the current module after the definition), or you can import __builtin__ and apply the redefinition to __builtin__.super, in which case it will apply globally in the Python session.
You can capture the original super function (if you need to call it from your implementation) using a default argument:
def super(type, obj=None, super=super):
# inside the function, super refers to the built-in
I played around with mocking out super() as suggested by kindall. Unfortunately, after a great deal of effort it became quite complicated to handle complex inheritance cases.
After some work I realized that super() accesses the __dict__ of classes directly when resolving attributes through the MRO (it does not do a getattr type of call). The solution is to extend a mock.MagicMock() object and wrap it with a class to accomplish this. The wrapped class can then be placed in the __bases__ variable of a subclass.
The wrapped object reflects all defined attributes of the target class to the __dict__ of the wrapping class so that super() calls resolve to the properly patched in attributes within the internal MagicMock().
The following code is the solution that I have found to work thus far. Note that I actually implement this within a context handler. Also, care has to be taken to patch in the proper namespaces if importing from other modules.
This is a simple example illustrating the approach:
from mock import MagicMock
import inspect
class _WrappedMagicMock(MagicMock):
def __init__(self, *args, **kwds):
object.__setattr__(self, '_mockclass_wrapper', None)
super(_WrappedMagicMock, self).__init__(*args, **kwds)
def wrap(self, cls):
# get defined attribtues of spec class that need to be preset
base_attrs = dir(type('Dummy', (object,), {}))
attrs = inspect.getmembers(self._spec_class)
new_attrs = [a[0] for a in attrs if a[0] not in base_attrs]
# pre set mocks for attributes in the target mock class
for name in new_attrs:
setattr(cls, name, getattr(self, name))
# eat up any attempts to initialize the target mock class
setattr(cls, '__init__', lambda *args, **kwds: None)
object.__setattr__(self, '_mockclass_wrapper', cls)
def unwrap(self):
object.__setattr__(self, '_mockclass_wrapper', None)
def __setattr__(self, name, value):
super(_WrappedMagicMock, self).__setattr__(name, value)
# be sure to reflect to changes wrapper class if activated
if self._mockclass_wrapper is not None:
setattr(self._mockclass_wrapper, name, value)
def _get_child_mock(self, **kwds):
# when created children mocks need only be MagicMocks
return MagicMock(**kwds)
class A(object):
x = 1
def __init__(self, value):
self.value = value
def get_value_direct(self):
return self.value
def get_value_indirect(self):
return self.value
class B(A):
def __init__(self, value):
super(B, self).__init__(value)
def f(self):
return 2
def get_value_direct(self):
return A.get_value_direct(self)
def get_value_indirect(self):
return super(B, self).get_value_indirect()
# nominal behavior
b = B(3)
assert b.get_value_direct() == 3
assert b.get_value_indirect() == 3
assert b.f() == 2
assert b.x == 1
# using mock class
MockClass = type('MockClassWrapper', (), {})
mock = _WrappedMagicMock(A)
mock.wrap(MockClass)
# patch the mock in
B.__bases__ = (MockClass, )
A = MockClass
# set values within the mock
mock.x = 0
mock.get_value_direct.return_value = 0
mock.get_value_indirect.return_value = 0
# mocked behavior
b = B(7)
assert b.get_value_direct() == 0
assert b.get_value_indirect() == 0
assert b.f() == 2
assert b.x == 0
I would like to replace an object instance by another instance inside a method like this:
class A:
def method1(self):
self = func(self)
The object is retrieved from a database.
It is unlikely that replacing the 'self' variable will accomplish whatever you're trying to do, that couldn't just be accomplished by storing the result of func(self) in a different variable. 'self' is effectively a local variable only defined for the duration of the method call, used to pass in the instance of the class which is being operated upon. Replacing self will not actually replace references to the original instance of the class held by other objects, nor will it create a lasting reference to the new instance which was assigned to it.
As far as I understand, If you are trying to replace the current object with another object of same type (assuming func won't change the object type) from an member function. I think this will achieve that:
class A:
def method1(self):
newObj = func(self)
self.__dict__.update(newObj.__dict__)
It is not a direct answer to the question, but in the posts below there's a solution for what amirouche tried to do:
Python object conversion
Can I dynamically convert an instance of one class to another?
And here's working code sample (Python 3.2.5).
class Men:
def __init__(self, name):
self.name = name
def who_are_you(self):
print("I'm a men! My name is " + self.name)
def cast_to(self, sex, name):
self.__class__ = sex
self.name = name
def method_unique_to_men(self):
print('I made The Matrix')
class Women:
def __init__(self, name):
self.name = name
def who_are_you(self):
print("I'm a women! My name is " + self.name)
def cast_to(self, sex, name):
self.__class__ = sex
self.name = name
def method_unique_to_women(self):
print('I made Cloud Atlas')
men = Men('Larry')
men.who_are_you()
#>>> I'm a men! My name is Larry
men.method_unique_to_men()
#>>> I made The Matrix
men.cast_to(Women, 'Lana')
men.who_are_you()
#>>> I'm a women! My name is Lana
men.method_unique_to_women()
#>>> I made Cloud Atlas
Note the self.__class__ and not self.__class__.__name__. I.e. this technique not only replaces class name, but actually converts an instance of a class (at least both of them have same id()). Also, 1) I don't know whether it is "safe to replace a self object by another object of the same type in [an object own] method"; 2) it works with different types of objects, not only with ones that are of the same type; 3) it works not exactly like amirouche wanted: you can't init class like Class(args), only Class() (I'm not a pro and can't answer why it's like this).
Yes, all that will happen is that you won't be able to reference the current instance of your class A (unless you set another variable to self before you change it.) I wouldn't recommend it though, it makes for less readable code.
Note that you're only changing a variable, just like any other. Doing self = 123 is the same as doing abc = 123. self is only a reference to the current instance within the method. You can't change your instance by setting self.
What func(self) should do is to change the variables of your instance:
def func(obj):
obj.var_a = 123
obj.var_b = 'abc'
Then do this:
class A:
def method1(self):
func(self) # No need to assign self here
In many cases, a good way to achieve what you want is to call __init__ again. For example:
class MyList(list):
def trim(self,n):
self.__init__(self[:-n])
x = MyList([1,2,3,4])
x.trim(2)
assert type(x) == MyList
assert x == [1,2]
Note that this comes with a few assumptions such as the all that you want to change about the object being set in __init__. Also beware that this could cause problems with inheriting classes that redefine __init__ in an incompatible manner.
Yes, there is nothing wrong with this. Haters gonna hate. (Looking at you Pycharm with your in most cases imaginable, there's no point in such reassignment and it indicates an error).
A situation where you could do this is:
some_method(self, ...):
...
if(some_condition):
self = self.some_other_method()
...
return ...
Sure, you could start the method body by reassigning self to some other variable, but if you wouldn't normally do that with other parametres, why do it with self?
One can use the self assignment in a method, to change the class of instance to a derived class.
Of course one could assign it to a new object, but then the use of the new object ripples through the rest of code in the method. Reassiging it to self, leaves the rest of the method untouched.
class aclass:
def methodA(self):
...
if condition:
self = replace_by_derived(self)
# self is now referencing to an instance of a derived class
# with probably the same values for its data attributes
# all code here remains untouched
...
self.methodB() # calls the methodB of derivedclass is condition is True
...
def methodB(self):
# methodB of class aclass
...
class derivedclass(aclass):
def methodB(self):
#methodB of class derivedclass
...
But apart from such a special use case, I don't see any advantages to replace self.
You can make the instance a singleton element of the class
and mark the methods with #classmethod.
from enum import IntEnum
from collections import namedtuple
class kind(IntEnum):
circle = 1
square = 2
def attr(y): return [getattr(y, x) for x in 'k l b u r'.split()]
class Shape(namedtuple('Shape', 'k,l,b,u,r')):
self = None
#classmethod
def __repr__(cls):
return "<Shape({},{},{},{},{}) object at {}>".format(
*(attr(cls.self)+[id(cls.self)]))
#classmethod
def transform(cls, func):
cls.self = cls.self._replace(**func(cls.self))
Shape.self = Shape(k=1, l=2, b=3, u=4, r=5)
s = Shape.self
def nextkind(self):
return {'k': self.k+1}
print(repr(s)) # <Shape(1,2,3,4,5) object at 139766656561792>
s.transform(nextkind)
print(repr(s)) # <Shape(2,2,3,4,5) object at 139766656561888>