I know how fierce the SO community is so I'll try my best to keep the question minimal, complete and verifiable.
What I simply want to know is can monkey patching be used to replace the definition of an existing function?
for example:
class A():
def foo():
print '2'
def foo():
print '5'
A.foo = foo
This way doesn't seem to work also as to why I don't just add a new function instead of replacing an existing one, I call these functions in other classes and it is my understanding that monkey patching adds those functions at run-time and I need my python code to run on an Apache spark server which throws an error deeming the calls to that function unreferenced.
So please be nice and help me out or suggest a work around.
Thanks.
Edit: The goal of the code is to print 5 when A.foo is called.
Your only problem is that you aren't defining foo correctly in the first place. It needs to take an explicit argument for the instance calling it.
class A(object):
def __init__(self)
self.x = 2
def foo(self):
print(self.x)
def foo(this):
print(this.x + 3)
A.foo = foo
a = A()
a.foo() # outputs 5 in Python 2 and Python 3
In a very real sense, monkey patching is how classes are created in the first place. A class statement is almost just syntactic sugar for the following code:
def foo(self):
print(self.x)
A = type('A', (object,), {'foo': foo})
del foo
It's not too much of a simplification to image the definition of type being something like
def type(name, bases, d):
new_class = magic_function_to_make_a_class()
new_class.name = name
new_class.bases = bases
for k, v in d.items():
setattr(new_class, k, v)
return new_class
I hope I understand what you are trying to do here. This would work in Python 3:
class A():
def foo():
print('2')
def foo():
A.foo = lambda: print('5')
A.foo() # Print '2'
foo() # Assign the new method
A.foo() # Prints '5'
In Python 2 however there are several caveats.
print is not a method as it is in Python 3 (see here: Why doesn't print work in a lambda?)
You can't just call unbound messages (see here: In Python, can you call an instance method of class A, but pass in an instance of class B?)
So you have to do it like this:
from __future__ import print_function
class A():
def foo():
print('2')
def foo():
A.foo = lambda: print('5')
A.foo.__func__() # Print '2'
foo() # Assign the new method
A.foo.__func__() # Prints '5'
Edit:
After seeing your question in the comment I think you actually want something different. Which is this:
class A():
def foo(self):
print '2'
def foo(self):
print '5'
a = A()
a.foo() # Print '2'
A.foo = foo # Assign the new method
a.foo() # Prints '5'
This works just fine in Python 2.
The self is a reference to the current instance the method is bound to. It is not used when you just call something like print which access any properties or methods attached to that instance. But for a different case please have a look at the following example:
class A():
msg = "Some message"
def foo(self):
print self.msg
def bar(self):
self.msg = "Some other message"
a = A()
a.foo() # Print old msg
A.bar = bar # Assign the new method
a.bar() # Assigns new message
a.foo() # Prints new message
Also as chepner points out in a comment under his post:
The name self isn't special; it's just a convention. You could use
this if you really wanted to, and it doesn't need to be the same name
in both functions defined here. What is important is that the first
argument to a function used as an instance method will be a reference
to the calling object. a.foo() is almost exactly the same as A.foo(a)
Related
New to python here and I am looking to attach some class, ClassA.methodA to another ClassB as below
def methodA(self, x=None, y=None, z='somevalue', l=None, m=False, **kwds):
... Some logic here....
a = self.getMyInstanceOfClassA() //a is of type ClassA, not ClassB
a.methodA(x,y,z,l,m,**kwds)
ClassB.methodA = methodA
...
h = ClassB()
h.methodA("p1", "p2", m=True)
The goal is to keep the signature of ClassB.methodA the same as it is supposed to be on the actual ClassA.methodA which is accessible on ClassB.getMyInstanceOfClassA() method.
The reason I am doing this is to avoid my user to write this code
h = ClassB()
---the extra logic---
a = h.getMyInstanceOfClassA()
a.methodA(....)
an instead just say
h = ClassB()
h.methodA(....)
and I embed the extra logic inside the new methodA on ClassB.
methodA is something meaningful in our domain and I have to keep its name and exact signature
I have to import both ClassA and ClassB from our internal libs and I can't make ClassB inherit from ClassA.
The point is that methodA is usually not called with all of its arguments and the arguments passed depends on what you want methodA to do. This h.methodA("p1", "p2", m=True) fails with some error complaining about passing too many argument to it.
Am I wrapping the methodA correctly?
I think you would typically just use inheritance or just make that method a function unrelated to the class with one of the inputs being the instance of that class. Or you would just wrap it in a lambda function.
Python: Bind an Unbound Method?
If however you want to take a method and bind it to an instance of a class use something like the below code.
inst = MyClass()
inst.methodA = methodA.__get__(inst, MyClass)
I am not sure what is causing your error (We cannot be sure unless you atleast show the complete traceback or atleast a demo code that reproduces your issue).
But from the code that you have pasted, what you are trying to do would not be possible. Lets take a simple example -
>>> class A:
... def methodA(self, a):
... print(a)
... print("In A")
...
>>> def methodA(self, a, b=None):
... print("In Main")
... if b == None:
... return
... else:
... h = type(self)
... h.methodA(self, a)
...
>>> A.methodA = methodA
>>> h = A()
>>> h.methodA(1,2)
In Main
In Main
I just put the b == None part, so that the code does not do inifinite recursion, but what you are trying to would cause infinite recursion. Why?
Because you have changed methodA to point to a new method at the class level, so even if you call methodA for the class, it would still call the newly bound methodA , not the old one.
If you really do not want the users to call the old methodA directly, one thing you can do is to bind that method to a different name , maybe _methodA , and then bind your new function to methodA. Example -
>>> class A:
... def _methodA(self, a):
... print(a)
... print("In A")
...
>>> def methodA(self, a, b=None):
... print("In Main")
... if b == None:
... return
... else:
... h = type(self)
... h._methodA(self, a)
...
>>> A.methodA = methodA
>>> h = A()
>>> h.methodA(1,2)
In Main
1
In A
I can not understand following commented line.
It's a python program.
class B:
def bbb(self):
method = self.commands[0]
method(self) #I can't umderstand this line
class A(B):
def aaa(self):
print 'aaa was called'
commands = [aaa]
c = A()
c.bbb()
Output:
aaa was called
I think above aaa method takes no arguments.
But to run this code, I need to pass "self" to aaa argument. Why? Are there any docs explaining this?What category this problem belongs?
Any simple codes are very welcome.
Because my English skill is too low.
So improving this question is welcome too.
I encountered this problem, when I was reading cpython/Lib/distutils/cmd.py:Command.get_sub_commands().
Thank you for reading.
Wow, this is confusingly written. Working backward from the code itself:
c = A()
Creates an instance of A. Looking at A:
def aaa(self):
print 'aaa was called'
commands = [aaa]
This is a bit confusingly written; it makes more sense like this:
def aaa(self):
print 'aaa was called'
commands = [aaa]
Defines a method aaa, then a class variable commands which contains aaa as an element. Now, looking at the next line of the program:
c.bbb()
Since A has no bbb and A inherits from B, we consult B:
class B:
def bbb(self):
method = self.commands[0]
method(self)
Since we've established that commands is [aaa], the first line means method = aaa. So the second line is effectively aaa(self).
This line:
method(self) #I can't umderstand this line
Calls the function aaa(). In your function declaration:
def aaa(self):
aaa does takes an argument (self). That's why you have to call it with method(self).
Since self.commands[0] is a function, calling method(self) is equal to:
aaa(self)
Comment if you have something else to ask!
The way the code sample is done makes it a little harder to tell what's going on. However, it's equivalent to:
child_methods = [] # a list of all the methods in `Child`
class Parent(object):
def parent_method(self):
print "parent_method() called"
method = child_methods[0]
method(self)
class Child(Parent):
def child_method(self):
print "child_method() called"
# add child_method to child_methods
child_methods.append(Child.child_method)
As you can see, child_methods[0] will actually be the function Child.child_method, which is a plain function, and not a bound method. It's not associated with an instance of Child, which is why you can and have to pass in the self yourself. You'd get a bound method from a Child instance with:
child_obj = Child()
bound_child_method = child_obj.child_method
This is made unclear by the fact Python will look up attributes in the type of an object if they're not found in the instance. For example:
# A dummy class to hold attributes
class Foo(object):
pass
Foo.bar = 123 # we're adding an attribute to the type itself
foo1 = Foo()
print foo1.bar # prints 123
foo1.bar = 456 # this `bar` attribute "hides" the one on the type
print foo1.bar # prints 456
foo2 = Foo()
print foo2.bar # prints the 123 from the type again
This is why, in your code sample, commands is really a "global" variable, it just gets accessed confusingly through an instance of B. (This is not necessarily a bad practice if only objects of B or its children access this variable, but the lookup rules are a minor gotcha.)
By the way it is better to use new-style classes, class A(object):...
All methods of class in python have self as first argument, except of class methods.
Here is example about self:
def x(first, arg):
print "Called x with arg=",arg
print first
class A(object):
some_method = x
a = A()
a.some_method("s")`
http://docs.python.org/2/tutorial/classes.html#random-remarks
>>> class Class:
... def method(self):
... print 'I have a self!'
...
>>> def function():
... print "I don't..."
...
>>> instance = Class()
>>> instance.method()
I have a self!
>>> instance.method = function
>>> instance.method()
I don't...
Okay the book quotes 'The self parameter (mentioned in the previous section) is, in fact, what distinguishes methods
from functions. Methods (or, more technically, bound methods) have their first parameter
bound to the instance they belong to: you don’t have to supply it. So while you can certainly
bind an attribute to a plain function, it won’t have that special self parameter:'
I am not able to understand what the author is trying to convey here ! I am new to oop in python . Please explain me .
Methods only exist on the class; assigning the function to the instance attribute as your example does creates an instance attribute containing the function, and not a method.
It means that affect of
class A:
def a(self):
print 'a'
def b(self, arg):
print arg
can be roughly represented by:
def A_a(self):
print a
def A_b(self, arg):
print arg
class A:
def __init__(self):
self.a = lambda: A_a(self)
self.b = lambda arg: A_b(self, arg)
So instance.a is not original function a which is written in class A, but another function which calls original with additional self argument.
When should I use __init__ and when __call__ method ?
I am confused about whether should I use the first or the second.
At the moment I can use them both, but I don't know which is more appropriate.
These two are completely different.
__init__() is the constructor, it is run on new instances of the object.
__call__() is run when you try to call an instance of an object as if it were a function.
E.g: Say we have a class, Test:
a = Test() #This will call Test.__init__() (among other things)
a() #This will call Test.__call__()
A quick test shows the difference between them
class Foo(object):
def __init__(self):
print "init"
def __call__(self):
print "call"
f = Foo() # prints "init"
f() # prints "call"
In no way are these interchangeable
Most likely, you want to use __init__. This is the method used to initialize a new instance of your class, which you make by calling the class. __call__ is in case you want to make your instances callable. That's not something frequently done, though it can be useful. This example should illustrate:
>>> class C(object):
... def __init__(self):
... print 'init'
... def __call__(self):
... print 'call'
...
>>> c = C()
init
>>> c()
call
>>>
A simple code snippet will elaborate this better.
>>> class Math:
... def __init__(self):
... self.x,self.y=20,30
... def __call__(self):
... return self.x+self.y
...
>>> m=Math()
>>> m()
50
I want to mock out methods on any instance of some class in the production code in order to facilitate testing. Is there any library in Python which could facilitate this?
Basically, I want to do the following, but in Python (the following code is Ruby, using the Mocha library):
def test_stubbing_an_instance_method_on_all_instances_of_a_class
Product.any_instance.stubs(:name).returns('stubbed_name')
assert_equal 'stubbed_name', SomeClassThatUsesProduct.get_new_product_name
end
The important thing to note from above is that I need to mock it out on the class level, since I'm actually need to mock out methods on an instance created by the thing I'm testing.
Use Case:
I have a class QueryMaker which calls a method on an instance of RemoteAPI. I want to mock out the RemoteAPI.get_data_from_remote_server method to return some constant. How do I do this inside a test without having to put a special case within the RemoteAPI code to check for what environment it's running in.
Example of what I wanted in action:
# a.py
class A(object):
def foo(self):
return "A's foo"
# b.py
from a import A
class B(object):
def bar(self):
x = A()
return x.foo()
# test.py
from a import A
from b import B
def new_foo(self):
return "New foo"
A.foo = new_foo
y = B()
if y.bar() == "New foo":
print "Success!"
Needing to mock out methods when testing is very common and there are lots of tools to help you with it in Python. The danger with "monkey patching" classes like this is that if you don't undo it afterwards then the class has been modified for all other uses throughout your tests.
My library mock, which is one of the most popular Python mocking libraries, includes a helper called "patch" that helps you to safely patch methods or attributes on objects and classes during your tests.
The mock module is available from:
http://pypi.python.org/pypi/mock
The patch decorator can be used as a context manager or as a test decorator. You can either use it to patch out with functions yourself, or use it to automatically patch with Mock objects that are very configurable.
from a import A
from b import B
from mock import patch
def new_foo(self):
return "New foo"
with patch.object(A, 'foo', new_foo):
y = B()
if y.bar() == "New foo":
print "Success!"
This handles the unpatching for you automatically. You could get away without defining the mock function yourself:
from mock import patch
with patch.object(A, 'foo') as mock_foo:
mock_foo.return_value = "New Foo"
y = B()
if y.bar() == "New foo":
print "Success!"
Mock is the way to do it, alright.
It can be a bit tricky to make sure you're patching the instance method on any instances created from the class.
# a.py
class A(object):
def foo(self):
return "A's foo"
# b.py
from a import A
class B(object):
def bar(self):
x = A()
return x.foo()
# test.py
from a import A
from b import B
import mock
mocked_a_class = mock.Mock()
mocked_a_instance = mocked_a_class.return_value
mocked_a_instance.foo.return_value = 'New foo'
with mock.patch('b.A', mocked_a_class): # Note b.A not a.A
y = B()
if y.bar() == "New foo":
print "Success!"
Referenced in the docs, at the para starting "To configure return values on methods of instances on the patched class..."
Easiest way is probably to use a class method. You really should use an instance method, but it's a pain to create those, whereas there's a built-in function that creates a class method. With a class method, your stub will get a reference to the class (rather than the instance) as the first argument, but since it's a stub this probably doesn't matter. So:
Product.name = classmethod(lambda cls: "stubbed_name")
Note that the signature of the lambda must match the signature of the method you're replacing. Also, of course, since Python (like Ruby) is a dynamic language, there is no guarantee that someone won't switch out your stubbed method for something else before you get your hands on the instance, though I expect you will know pretty quickly if that happens.
Edit: On further investigation, you can leave out the classmethod():
Product.name = lambda self: "stubbed_name"
I was trying to preserve the original method's behavior as closely as possible, but it looks like it's not actually necessary (and doesn't preserve the behavior as I'd hoped, anyhow).
I don't know Ruby quite well enough to tell exactly what you're trying to do, but check out the __getattr__ method. If you define it in your class, Python will call it when code tries to access any attribute of your class that isn't otherwise defined. Since you want it to be a method, it will need to create a method on the fly that it returns.
>>> class Product:
... def __init__(self, number):
... self.number = number
... def get_number(self):
... print "My number is %d" % self.number
... def __getattr__(self, attr_name):
... return lambda:"stubbed_"+attr_name
...
>>> p = Product(172)
>>> p.number
172
>>> p.name()
'stubbed_name'
>>> p.get_number()
My number is 172
>>> p.other_method()
'stubbed_other_method'
Also note that __getattr__ needs to not use any other undefined attributes of your class, or else it will be infinitely recursive, calling __getattr__ for the attribute that doesn't exist.
... def __getattr__(self, attr_name):
... return self.x
>>> p.y
Traceback (most recent call last):
#clipped
RuntimeError: maximum recursion depth exceeded while calling a Python object
If this is something you only want to do from your test code, not the production code, then put your normal class definition in the production code file, then in the test code define the __getattr__ method (unbound), and then bind it to the class you want:
#production code
>>> class Product:
... def __init__(self, number):
... self.number = number
... def get_number(self):
... print "My number is %d" % self.number
...
#test code
>>> def __getattr__(self, attr):
... return lambda:"stubbed_"+attr_name
...
>>> p = Product(172)
>>> p.number
172
>>> p.name()
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
AttributeError: Product instance has no attribute 'name'
>>> Product.__getattr__ = __getattr__
>>> p.name()
'stubbed_name'
I'm not sure how this would react with a class that was already using __getattribute__ (as opposed to __getattr__, __getattribute__ is called for all attributes whether or not they exist).
If you only want to do this for specific methods that already exist, then you could do something like:
#production code
>>> class Product:
... def __init__(self, number):
... self.number = number
... def get_number(self):
... return self.number
...
>>> p = Product(172)
>>> p.get_number()
172
#test code
>>> def get_number(self):
... return "stub_get_number"
...
>>> Product.get_number = get_number
>>> p.get_number()
'stub_get_number'
Or if you really wanted to be elegant, you could create a wrapper function to make doing multiple methods easy:
#test code
>>> import functools
>>> def stubber(fn):
... return functools.wraps(fn)(lambda self:"stub_"+fn.__name__)
...
>>> Product.get_number = stubber(Product.get_number)
>>> p.get_number()
'stub_get_number'
#Orignal Class definition - path "module.Product"
class Product:
def method_A(self):
# do something
pass
def method_B(self):
self.random_attr = 1
#Test case
from module import Product
class MockedProduct(Product):
def method_B(self):
self.random_attr = 2
with mock.patch('module.Product', new=MockedProduct):
#Write test case logic here
#Now method_B function call on product class instance should return 2
#instead of 1
minimal reproducible example using pytest and monkeypatch
# a.py
class A(object):
def foo(self):
return "A's foo"
# b.py
from a import A
class B(object):
def bar(self):
x = A()
return x.foo()
# test_ab.py
import pytest
from a import A
from b import B
def new_foo(self):
return "New foo"
def test_mock_instance_method(monkeypatch):
y = B()
print(y.bar())
monkeypatch.setattr(A, 'foo', new_foo)
print(y.bar())
gives you
$ pytest -rP .
============================= test session starts ==============================
platform linux -- Python 3.8.8, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /home/user/220225so
plugins: cov-3.0.0
collected 1 item
test_ab.py . [100%]
==================================== PASSES ====================================
__________________________ test_mock_instance_method ___________________________
----------------------------- Captured stdout call -----------------------------
A's foo
New foo
============================== 1 passed in 0.01s ===============================
$