I'm trying to create an object from a custom class with a decorator in it.
Then I need to use the decorator of that object to create a bunch of functions in a different file.
I found the solution of using builtins online but it feels a bit hacky and makes my lsp spit out a bunch of errors.
I'm wondering if anyone knows a better solution.
This is my file structure:
main.py
mylib
├── A.py
└── B.py
main.py
import builtins
from mylib.A import A
a = A("This is printing in the decorator")
builtins.FOO = a
import mylib.B as B
B.foo()
B.poo()
A.py
class A:
def __init__(self, _message):
self.message = _message
def our_decorator(self, func):
def function_wrapper():
print(self.message)
func()
return function_wrapper
B.py
import builtins
#builtins.FOO.our_decorator
def foo():
print("foo")
#builtins.FOO.our_decorator
def poo():
print("poo")
I don't want to change the file structure if it can be avoided.
Using builtins to have a magically created decorator is indeed hacky.
An alternative would be to patch the functions in B from main. It is slightly cleaner (and linters should not complain) because the B module has no longer to be aware of the decorator:
main.py:
from mylib.A import A
a = A()
import mylib.B as B
# decorate here the functions of the B module
B.foo = a.our_decorator(B.foo)
B.poo = a.our_decorator(B.poo)
B.foo()
B.poo()
A.py is unchanged...
B.py:
def foo():
print("foo")
def poo():
print("poo")
As only one version of a module exists in a process, you can even use the functions of B from a third module, provided:
either it is imported after the decoration
or it only imports the module name (import mylib.B as B) and not directly the functions (from mylib.B import foo)
Related
I would like to mock a function inside a module, however, this function is not explicitly imported. Consider following scenario.
helper_functions.py
def foo_1():
...
def foo_2():
...
helper_functions = [foo1, foo2]
my_module
from helper_functions import helper_functions
def do_something(foo):
...
some_results = [do_something(foo) for foo in helper_functions]
I need to patch ONLY foo2 but in my_module, because I'm basically testing how do_something works.
When I try this
from unittest import patch
#patch('my_module.foo2')
def test_do_something():
...
I get an obvious error saying that foo2 does not exist in the namespace of my_module.
Can I bypass it somehow?
I desire the following behaviour
# module.py
""" module's docString """
def foo():
print(??something??)
do other stuff`
# script.py
""" Scripts docString """
from module import foo
foo()
do otherstuff
Now when I run python script.py I would like the call to foo() to print the docString of script.py
I have tried
# module.py
""" module's docString """
def foo():
print(__doc__)
do other stuff`
But this prints the docString of module.py
You can use sys.modules to get a reference to the __main__ module, whatever module that might be. With that, you can access its __doc__ attribute.
# module.py
""" module's docString """
import sys
def foo():
print(sys.modules['__main__'].__doc__)
The docstring is stored in the module's __doc__ global.
def foo():
print("something")
import script
print(script.__doc__)
I have class with few methods. Let's say A
file a.py
class A:
def foo():
...
def bar():
...
during normal runtime (not testing) I use it in different modules, like that.
file functions.py
from a import A
def sample_function():
a_instance = A()
result = a_instance.foo()
return result
But during tests I would like to replace it with different class, let's say MockA.
file mock_a.py
class MockA:
# same methods name, but with different implementation
def foo():
...
def bar():
...
Now I would like to test module with some functionality
tests
from functions import sample_function
def test_sample_function():
assert sample_function() == expected_output
The QUESTION is:
Can I somehow "globally" set alias A = MockA (or do this in other way), so that during tests sample_function use functionality from MockA?
sample_function uses whatever A is bound to in the global namespace of the module functions. You can rebind your own class to that name.
from functions import one_function
class MockA:
...
functions.A = MockA
def test_sample_function():
assert sample_function() == expected_output
This is exactly what unittest.mock.patch is for
How about you use:
tests
from unittest.mock import patch
from mock_a import MockA
from functions import sample_function
#patch('functions.A', new_callable=MockA)
def test_sample_function(mocked_A):
assert sample_function() == expected_output
Try changeing this:
from a import A
into this:
from mock_a import MockA as A
First correct me in comments if I am wrong, but is it a mistake that you wrote one_function in your last code snippet, where you should have written sample_function.
I think your concern is that you don't want to change the functions.py code by replacing a_instance = A() with a_instance = MockA() everywhere in the code. So just make mock_a.py with same class name class A: and same methods name, but different implementations(like you said). All you will have to change in your functions.py code is from mock_a import A instead all all instance of class A() to class MockA(). This way I think your tests should work perfectly.
I want to do something like the following example (found here)
>>> with patch.object(ProductionClass, 'method', return_value=None) as mock_method:
... thing = ProductionClass()
... thing.method(1, 2, 3)
However this is patching the method called method on ProductionClass. I want to patch a generic function within a context. Ideally something looking like...
with path.something(my_fn, return_value=my_return) as mock_function:
do_some_other_fn()
my_fn is called deep within do_some_other_fn and therefore is difficult to mock out directly. This seems like it should be straight forward but I can't find the right syntax
EDIT In the module that do_some_other_fn lives I import my_fn like followings
from my_module import my_fn
So I need a way to be able to tell mock to patch that from outside the module. Is this possible?
EDIT 2 I think this makes it more clear what I am looking for
This works but is not ideal:
import my_module
with patch('my_module.fn', return_value='hello') as patch_context:
x = my_module.fn()
# x now contains 'hello'
However I would much rather have it work like this (or something similar)
from my_module import fn
with patch('my_module.fn', return_value='hello') as patch_context:
x = fn()
# x contains real result from real call to fn()
Your attempt to patch with from my_module import fn does not work because the import statement creates a local symbol fn which points to whatever value fn has in my_module at the time of import. You later patch my_module.fn but it does not matter because you already have a local copy of fn.
If the file that contains the patch call is the main module (the file that python initially loaded), you should be able to do it by patching __main__.fn:
from my_module import fn
with patch('__main__.fn', return_value='hello') as patch_context:
x = fn()
If the file that contains the patch call is loaded as a module from the main module then __main__ won't work and you need to pass the absolute module name of the module that contains your patch call to patch rather than __main__.
You can see function like module object's static method. To patch a function func in module mymodule you can use
patch("mymodule.func", return_value=my_return)
You should take care of Where to patch and if the function is in the same module where you have the test should use "__main__.func" as patch argument.
patch like patch.object can be useed as decorator, context or by start() and stop() method.
Now when in a module you import a function from an other module like:
from mymodule import func as foo
You create a new reference to func in the new module called foo. Every
time in this module you call foo you will use the reference to mymodule.func that you load when you imported it: if you whould like change this behavior you should patch foo in the new module.
To make it more clear I build an example where you have mymodule that contain func, module_a that include mymodule and use mymodule.func, module_b that use from mymodule import func as foo and use bot foo and mymodule.func
mymodule.py
def func():
return "orig"
module_a.py
import mymodule
def a():
return mymodule.func()
module_b.py
from mymodule import func as foo
import mymodule
def b_foo():
return foo()
def b():
return mymodule.func()
test.py
import unittest
from unittest.mock import *
import mymodule
import module_a
import module_b
class Test(unittest.TestCase):
def test_direct(self):
self.assertEqual(mymodule.func(), "orig")
with patch("mymodule.func", return_value="patched"):
self.assertEqual(mymodule.func(), "patched")
def test_module_a(self):
self.assertEqual(module_a.a(), "orig")
with patch("mymodule.func", return_value="patched"):
self.assertEqual(module_a.a(), "patched")
def test_module_b(self):
self.assertEqual(module_b.b(), "orig")
with patch("mymodule.func", return_value="patched"):
self.assertEqual(module_b.b(), "patched")
self.assertEqual(module_b.b_foo(), "orig")
with patch("module_b.foo", return_value="patched"):
self.assertEqual(module_b.b(), "orig")
self.assertEqual(module_b.b_foo(), "patched")
if __name__ == '__main__':
unittest.main()
In other words what really rules on choosing where to patch is how the function is referenced where you want use the patched version.
I want to separate code in different class and put them to different files. However those class are dependent on each other.
main.py:
from lib import A, B
def main():
a = A()
b = B()
a.hello()
b.hello()
if __name__ == '__main__':
main()
lib/_init_.py:
from a import A
from b import B
lib/a.py:
import lib.B
class A():
def __init__(self):
print "A"
def hello(self):
print "hello A"
b = B()
lib/b.py:
import lib.A
class B():
def __init__(self):
print "B"
def hello(self):
print "hello B"
a = A()
Is it possible to do that in Python?
EDIT:
I get this error message:
pydev debugger: starting
Traceback (most recent call last):
File "eclipse-python/plugins/org.python.pydev_2.7.1.2012100913/pysrc/pydevd.py", line 1397, in <module>
debugger.run(setup['file'], None, None)
File "eclipse-python/plugins/org.python.pydev_2.7.1.2012100913/pysrc/pydevd.py", line 1090, in run
pydev_imports.execfile(file, globals, locals) #execute the script
File "main.py", line 2, in <module>
from lib import A, B
File "lib/__init__.py", line 1, in <module>
from a import A
File "lib/a.py", line 1, in <module>
import lib.B
ImportError: No module named B
Instead of importing the modules on top, you could import the other module within the hello function.
class B():
def __init__(self):
print "B"
def hello(self):
from lib import A
print "hello B"
a = A()
When you have two classes depending on each other usually means that either they really belong to the same module or that you have a too tight coupling that should be resolved using dependency injection.
Now there are indeed a couple corner cases where importing from within the function is the "least worst" solution but that's still something you should avoid as much as possible.
Your main problem is that you're trying to import a class, but using syntax that only works to import a module. Specifically, import lib.A is never going to work if A is a class defined in module lib.a (and imported into the top-level namespace of lib).
What I suggest is that you avoid using the from _ import _ syntax unless you really need it. That makes the dependencies much easier to resolve:
lib/a.py:
import lib.b # note, we're not importing the class B, just the module b!
class A():
def foo(self):
return lib.b.B() # use the class later, with a qualified name
lib/b.py:
import lib.a # again, just import the module, not the class
class B():
def foo(self):
return lib.a.A() # use another qualified name reference
lib/__init__.py:
from a import A # these imports are fine, since the sub-modules don't rely on them
from b import B # they can be the public API for the A and B classes
You could also use relative module imports if you don't want a and b to be dependent on the name of their package lib.
This is sure to work because neither class A or B actually requires the other to exist yet in order to be defined. It's only after they're imported that instances of A need to know about the class B (and vise versa).
If one of the classes inherited from the other, or otherwise used an instance of the other at top level, you'd need to be more careful about which module was loaded up first, or it might still break.
If you want to import it only once you can import it in the constructor of the class and make the variable global:
class B():
def __init__(self):
global A
from lib import A
print "B"
def hello(self):
print "hello B"
a = A()
This would import A into a global variable and make it accessible form within the module.