Develop Reference

Can we assign an object to a callable (or to a function) in python?

Introduction:
We know that, in python, a function is a class. To some extent, We can look at it as a data type which can be called and return a value. So it is a callable. We also know that Python classes are callables. When they are called, we are actually making objects as their instances.
My implementation: In a current task, I have defined the following class with two methods:
class SomeClass():
def some_method_1():
some_code
def some_method_2():
some_code
self.some_method_1()
some_code
To describe the code above, some_method_2 is using some_method_1 inside it.
Now I am seeking to test some_method_2. In this case, I need to replace some_method_1 with a mock object and specify the mock object to return what I define:
from unittest.mock import Mock
import unittest
class TestSomeClass(unittest.TestCase):
some_object=Some_Class()
some_object.some_method_1 = Mock(return_value=foo)
self.assertEqual(an_expected_value, some_object.some_method_2())
This works totally fine and the script runs without error and the test result is also ok.
A fast Intro about Mypy: Mypy is a python type checking tool which can run and check if the script is written correctly based on variable types. It does this process by some criteria such as type annotations, variable assignments and libraries stub files. This process is done without interpreting and running the code.
What is the Problem?
When I try to check my code with mypy, it gives error for this line:
some_object.some_method_1 = Mock(return_value=foo)
The error indicates that I am not allowed to assign an object to a callable in python. Sometimes mypy does not report real errors and I doubt if this is the case. Especially because I can run my code with no problem.
Now, my question is, have I done my job wrong or just the mypy report is wrong? If I have done wrong, how can I implement the same scenario in a correct manner?

Python 'call_command' mock is used in other tests as well

Using Django 1.10 and python 3.5.1.
I'm trying to mock 'call_command' function to throw an exception. The problem is that seems like the moment it gets the 'side_effect' function - it keeps to it also for other tests. What am I doing wrong or how can I 'revert' the side_effect from that function?
In this example, after running one of the tests, all other tests that run afterwards will throw the same exception even if it's not supposed to throw exception in that test.
def test_run_migrations_raise_exception(self):
with mock.patch('django.core.management.call_command', return_value=None, side_effect=Exception('e message')):
self.check_migrations_called(MigrationTracker.objects.all(), data_migrations_settings_in_db)
call_command('run_data_migrations')
self.check_migrations_called(MigrationTracker.objects.all(), data_migrations_settings_in_db)
def test_run_migrations_raise_flow_exception(self):
with mock.patch('django.core.management.call_command', return_value=None, side_effect=FlowException(500, 'fe message', {'a': 1})):
self.check_migrations_called(MigrationTracker.objects.all(), data_migrations_settings_in_db)
call_command('run_data_migrations')
self.check_migrations_called(MigrationTracker.objects.all(), data_migrations_settings_in_db)

You should not patch a function that is in your module-local (i.e. Python's "global" - which is actually "module") namespace.
When in Python you do
from module.that import this
this becomes a variable on the module that contains the import statement. Any changes to "module.that.this" will affect the object pointed in the other module, but using only this will still reefer to the original object.
Perhaps your code is not exactly as you show us, or maybe "mock.pacth" can find out that the module-local call_command is pointing to django.core.management.call_command in the other module when it makes the patch - but not when reversing the patch. The fact is your module-local name call_command is being changed.
You can fix that by simply changing your code to not bind a module variable directly to the function you want to change:
from django.core import management
def test_run_migrations_raise_exception(self):
with mock.patch('django.core.management.call_command', return_value=None, side_effect=Exception('e message')):
self.check_migrations_called(MigrationTracker.objects.all(), data_migrations_settings_in_db)
management.call_command('run_data_migrations')
self.check_migrations_called(MigrationTracker.objects.all(), data_migrations_settings_in_db)
I hope you can understand that and solve this problem. Now, that said, this use of mock makes no sense at all: the idea of using mock is that some callable used indirectly by code you call within the code-block that applies the patch does not have the original effect - so the intermetiate code can run and be tested. You are calling directly the mock object - so it will have none of the original code - calling call_command('run_data_migrations') runs no code on your code base at all, and thus, there is nothing there to test. It just calls the mocked instance, and it will not change the status of anything that could be detected with check_migrations_called.

How to mock Python classes when nested several dependencies deep

If I have the following architecture...
Please note the edits below. It occurred to me (after some recent refactoring) that there are actually three classes in three different files. Sorry that the file/class names are getting ridiculous. I assure you those are not the real names. :)
main_class.py
class MainClass(object):
def do_some_stuff(self):
dependent_class = DependentClass()
dependent_class.py
class DependentClass(object):
def __init__():
dependent_dependent_class = DependentDependentClass()
dependent_dependent_class.do_dependent_stuff()
dependent_dependent_class.py
class DependentDependentClass(object):
def do_dependent_stuff(self):
print "I'm gonna do production stuff that I want to mock"
print "Like access a database or interact with a remote server"
class MockDependentDependentClass(object):
def do_dependent_stuff(self):
print "Respond as if the production stuff was all successful."
and I want to call main_class.do_some_stuff during testing but, during its execution, I want instances of DependentDependentClass replaced with MockDependentDependentClass how can I do that pythonically using best practices.
Currently, the best thing I could come up with is to conditionally instantiate one class or the other based on the presence/value of an environment variable. It certainly works but is pretty dirty.
I spent some time reading about the unittest.mock and mock.patch functions and they seem like they might be able to help but each description that I could wrap my head around seemed to be a little different than my actual use case.
The key is that I don't want to define mock return values or attributes but that I want the namespace changed, globally, I guess, such that when my application thinks it is instantiating DependentClass it is actually instantiating MockDependentClass.
The fact that I can't find any examples of anyone doing exactly this means one of two things:
It's because I'm doing it in a very dumb/naive way.
I'm doing something so genius no else has ever encountered it.
... I assume it's number 1...
Full disclosure, unit testing is not something with which I am skilled. It's an effort that my internal tools development team is trying to catch up to step our game up a bit. It's possible that I'm not thinking about testing correctly.
Any thoughts would be most welcome. Thank you, in advance!
SOLUTION!!!
Thanks to #de1 for the help. Given my clever architecture shown above the following accomplishes what I want.
The following code is located in main_class.py
import dependent_class
from dependent_dependent_class import MockDependentDependentClass
with patch.object(dependent_class, "DependentDependentClass", MockDependentDependentClass):
main_class = MainClass()
main_class.do_some_stuff()
The code seems to (and hell if I know how it's doing this) manipulate the namespace within the module dependent_class so that, while inside the with block (that's a context manager for anyone who is hung up on that part) anything referring to the class object DependentDependentClass will actually be referencing MockDependentDependentClass.

The mock module does indeed seem to be a good fit in this case. You can specify the mock (your mock) to use when calling the various patch methods.
If you are importing only the class rather than the module you can patch the imported DependentDependentClass in DependentClass:
import .DependentClass as dependent_class
from .DependentDependentClass import MockDependentDependentClass
with patch.object(dependent_class, 'DependentDependentClass', MockDependentDependentClass):
# do something while class is patched
Alternatively:
with patch('yourmodule.DependentClass.DependentDependentClass', MockDependentDependentClass):
# do something while class is patched
or the following will only work if you are accessing the class via a module or import it after it is being patched:
with patch('yourmodule.DependentDependentClass.DependentDependentClass', MockDependentDependentClass):
# do something while class is patched
Just bare in mind what object is being patched, when.
Note: you might find it less confusing naming your files in lower case, slightly different to the embedded class(es).
Note 2: If you need to mock a dependency of a dependency of the module under test then it might suggest that you are not testing at the right level.

Override a "private" method in a python module

I want to test a function in python, but it relies on a module-level "private" function, that I don't want called, but I'm having trouble overriding/mocking it. Scenario:
module.py
_cmd(command, args):
# do something nasty
function_to_be_tested():
# do cool things
_cmd('rm', '-rf /')
return 1
test_module.py
import module
test_function():
assert module.function_to_be_tested() == 1
Ideally, in this test I dont want to call _cmd. I've looked at some other threads, and I've tried the following with no luck:
test_function():
def _cmd(command, args):
# do nothing
pass
module._cmd = _cmd
although checking module._cmd against _cmd doesn't give the correct reference. Using mock:
from mock import patch
def _cmd_mock(command, args):
# do nothing
pass
#patch('module._cmd', _cmd_mock)
test_function():
...
gives the correct reference when checking module._cmd, although `function_to_be_tested' still uses the original _cmd (as evidenced by it doing nasty things).
This is tricky because _cmd is a module-level function, and I dont want to move it into a module

[Disclaimer]
The synthetic example posted in this question works and the described issue become from specific implementation in production code. Maybe this question should be closed as off topic because the issue is not reproducible.
[Note] For impatient people Solution is at the end of the answer.
Anyway that question given to me a good point to thought: how we can patch a method reference when we cannot access to the variable where the reference is?
Lot of times I found some issue like this. There are lot of ways to meet that case and the commons are
Decorators: the instance we would like replace is passed as decorator argument or used in decorator static implementation
What we would like to patch is a default argument of a method
In both cases maybe refactor the code is the best way to play with that but what about if we are playing with some legacy code or the decorator is a third part decorator?
Ok, we have the back on the wall but we are using python and in python nothing is impossible. What we need is just the reference of the function/method to patch and instead of patching its reference we can patch the __code__: yes I'm speaking about patching the bytecode instead the function.
Get a real example. I'm using default parameter case that is simple, but it works either in decorator case.
def cmd(a):
print("ORIG {}".format(a))
def cmd_fake(a):
print("NEW {}".format(a))
def do_work(a, c=cmd):
c(a)
do_work("a")
cmd=cmd_fake
do_work("b")
Output:
ORIG a
ORIG b
Ok In this case we can test do_work by passing cmd_fake but there some cases where is impossible do it: for instance what about if we need to call something like that:
def what_the_hell():
list(map(lambda a:do_work(a), ["c","d"]))
what we can do is patch cmd.__code__ instead of _cmd by
cmd.__code__ = cmd_fake.__code__
So follow code
do_work("a")
what_the_hell()
cmd.__code__ = cmd_fake.__code__
do_work("b")
what_the_hell()
Give follow output:
ORIG a
ORIG c
ORIG d
NEW b
NEW c
NEW d
Moreover if we want to use a mock we can do it by add follow lines:
from unittest.mock import Mock, call
cmd_mock = Mock()
def cmd_mocker(a):
cmd_mock(a)
cmd.__code__=cmd_mocker.__code__
what_the_hell()
cmd_mock.assert_has_calls([call("c"),call("d")])
print("WORKS")
That print out
WORKS
Maybe I'm done... but OP still wait for a solution of his issue
from mock import patch, Mock
cmd_mock = Mock()
#A closure for grabbing the right function code
def cmd_mocker(a):
cmd_mock(a)
#patch.object(module._cmd,'__code__', new=cmd_mocker.__code__)
test_function():
...
Now I should say never use this trick unless you are with the back on the wall. Test should be simple to understand and to debug ... try to debug something like this and you will become mad!

Is it worth to ensure that non-callable Python classes do not get called?

If yes, can you suggest a better way than below? Please elaborate/justify.
class X:
...
if __name__ == '__main__':
# TODO: Should we bother doing this?
errorMessage = 'This class is not meant to have a main method. Do not execute directly.'
print(errorMessage)
raise Error(errorMessage)

I would not do this.
Here's why: python -i scriptname.py can be used to load in the script and drop to the interactive console post running of the script. So what you then get is all the defined functions, which you can then manipulate / test as you see fit.
Your way, you'd get an error doing this.

A common pattern is to place unit tests for a Python module inside the module body. For example:
if __name__ == "__main__":
run_unit_tests()
However, if you're not doing this, I wouldn't even bother adding any module body code. If executed as a script, a Python module with no body code will do nothing.

No, not worth it. After all, if there are no top level function calls, the code won't do anything,

1) No. If something "can't be called", it will raise an exception anyway.
2) The if __name__ == '__main__' construct refers to the module, not the class. "Calling" a class creates an instance of the class. You don't "call" modules, either; you either import or run them.
Please be more careful with your terminology. When you understand the concepts properly, the answers to these sorts of questions become obvious.