I am trying to extend my custom class 'Test' with a few of the output functions contained within the 'logging' module. I have a custom logging level called 'print' which is causing me issues. I have tried multiple methods and each has some drawback. Is there any other methods that would be better that I am missing.
I want to be able to use vars() on the object while and only print out the attributes that aren't logging methods.
This method works but prints out the 'print' method (which I don't want)
#Test.py
logger = logging.getLogger('console')
class Test:
def __init__(self):
self._some_obj = True
self.print= logger.print
#main.py
vars(Test()) # Prints {'_some_obj':True,'print': <bound method Logger.printof <Logger console (PRINT)>>}
This method correctly only prints out only correct attributes but overwrites default print in the process
#Test.py
logger = logging.getLogger('console')
class Test:
print = logger.print # Overwrites default print
def __init__(self):
self._some_obj = True
#main.py
vars(Test()) # Prints {'_some_obj':True}
This method also works but doesn't seem very pythonic. Since I will be doing this to alot of functions it will take up alot of space and I have also had issues passing through optional parameters
#Test.py
logger = logging.getLogger('console')
class Test:
def __init__(self):
self._some_obj = True
def print(msg):
logger.print(msg)
#main.py
vars(Test()) # Prints {'_some_obj':True}
Related
Let's say I have different classes defined like this
class PS1:
def __init__(self):
pass
def do_sth(self):
logging.info('PS1 do sth')
class PS2:
def __init__(self):
pass
def do_sth(self):
logging.info('PS2 do sth')
How can I make PS1 log into PS1.log and PS2 log into PS2.log? I know I can set up loggers and file handlers and do something likes ps1_logger.info instead of logging.info. However, my current code base already has hundreds of logging.info, do I have to change them all?
When you call logging methods on the module level you are creating logs directly at the root logger, so you no longer can separate them by logger. The only chance left is to separate your logs by Handler with a filter. Since you don't want to update your logging calls to add any information about where the call happens you have to inspect the stack to find the calling class. The code below does this, but I highly recommend not using this solution in production and refactor your code to not send everything to the root logger. Look into logging adapters and/or the extra keyword argument for an alternative solution that modifies the logging calls.
import logging
import inspect
root = logging.getLogger()
ps1_handler = logging.FileHandler('ps1.log')
ps2_handler = logging.FileHandler('ps2.log')
def make_class_filter(class_name):
def filter(record):
for fi in inspect.stack():
if 'self' in fi[0].f_locals and class_name in str(fi[0].f_locals['self']):
return True
return False
return filter
ps1_handler.addFilter(make_class_filter('PS1'))
ps2_handler.addFilter(make_class_filter('PS2'))
root.addHandler(ps1_handler)
root.addHandler(ps2_handler)
class PS1:
def do_sth(self):
logging.warning('PS1 do sth')
class PS2:
def do_sth(self):
logging.warning('PS2 do sth')
PS1().do_sth()
PS2().do_sth()
I have the following function in a file called my_file.py:
import logging
log = logging.getLogger(__name__)
class MyClass:
def my_func(self):
print("Entered my_func()")
print("log.error = {}".format(log.error))
log.error("Hello World")
When my_func() is run, I see this output:
Entered my_func()
log.error = <bound method ProcessAwareLogger.error of <celery.utils.log.ProcessAwareLogger object at 0x7fd52d4f0d50>>
I want to write a unit-test case to check that whenever my_func() is called, log.error() is called with the argument "Hello World".
So my test case looks like this:
import mock
from my_file import MyClass
#mock.patch("my_file.logging.error")
def test_parse_response_with_error(self, mock_log_error):
m = MyClass()
m.my_func()
mock_log_error.assert_called_once_with("Hello World")
When I run this test case, I get the error: AssertionError: Expected to be called once. Called 0 times. So my patch apparently didn't work. Did I specify the path my_file.logging.error wrong? How can I make this work properly so that the unit-test case passes?
You mocked the wrong thing. log.error creates a bound method that is distinct from logging.error, and that method already has a reference to the "real" logging.error saved before your patch was created.
Instead, mock the error attribute of the Logger instance directly.
#mock.patch.object(my_file.log, 'error'):
def test_it(self, mock_log_error):
m = MyClass()
m.my_func()
mock_log_error.assert_called_once_with("Hello World")
(It's possible that patching logging.error before importing my_file might work.)
I have some code that creates instances from a list of classes that is passed to it. This cannot change as the list of classes passed to it has been designed to be dynamic and chosen at runtime through configuration files). Initialising those classes must be done by the code under test as it depends on factors only the code under test knows how to control (i.e. it will set specific initialisation args). I've tested the code quite extensively through running it and manually trawling through reams of output. Obviously I'm at the point where I need to add some proper unittests as I've proven my concept to myself. The following example demonstrates what I am trying to test:
I would like to test the run method of the Foo class defined below:
# foo.py
class Foo:
def __init__(self, stuff):
self._stuff = stuff
def run():
for thing in self._stuff:
stuff = stuff()
stuff.run()
Where one (or more) files would contain the class definitions for stuff to run, for example:
# classes.py
class Abc:
def run(self):
print("Abc.run()", self)
class Ced:
def run(self):
print("Ced.run()", self)
class Def:
def run(self):
print("Def.run()", self)
And finally, an example of how it would tie together:
>>> from foo import Foo
>>> from classes import Abc, Ced, Def
>>> f = Foo([Abc, Ced, Def])
>>> f.run()
Abc.run() <__main__.Abc object at 0x7f7469f9f9a0>
Ced.run() <__main__.Abc object at 0x7f7469f9f9a1>
Def.run() <__main__.Abc object at 0x7f7469f9f9a2>
Where the list of stuff to run defines the object classes (NOT instances), as the instances only have a short lifespan; they're created by Foo.run() and die when (or rather, sometime soon after) the function completes. However, I'm finding it very tricky to come up with a clear method to test this code.
I want to prove that the run method of each of the classes in the list of stuff to run was called. However, from the test, I do not have visibility on the Abc instance which the run method creates, therefore, how can it be verified? I can't patch the import as the code under test does not explicitly import the class (after all, it doesn't care what class it is). For example:
# test.py
from foo import Foo
class FakeStuff:
def run(self):
self.run_called = True
def test_foo_runs_all_stuff():
under_test = Foo([FakeStuff])
under_test.run()
# How to verify that FakeStuff.run() was called?
assert <SOMETHING>.run_called, "FakeStuff.run() was not called"
It seems that you correctly realise that you can pass anything into Foo(), so you should be able to log something in FakeStuff.run():
class Foo:
def __init__(self, stuff):
self._stuff = stuff
def run(self):
for thing in self._stuff:
stuff = thing()
stuff.run()
class FakeStuff:
run_called = 0
def run(self):
FakeStuff.run_called += 1
def test_foo_runs_all_stuff():
under_test = Foo([FakeStuff, FakeStuff])
under_test.run()
# How to verify that FakeStuff.run() was called?
assert FakeStuff.run_called == 2, "FakeStuff.run() was not called"
Note that I have modified your original Foo to what I think you meant. Please correct me if I'm wrong.
I'm encountering a problem with unit testing in Python. Specifically, when I try to mock a function my code imports, variables assigned to the output of that function get assigned to a MagicMock object instead of the mock-function's return_value. I've been digging through the docs for python's unittest library, but am not having any luck.
The following is the code I want to test:
from production_class import function_A, function_B, function_M
class MyClass:
def do_something(self):
variable = functionB()
if variable:
do_other_stuff()
else:
do_something_else
this is what I've tried:
#mock.patch(path.to.MyClass.functionB)
#mock.patch(<other dependencies in MyClass>)
def test_do_something(self, functionB_mock):
functionB_mock.return_value = None # or False, or 'foo' or whatever.
myClass = MyClass()
myClass.do_something()
self.assertTrue(else_block_was_executed)
The issue I have is that when the test gets to variable = functionB in MyClass, the variable doesn't get set to my return value; it gets set to a MagicMock object (and so the if-statement always evaluates to True). How do I mock an imported function such that when executed, variables actually get set to the return value and not the MagicMock object itself?
We'd have to see what import path you're actually using with path.to.MyClass.functionB. When mocking objects, you don't necessarily use the path directly to where the object is located, but the one that the intepreter sees when recursively importing modules.
For example, if your test imports MyClass from myclass.py, and that file imports functionB from production_class.py, the mock path would be myclass.functionB, instead of production_class.functionB.
Then there's the issue that you need additional mocks of MyClass.do_other_stuff and MyClass.do_something_else in to check whether MyClass called the correct downstream method, based on the return value of functionB.
Here's a working example that tests both possible return values of functionB, and whether they call the correct downstream method:
myclass.py
from production_class import functionA, functionB, functionM
class MyClass:
def do_something(self):
variable = functionB()
if variable:
self.do_other_stuff()
else:
self.do_something_else()
def do_other_stuff(self):
pass
def do_something_else(self):
pass
production_class.py
import random
def functionA():
pass
def functionB():
return random.choice([True, False])
def functionM():
pass
test_myclass.py
import unittest
from unittest.mock import patch
from myclass import MyClass
class MyTest(unittest.TestCase):
#patch('myclass.functionB')
#patch('myclass.MyClass.do_something_else')
def test_do_something_calls_do_something_else(self, do_something_else_mock, functionB_mock):
functionB_mock.return_value = False
instance = MyClass()
instance.do_something()
do_something_else_mock.assert_called()
#patch('myclass.functionB')
#patch('myclass.MyClass.do_other_stuff')
def test_do_something_calls_do_other_stuff(self, do_other_stuff_mock, functionB_mock):
functionB_mock.return_value = True
instance = MyClass()
instance.do_something()
do_other_stuff_mock.assert_called()
if __name__ == '__main__':
unittest.main()
calling python test_myclass.py results in:
..
----------------------------------------------------------------------
Ran 2 tests in 0.002s
OK
What I wound up doing was changing the import statements in MyClass to import the object instead of the individual methods. I was then able to mock the object without any trouble.
More explicitly I changed MyClass to look like this:
import production_class as production_class
class MyClass:
def do_something(self):
variable = production_class.functionB()
if variable:
do_other_stuff()
else:
do_something_else
and changed my test to
#mock.patch(path.to.MyClass.production_class)
def test_do_something(self, prod_class_mock):
prod_class_mock.functionB.return_value = None
myClass = MyClass()
myClass.do_something()
self.assertTrue(else_block_was_executed)
I'm getting confused by using Mock in my python unittests. I've made this simplified version of my problem:
I have this dummy class and methods:
# app/project.py
class MyClass(object):
def method_a(self):
print(FetcherA)
results = FetcherA()
Which is using this class:
# app/fetch.py
class FetcherA(object):
pass
And then this test:
# app/tests/test.py
from mock import patch
from django.test import TestCase
from ..project import MyClass
class MyTestCase(TestCase):
#patch('app.fetch.FetcherA')
def test_method_a(self, test_class):
MyClass().method_a()
test_class.assert_called_once_with()
I would expect that running this test would pass and that print statement, for debugging, would output something like <MagicMock name=...>. Instead it prints out <class 'app.fetch.FetcherA'> and I get:
AssertionError: Expected to be called once. Called 0 times.
Why isn't FetcherA being patched?
OK, fourth time through I think I understood the 'Where to patch' section of the Mock docs.
So, instead of:
#patch('app.fetch.FetcherA')
I should use:
#patch('app.project.FetcherA')
Because we're testing the code in app.project.MyClass where FetcherA has been imported already. So at that point FetcherA is availably globally(?) within app.project.