Python's unittest framework defines the addTypeEqualityFunc method, which "Registers a type-specific method called by assertEqual() to check if two objects of exactly the same typeobj (not subclasses) compare equal."
What are the scope an mechanism of this registration? The documentation does not discuss either, and "python registration" is resistant to searching.
Mostly, I am curious about whether this "registration" is some kind of assignment in local scope, or if Python has a broader concept of registration I haven't been able to unearth in the docs.
It applies to the current test only. Python unittest does not have some broader concept of "registration" in this case. Both the tests below will pass:
# run these tests with "python -m unittest" in same directory as this file.
from unittest import TestCase
class Potato:
pass
def potato_compare(potato1, potato2, msg=None):
return True
class Test1(TestCase):
def test_one(self):
spud = Potato()
yam = Potato()
self.addTypeEqualityFunc(Potato, potato_compare)
self.assertEqual(spud, yam)
def test_two(self):
spud = Potato()
yam = Potato()
self.assertNotEqual(spud, yam)
If you want it to apply to every test, you can add it to setUp:
class Test2(TestCase):
def setUp(self):
self.addTypeEqualityFunc(Potato, potato_compare)
def test_one(self):
spud = Potato()
yam = Potato()
self.assertEqual(spud, yam)
def test_two(self):
spud = Potato()
yam = Potato()
self.assertEqual(spud, yam)
Warning: This functionality does not recurse into containers. For example, [spud] and [yam] will not be recognized as equal lists.
Related
Context
I have a test class where all my tests inherit from. It cant run by itself as it really doesnt contain any setup info
I wanted to add a test which is executed by ALL tests (adding it to the baseclass seems logical)
But now I notice the basetestclass( => Foo) which I import is being detected as a test itself and runs and is visible in the reports
Code
the base class in base.py
from unittest import TestCase
class Foo(TestCase):
#classmethod
def setUpClass(cls):
# prepare the generic setup stuff based on what is defined in the child class
print("setupclass Foo done")
def test_run_in_all_inherited_tests(self):
print("fooBar")
assert True
the real test in test_something.py
from base import Foo # <= This is being detected as a testclass object and thus will be executed
class TestFoo(Foo):
#classmethod
def setUpClass(cls):
# define specific test setup
super().setUpClass()
print("setup TestFoo done")
def test_pass(self):
pass
def test_assert(self):
assert False
This triggers a testrun of the imported Foo
The Question
How can I import Foo without that its being detected as a 'test'
If I remove the test to run in all tests all is fine.
Adding #nottest decorator to Foo wont work since then also all inherited classes are defined nottest.
It needs to run on nose, pytest and unittest testrunners
I noticed if I changed the import statement like below that it also works. But that would mean adjusting a few hundreds of testfiles in different repos. (I'd like to avoid that)
import base
class TestFoo(base.Foo):
The key to the answer seems to be that each test has an attribute __test__ which is set to True when it is a test.
Setting it to False when the class should not be a test will then let the test collector ignore this class.
The answer assumes I can only do changes in the base.py
In python 3.9 classmethod and property decorators can be combined so I wrote a separate answer for that
answer for < py3.9
the base class in base.py
from unittest import TestCase
class MetaFoo(type):
#property
def __test__(cls):
return cls != Foo
class Foo(TestCase, metaclass=MetaFoo):
#classmethod
def setUpClass(cls):
# prepare the generic setup stuff based on what is defined in the child class
print("setupclass Foo done")
def test_run_in_all_inherited_tests(self):
print("fooBar")
assert True
answer for >= py3.9
the base class in base.py
from unittest import TestCase
class Foo(TestCase):
#classmethod
#property
def __test__(cls):
return cls != Foo
#classmethod
def setUpClass(cls):
# prepare the generic setup stuff based on what is defined in the child class
print("setupclass Foo done")
def test_run_in_all_inherited_tests(self):
print("fooBar")
assert True
the actual test
test_something.py
from base import Foo # <= This will not be detected as a test anymore as __test__ returns False
class TestFoo(Foo):
#classmethod
def setUpClass(cls):
# define specific test setup
super().setUpClass()
print("setup TestFoo done")
def test_pass(self):
pass
def test_assert(self):
assert False
This doesnt trigger a testrun of the imported Foo anymore
In my setUpClass I would like to create a resource in the database one time, which is then used for all of the tests in the class.
After I create the resource in setUpClass, I would like to perform assertions on it right then and there. However, I'm not sure how to call assertions in setUpClass, given that all of the assertion functions are instance methods, not class methods.
import unittest
class TestFoo(unittest.TestCase):
#classmethod
def setUpClass(cls):
cls.foo = cls.make_foo(name='bar')
# How would I assert that cls.foo['name'] == 'bar'?
# The following does not work, as the assertEquals function is not a class method
# cls.assertEquals(cls.foo['name'], 'bar')
#classmethod
def make_foo(cls, name):
# Calls a POST API to create a resource in the database
return {'name': name}
def test_foo_0(self):
# Do something with cls.foo
pass
def test_foo_1(self):
# do something else with cls.foo
pass
The only alternative I can think of is to raise an exception in setUpClass:
#classmethod
def setUpClass(cls):
cls.foo = cls.make_foo(name='bar')
if cls.foo['name'] != 'bar':
raise Exception("Failed to create resource, cannot do the tests")
Of course, I do not want to call the assertions from each test, as this will just duplicate the code.
Edit: I don't think this workaround is good, because the failure message will point to the self.assertFalse(self.flag) line, instead of the if cls.foo['name'] ~= 'bar' line. In addition, if you created multiple resources, this would need multiple flags to disambiguate.
flag=False
#classmethod
def setUpClass(cls):
cls.foo = cls.make_foo(name='bar')
if cls.foo['name'] != 'bar':
cls.flag=True
def setUp(self):
self.assertFalse(self.flag)
I have a code structure where we have a lot of sub classes from a common base class.
I want in an automated fashion test each subclass without having a separate class definition inheriting from TestCase for each subclass.
with a classs like
class MyBaseClass:
...
I can get a list of all subclasses that inherits from MyBaseClass with
import all_module_that_includes_subclasses
list_of_all_subclasses = MyBaseClass.__subclasses__()
But how do I connect to this list of scubclasses to avoid manually createing TestCases for each subclass like:
class TestSubClass1(TestCase):
def test_method_1(self):
...
...
class TestSubClass2(TestCase):
def test_method_1(self):
...
...
etc.
I thought of setting up an instance of all classes in setUp() method and looping through them in each def test_method_#(). This will somewhat work, I think, but it will break as soon as one single class fails. I want to be able to run through all classes and get a full report what class failed and in what tests.
Thankful for help :)
You can create those test cases dynamically using the load_tests protocol. For this you simply define a top-level function called load_tests which should return a TestSuite.
import unittest
class MyBaseClass:
pass
class Foo(MyBaseClass):
pass
class Bar(MyBaseClass):
pass
class TestBases:
# Wrapped into other class, so it won't be discovered by unittest.
class TestCase(unittest.TestCase):
subclass: MyBaseClass
def test(self):
self.assertIsInstance(self.subclass(), MyBaseClass)
def load_tests(loader, tests, pattern):
for cls in MyBaseClass.__subclasses__():
test_cls = type(f'Test{cls.__name__}', (TestBases.TestCase,), dict(subclass=cls))
tests.addTests(loader.loadTestsFromTestCase(test_cls))
return tests
if __name__ == '__main__':
unittest.main()
This runs the two dynamically generated test cases TestFoo and TestBar:
$ python /tmp/test.py -v
test (__main__.TestFoo) ... ok
test (__main__.TestBar) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
I have classes like these
class Test:
def __str__(self):
return "Test"
class Test1(Test):
def __str__(self):
return "Test1"
class Test2(Test):
def __str__(self):
return "Test2"
class Runner:
pass
class Runner1(Runner):
def run(self):
print("I'm a method, doing this and that")
print(f"And I use {Test1()}")
class Runner2(Runner):
def func2(self):
print("I'm a method, doing this and that")
test = Test2()
print(f"And I use {test}")
and I would like to discover all Runner classes, which use Test instances, like this:
for klass, func, ref in get_all_references(Runner):
if isinstance(ref, Test):
print(f"{klass.__name}.{func.__name} uses Test!")
That is, I'm looking for the get_all_references method, which returns all referenced objects of any classes of type Runner (and their methods), which I can inspect for class type/inheritance.
The motivation behind this is to discover all places (class/method names) where instances of Test are used.
I think part of the problem is solved by static analyzers/doc creators/cross reference builders, but I couldn't find any which could be used to get this information via an API.
i think gc module has several useful functions in that matter, but it sounds like gc.get_referrers() is what you need.
With python and nosetests I have the following setup:
- package
- __init__.py
- test1.py
- test2.py
The __init__.py module contains a set up function
def setup():
print("Setup called")
var = 42
which will be used later to create a unique identified (different between running the tests, but the same for all the tests inside the package).
How can the tests itself access this variable (in this example case var)? The test scripts are just some stubs:
from nose.tools import assert_true
class TestSuite(object):
def test1(self):
# How to get content of 'var' here?
assert_true(True)
Is there some pythonic way to do this, or just use an environment variable to do this?
nose calls .setup() methods inside classes:
class Test:
def setup(self):
self.var = 1
def test_print_var(self):
print(self.var)
This also applies to methods inherited from elsewhere:
class TestBase:
def setup(self):
self.var = 1
class Test(TestBase):
def test_print_var(self):
print(self.var)