I am trying to write a parser in Python for a special text file format. To get an idea how to structure the code I looked into the source code of the JSON parser which is part of the Python standard library (Python/Lib/json).
In this json directory there is a tests directory which holds a number of unit tests. I replaced the json tests with my tests but now I do not know how to call them.
Looking into the directory there is a __init__.py file making it a module and inside of this file there is the following code snippet for running the tests:
here = os.path.dirname(__file__)
def test_suite():
suite = additional_tests()
loader = unittest.TestLoader()
for fn in os.listdir(here):
if fn.startswith("test") and fn.endswith(".py"):
modname = "json.tests." + fn[:-3]
__import__(modname)
module = sys.modules[modname]
suite.addTests(loader.loadTestsFromModule(module))
return suite
def additional_tests():
suite = unittest.TestSuite()
for mod in (json, json.encoder, json.decoder):
suite.addTest(doctest.DocTestSuite(mod))
suite.addTest(TestPyTest('test_pyjson'))
suite.addTest(TestCTest('test_cjson'))
return suite
def main():
suite = test_suite()
runner = unittest.TextTestRunner()
runner.run(suite)
if __name__ == '__main__':
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
main()
My problem is now how are these unit tests executed? I am confused because the if __name__ == '__main__': if clause validates to true if this file is called directly and not being imported as a module. However as it is in the __init__.py file of the module it should be executed right after import.
Should an import tests in the python console start all the unit tests?
First: the __init__.py file in a directory marks the directory as a package and all other files with the .py ending therein are modules.
Second: To validate the __name__ == '__main__' conditional you have to execute the file. So a simple import won't do.
For further question on package and module structure i would suggest ro read the Official Python Documentation on Packages.
The unittests are structured into testsuites, which can contain one ore more TestCases which consists of one or more tests.
The tests are usually methods of a TestCase derived class. You can either run the tests by defining a runTest() method or defining multiple test_* methods, which will be executed autmatically. To execute a Unittest you can either use the convenience function unittest.main() which basicaly tries to construct a testsuite, testresult and testrunner object by using default-rules.
The execution of your unittest itself is done by the testrunner object. The standard testrunner-class is TextTestRunner, which uses a TextTestResult-class to store the testresults and prints them to stdout. See Official unittest Documentation for unittest.main() for the simplest variant.
Summary
1) A unittest is basically a TestSuite containing one ore more TestCases:
TestSuite <---executed by--- TestRunner
+ TestCaseA |
+test_a() |
+test_b() stores test-results
... into
+test_z() |
+ TestCaseB V
+ TestCaseC TestResult
2) The TestCases are subclasses of unittest.TestCase. You can create a TestSuite, by using a loader (e.g.: unittest.defaultTestLoader) which is basically a factory for a testsuite, or you can add the TestCases manually (suite.addTest(test) / suite.addTests(tests) - tests may be TestCases or even other TestSuites) or by combining those two methods.
3) To execute a testsuite you use a unittest.TestRunner-object which saves the results in a unittest.TestResult object.
Normaly you would be using unittest.TextTestRunner-object to get a simple output of the testresults to stdout.
That's excatly what happens also in your main-routine:
def main():
suite = test_suite() #1 create a TestSuite object
runner = unittest.TextTestRunner() #2 create a TextTestRunner object
runner.run(suite) #3 executes the TestSuite with TestSuite
# build by the function test_suite()
To execute your testsuite you would have then to execute python __init__.py .
Related
Within my python package, I use the standard modules logging to create logs, and unittest to create unit tests.
Since the files containing the unit tests are spread across the entire folder structure of the package, I use the test discovery mode to execute the unit tests:
python -m unittest discover \
--pattern "*_test.py" \
--start-directory "path/to/package/root"
Unfortunately, this results in a messed up text stream on the console because both log messages and output from the unit test runner are forwarded to stdout.
For single test modules (which I store in separate files <module_name>_test.py), I usually control the logging output globally within the __main__-block:
# In <module_name>_test.py
import logging
import unittest
# ...
if __name__ == "__main__":
# Forward log messages into a file test.log.
logging.basicConfig(level=logging.DEBUG,
filename="testing/output/test.log")
unittest.main()
When executing python <module_name>_test.py, this ensures that the log messages are separated from the unit test output.
However, in test discovery mode, the __main__-blocks of my test-modules are never executed. Therefore my question is: How to globally set up logging in test discovery mode? Ideally, I would have to configure the logger only in a single place, and not in all test modules separately. Is this possible?
The solution was relatively easy after rereading the documentation on unittest discovery more carefully. The functionality is implemented in TestLoader.discover().
To run all unit tests, I wrote a file run_tests.py that looks as follows:
import logging
import unittest
if __name__ == "__main__":
logging.basicConfig(level=logging.DEBUG,
filename="test.log")
loader = unittest.TestLoader()
tests = loader.discover(pattern="*_test.py",
start_dir="<path/to/package>")
runner = unittest.runner.TextTestRunner()
runner.run(tests)
Suppose I have this two nosetest ExampleTest1.py and ExampleTest2.py
ExampleTest1.py
class ExampleTest1(TestBase):
"""
"""
def testExampleTest1(self):
-----
-----
if __name__ == "__main__":
import nose
nose.run()
---------------
ExampleTest2.py
class ExampleTest2(TestBase):
"""
"""
def testExampleTest2(self):
-----
-----
if __name__ == "__main__":
import nose
nose.run()
Now I want to run such hundreds of test files from a single suite.
I am looking something like TestNG feature like testng.xml below where I can add all my test files which should run one by one
<suite name="Suite1">
<test name="ExampleTest1">
<classes>
<class name="ExampleTest1" />
</classes>
</test>
<test name="ExampleTest2">
<classes>
<class name="ExampleTest2" />
</classes>
</test>
</suite>
In case testng.xml like feature is not available in python, then what is other alternative to create test suites and include all my python test there? Thanks
Given that there may be multiple different reasons why you may want
to construct test suites, I’ll give you several options.
Just running tests from directory
Let’s say there is mytests dir:
mytests/
├── test_something_else.py
└── test_thing.py
Running all tests from that dir is a easy as
$> nosetests mytests/
For example, you could put smoke, unit, and integration tests into
different dirs and still be able to run “all tests”:
$> nosetests functional/ unit/ other/
Running tests by tag
Nose has
attribute selector plugin.
With test like this:
import unittest
from nose.plugins.attrib import attr
class Thing1Test(unittest.TestCase):
#attr(platform=("windows", "linux"))
def test_me(self):
self.assertNotEqual(1, 0 - 1)
#attr(platform=("linux", ))
def test_me_also(self):
self.assertFalse(2 == 1)
You’ll be able to run tests that have particular tag:
$> nosetests -a platform=linux tests/
$> nosetests -a platform=windows tests/
Running manually constructed test suite
Finally, nose.main supports suite argument: if it is passed,
discovery is not done.
Here I provide you basic example of how to manually construct
test suite and then run it with Nose:
#!/usr/bin/env python
import unittest
import nose
def get_cases():
from test_thing import Thing1Test
return [Thing1Test]
def get_suite(cases):
suite = unittest.TestSuite()
for case in cases:
tests = unittest.defaultTestLoader.loadTestsFromTestCase(case)
suite.addTests(tests)
return suite
if __name__ == "__main__":
nose.main(suite=get_suite(get_cases()))
As you can see, nose.main gets regular unittest test suite, constructed
and returned by get_suite. The get_cases function is where test cases
of your choice are “loaded” (in example above case class is just imported).
If you really need XML, get_cases may be the place where you return case
classes that you get from modules (imported via __import__ or
importlib.import_module) which you get from XML file you parsed.
And near nose.main call you could use argparse to get path to XML file.
I wrote a python script to do all my tests automatically for me, and generate a HTML report. I discovered discover for unittests the other day which lets me run all the unittests in a given directory without explicitly naming them, and I'd really like to be able to do my doctests the same way, rather than having to import each module explicitly.
I found some info on how to do this at https://docs.python.org/2/library/doctest.html but didn't really get it. Could you please help me with using discover with my doctests?
Python test discovery with doctests, coverage and parallelism is related, but still doesn't answer my question.
coverage_module
import coverage
import doctest
import unittest
import os
# import test_module
import my_module
cov = coverage.Coverage()
cov.start()
# running doctest by explicity naming the module
doctest.testmod(my_module)
# running unittests by just specifying the folder to look into
testLoad = unittest.TestLoader()
testSuite = testLoad.discover(start_dir=os.getcwd())
runner = unittest.TextTestRunner()
runner.run(testSuite)
cov.stop()
cov.save()
cov.html_report()
print "tests completed"
test_module
import unittest
import doctest
from my_module import My_Class
class My_Class_Tests(unittest.TestCase):
def setUp(self):
# setup variables
def test_1(self):
# test code
# The bit that should load up the doctests? What's loader, tests, and ignore though?
# Is this in the right place?
def load_tests(loader, tests, ignore):
tests.addTests(doctest.DocTestSuite(module_with_doctests))
return tests
if __name__ == '__main__':
unittest.main()
Lets figure out what's happening there
1) unittest.discovery
It has no clue of doctests as doctests is a different framework.
So unittest isn't supposed to discover doctests out of the box.
That means you'll need to glue them together by hand
2) doctest
It's essentially a separate framework although it has some glueing classes to convert doctests into unittest-like TestCases.
https://docs.python.org/2.7/library/doctest.html#doctest.DocTestSuite
3) discover
Didn't get what discover you mean, I suppose it's
python -m unittest discover
If not and you're talking about https://pypi.python.org/pypi/discover then just forget about it - it's a backport for earlier versions of python
4) what to do
either scatter a lot of load_tests hooks across your code as described here https://docs.python.org/2.7/library/doctest.html#unittest-api or code a method to collect all the modules your have in one place and convert them into a DocTestSuite[s] https://docs.python.org/2.7/library/doctest.html#doctest.DocTestSuite
But honestly neither approach makes any sense nowadays as it boils down to:
$ py.test --doctest-modules
or
$ nosetests --with-doctest
Of course coverage and lots of bells & whistles are also supplied by these frameworks and you may keep sticking to unittest.TestCase, and you won't even need to create a coverage_module so I would dig into one of them rather then trying to come up with your own solution
I am writing python selenium tests
if __name__ == "__main__":
unittest.main()
This is my main.py file, I import the tests from a different folder
from tests.AuthorTest import AuthorTest
When I run main.py it gives me a message
python main.py
----------------------------------------------------------------------
Ran 0 tests in 0.000s
for some reason, when I pull the code from git hub I can edit and change existing files but when I add new tests to main.py, it still says Ran 0 tests. Im guessing the main isn't importing it correctly? Been having this problem for a while and I can't seem to fix it.
Appreciate any and all help!
All test functions when using unittest modules should begin with test in your test file.
"A testcase is created by subclassing unittest.TestCase. The three individual tests are defined with methods whose names start with the letters test. This naming convention informs the test runner about which methods represent tests."
From http://docs.python.org/2/library/unittest.html
class TestSequenceFunctions(unittest.TestCase):
def choice(self):
self.assertTrue(False)
will not show any result but when def choice(self) is renamed to def testchoice(self), unittest now will show that one of the tests failed.
I have some test case classes organized in directories
foo_tests
foo_tests1.py
foo_tests2.py
...
bar_tests
bar_tests1.py
...
The test cases look like:
foo_tests1.py:
import unittest
class FooTestsOne(unittest.TestCase):
def test_1():
assert(1=1)
def test_2():
#...
How do you organize test suites out of test case classes like this? There are facilities in unittest for TestLoaders and TestSuite objects but where are they declared and used? What I want is to define certain suites in a separate file that i can run the tests with:
suite1.py
import unittest
import foo_test1
suite = unittest.TestSuite((unittest.makeSuite(foo_tests1.FooTestsOne),
unittest.makeSuite(foo_tests2.FooTeststwo),
))
if __name__ == "__main__":
result = unittest.TextTestRunner(verbosity=2).run(suite())
sys.exit(not result.wasSuccessful())
But this is not the right way to aggregate tests into suites. When I import the testcase class ("import foo_test1") to reference it so I can put it in a larger suite the tests inside are immediately run (during the import). What is the right way to aggregates tests into suites? I need fine grain control as to what tests go into which suites... I've read the TestSuite documentation, but it doesn't seem to provide many examples...
Tests are not supposed to run during import. Maybe you have unittest.main() at the bottom of foo_test1.py?
Your script should work, except that
result = unittest.TextTestRunner(verbosity=2).run(suite())
should be
result = unittest.TextTestRunner(verbosity=2).run(suite)