I am attempting to run some unit tests in python from what I believe is a module. I have a directory structure like
TestSuite.py
UnitTests
|__init__.py
|TestConvertStringToNumber.py
In testsuite.py I have
import unittest
import UnitTests
class TestSuite:
def __init__(self):
pass
print "Starting testting"
suite = unittest.TestLoader().loadTestsFromModule(UnitTests)
unittest.TextTestRunner(verbosity=1).run(suite)
Which looks to kick off the testing okay but it doesn't pick up any of the test in TestConvertNumberToString.py. In that class I have a set of functions which start with 'test'.
What should I be doing such that running python TestSuite.py actually kicks off all of my tests in UnitTests?
Here is some code which will run all the unit tests in a directory:
#!/usr/bin/env python
import unittest
import sys
import os
unit_dir = sys.argv[1] if len(sys.argv) > 1 else '.'
os.chdir(unit_dir)
suite = unittest.TestSuite()
for filename in os.listdir('.'):
if filename.endswith('.py') and filename.startswith('test_'):
modname = filename[:-2]
module = __import__(modname)
suite.addTest(unittest.TestLoader().loadTestsFromModule(module))
unittest.TextTestRunner(verbosity=2).run(suite)
If you call it testsuite.py, then you would run it like this:
testsuite.py UnitTests
Using Twisted's "trial" test runner, you can get rid of TestSuite.py, and just do:
$ trial UnitTests.TestConvertStringToNumber
on the command line; or, better yet, just
$ trial UnitTests
to discover and run all tests in the package.
Related
I have my Python unittest script like below. It takes an argument '-a' to determine whether the testcases should load the base module from foo_PC_A.py or foo_PC_B.py. I use shutil.move() to rename either .py file to foo.py, so all the testcase modules (e.g. tm1.py, tm2.py) can simply import foo. Though this looks like a workaround and not Pythonic. Is there any better way to do this? Or a better design to fundamentally resolve this issue.
(run_all_unittest.py)
if sys.argv[1] = '-a':
shutil.move('foo_PC_A.py', 'foo.py')
else:
shutil.move('foo_PC_B.py', 'foo.py')
test_module_list = ['tm1', 'tm2', ...]
for test_module_name in test_module_list:
test_module = __import__(test_module_name)
test_suites.append(unittest.TestLoader().loadTestsFromModule(test_module))
alltests = unittest.TestSuite(test_suites)
unittest.TextTestRunner().run(alltests)
if sys.argv[1] = '-a':
shutil.move('foo.py', 'foo_PC_A.py')
else:
shutil.move('foo.py', 'foo_PC_B.py')
(tm1.py)
from foo import MyTestCase
...
(foo_PC_A.py)
import <some module only available on PC A>
class MyTestCase(unittest.TestCase):
...
(foo_PC_B.py)
# since the PC A modules are not available on PC B,
# just call the pre-built executable via subprocess
import subprocess
class MyTestCase(unittest.TestCase):
...
def test_run(self):
subprocess.call(...)
You can fool Python into thinking the module has already been loaded. Just import the module dynamically and use sys.modules:
import sys
import importlib
if sys.argv[1] = '-a':
sys.modules['foo'] = importlib.import_module('foo_PC_A')
else:
sys.modules['foo'] = importlib.import_module('foo_PC_A')
When any module runs import foo or from foo import ..., Python will use that path.
Note that if foo is moved to a package, the full Python path must be specified, as in:
sys.modules['path.to.foo'] = ...
Lets say my code looks like this
import pytest
import json
#pytest.fixture
def test_item():
test_item = json.load(open('./directory/sample_item_for_test.json','rb'))
return test_item
def test_fun(test_document):
assert type(test_item.description[0]) == unicode
And I would like to run this test via Py.Test
If I run Py.test from the directory that it is in, it is fine. BUT if I run it from an above directory, it fails due to not being able to find 'sample_item_for_test.json'. Is there a way to make this test run correctly no matter where I execute Py.test?
The magic attribute __file__ is the path to the python file on the filesystem. So, you can use that with some magic from os to get the current directory...
import pytest
import json
import os
_HERE = os.path.dirname(__file__)
_TEST_JSON_FILENAME = os.path.join(_HERE, 'directory', 'sample_item_for_test.json')
#pytest.fixture
def test_item():
with open(_TEST_JSON_FILENAME, 'rb') as file_input:
return json.load(file_input)
When I migrated to py.test, I had a large set of legacy tests that were accustomed to being executed in the directory where the test file lives. Instead of tracking down every test failure, I added a pytest hook to my conftest.py to chdir to the test directory before each test starts:
import os
import functools
def pytest_runtest_setup(item):
"""
Execute each test in the directory where the test file lives.
"""
starting_directory = os.getcwd()
test_directory = os.path.dirname(str(item.fspath))
os.chdir(test_directory)
teardown = functools.partial(os.chdir, starting_directory)
# There's probably a cleaner way than accessing a private member.
item.session._setupstate.addfinalizer(teardown, item)
I've got the following tests.py file:
from django.test import TestCase
from lxml import etree
import tempfile
import utils
class CreateSurveyFromCsvTextTests(TestCase):
def parsesSurveyPassedInAsCsvAndReturnsXmlRepresentation(self):
text = """"survey",,,,,
,"name","type","label","hint","required"
,"gps","geopoint","Record your current location",,"false"
,"start","start",,,
,"end","end",,,
"settings",
,"form_title"
,"New survey" """
xml = create_survey_from_csv_text(text)
lxml.fromstring(xml)
when I run my module with python manage.py test, the output is
Ran 0 tests in 0.000s
I know the runner is picking up the file because if I make an invalid import it throws an error.
Why is the test not being called?
The name of the test methods need to start with test_. This allows the class to have both test methods and helper methods that you may write as well.
Hence you should rename your method to test_parsesSurveyPassedInAsCsvAndReturnsXmlRepresentation (and perhaps shorten the name too).
I'm trying to figure out how to get python setup.py test to run the equivalent of python -m unittest discover. I don't want to use a run_tests.py script and I don't want to use any external test tools (like nose or py.test). It's OK if the solution only works on python 2.7.
In setup.py, I think I need to add something to the test_suite and/or test_loader fields in config, but I can't seem to find a combination that works correctly:
config = {
'name': name,
'version': version,
'url': url,
'test_suite': '???',
'test_loader': '???',
}
Is this possible using only unittest built into python 2.7?
FYI, my project structure looks like this:
project/
package/
__init__.py
module.py
tests/
__init__.py
test_module.py
run_tests.py <- I want to delete this
setup.py
Update: This is possible with unittest2 but I want find something equivalent using only unittest
From https://pypi.python.org/pypi/unittest2
unittest2 includes a very basic setuptools compatible test collector. Specify test_suite = 'unittest2.collector' in your setup.py. This starts test discovery with the default parameters from the directory containing setup.py, so it is perhaps most useful as an example (see unittest2/collector.py).
For now, I'm just using a script called run_tests.py, but I'm hoping I can get rid of this by moving to a solution that only uses python setup.py test.
Here's the run_tests.py I'm hoping to remove:
import unittest
if __name__ == '__main__':
# use the default shared TestLoader instance
test_loader = unittest.defaultTestLoader
# use the basic test runner that outputs to sys.stderr
test_runner = unittest.TextTestRunner()
# automatically discover all tests in the current dir of the form test*.py
# NOTE: only works for python 2.7 and later
test_suite = test_loader.discover('.')
# run the test suite
test_runner.run(test_suite)
If you use py27+ or py32+, the solution is pretty simple:
test_suite="tests",
From Building and Distributing Packages with Setuptools (emphasis mine):
test_suite
A string naming a unittest.TestCase subclass (or a package or module
containing one or more of them, or a method of such a subclass), or naming
a function that can be called with no arguments and returns a unittest.TestSuite.
Hence, in setup.py you would add a function that returns a TestSuite:
import unittest
def my_test_suite():
test_loader = unittest.TestLoader()
test_suite = test_loader.discover('tests', pattern='test_*.py')
return test_suite
Then, you would specify the command setup as follows:
setup(
...
test_suite='setup.my_test_suite',
...
)
You don't need config to get this working. There are basically two main ways to do it:
The quick way
Rename your test_module.py to module_test.py (basically add _test as a suffix to tests for a particular module), and python will find it automatically. Just make sure to add this to setup.py:
from setuptools import setup, find_packages
setup(
...
test_suite = 'tests',
...
)
The long way
Here's how to do it with your current directory structure:
project/
package/
__init__.py
module.py
tests/
__init__.py
test_module.py
run_tests.py <- I want to delete this
setup.py
Under tests/__init__.py, you want to import the unittest and your unit test script test_module, and then create a function to run the tests. In tests/__init__.py, type in something like this:
import unittest
import test_module
def my_module_suite():
loader = unittest.TestLoader()
suite = loader.loadTestsFromModule(test_module)
return suite
The TestLoader class has other functions besides loadTestsFromModule. You can run dir(unittest.TestLoader) to see the other ones, but this one is the simplest to use.
Since your directory structure is such, you'll probably want the test_module to be able to import your module script. You might have already done this, but just in case you didn't, you could include the parent path so that you can import the package module and the module script. At the top of your test_module.py, type:
import os, sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
import unittest
import package.module
...
Then finally, in setup.py, include the tests module and run the command you created, my_module_suite:
from setuptools import setup, find_packages
setup(
...
test_suite = 'tests.my_module_suite',
...
)
Then you just run python setup.py test.
Here is a sample someone made as a reference.
One possible solution is to simply extend the test command for distutilsand setuptools/distribute. This seems like a total kluge and way more complicated than I would prefer, but seems to correctly discover and run all the tests in my package upon running python setup.py test. I'm holding off on selecting this as the answer to my question in hopes that someone will provide a more elegant solution :)
(Inspired by https://docs.pytest.org/en/latest/goodpractices.html#integrating-with-setuptools-python-setup-py-test-pytest-runner)
Example setup.py:
try:
from setuptools import setup
except ImportError:
from distutils.core import setup
def discover_and_run_tests():
import os
import sys
import unittest
# get setup.py directory
setup_file = sys.modules['__main__'].__file__
setup_dir = os.path.abspath(os.path.dirname(setup_file))
# use the default shared TestLoader instance
test_loader = unittest.defaultTestLoader
# use the basic test runner that outputs to sys.stderr
test_runner = unittest.TextTestRunner()
# automatically discover all tests
# NOTE: only works for python 2.7 and later
test_suite = test_loader.discover(setup_dir)
# run the test suite
test_runner.run(test_suite)
try:
from setuptools.command.test import test
class DiscoverTest(test):
def finalize_options(self):
test.finalize_options(self)
self.test_args = []
self.test_suite = True
def run_tests(self):
discover_and_run_tests()
except ImportError:
from distutils.core import Command
class DiscoverTest(Command):
user_options = []
def initialize_options(self):
pass
def finalize_options(self):
pass
def run(self):
discover_and_run_tests()
config = {
'name': 'name',
'version': 'version',
'url': 'http://example.com',
'cmdclass': {'test': DiscoverTest},
}
setup(**config)
Another less than ideal solution slightly inspired by http://hg.python.org/unittest2/file/2b6411b9a838/unittest2/collector.py
Add a module that returns a TestSuite of discovered tests. Then configure setup to call that module.
project/
package/
__init__.py
module.py
tests/
__init__.py
test_module.py
discover_tests.py
setup.py
Here's discover_tests.py:
import os
import sys
import unittest
def additional_tests():
setup_file = sys.modules['__main__'].__file__
setup_dir = os.path.abspath(os.path.dirname(setup_file))
return unittest.defaultTestLoader.discover(setup_dir)
And here's setup.py:
try:
from setuptools import setup
except ImportError:
from distutils.core import setup
config = {
'name': 'name',
'version': 'version',
'url': 'http://example.com',
'test_suite': 'discover_tests',
}
setup(**config)
Python's standard library unittest module supports discovery (in Python 2.7 and later, and Python 3.2 and later). If you can assume those minimum versions, then you can just add the discover command line argument to the unittest command.
Only a small tweak is needed to setup.py:
import setuptools.command.test
from setuptools import (find_packages, setup)
class TestCommand(setuptools.command.test.test):
""" Setuptools test command explicitly using test discovery. """
def _test_args(self):
yield 'discover'
for arg in super(TestCommand, self)._test_args():
yield arg
setup(
...
cmdclass={
'test': TestCommand,
},
)
This won't remove run_tests.py, but will make it work with setuptools. Add:
class Loader(unittest.TestLoader):
def loadTestsFromNames(self, names, _=None):
return self.discover(names[0])
Then in setup.py: (I assume you're doing something like setup(**config))
config = {
...
'test_loader': 'run_tests:Loader',
'test_suite': '.', # your start_dir for discover()
}
The only downside I see is it's bending the semantics of loadTestsFromNames, but the setuptools test command is the only consumer, and calls it in a specified way.
I have 2 files in a directory: loader.py & mod1.py. Loader.py dynamically instanciates a class in mod1.py and calls a method on it. Here is mod1.py
class MyClass1:
def run(self):
print "Class1 running"
Here is loader:
def run():
mod = __import__('mod1')
cls = getattr(mod, 'MyClass1')
inst = cls()
inst.run()
run()
If I run this just straight python: "python loader.py" I see:
Class1 running
Which is what you expect. If I then run it under fabric: "fab -f loader.py run" I see
Class1 running
Class1 running
Done.
Which makes sense, run() is called by fabric AND loader.py when it's loaded by fabric.
However, if I remove the explicit call to run in loader.py so it's only called once under fabric, I get
ImportError: No module named mod1
Why does running under fabric make a difference? Is there a way to make this work under fabric?
Here's my insight.
This weird behavior is explained in docs, quote:
Fabric does a normal import (actually an __import__) of your fabfile
in order to access its contents – it does not do any eval-ing or
similar. In order for this to work, Fabric temporarily adds the found
fabfile’s containing folder to the Python load path (and removes it
immediately afterwards.)
Look at the modified loader.py:
import os
import sys
print sys.path # current dir is in sys.path
def run():
print sys.path # current dir is NOT in sys.path
# no problem - we'll just add it
sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
mod = __import__('mod1')
cls = getattr(mod, 'MyClass1')
inst = cls()
inst.run()
Looks ugly, but it fixes the problem.
Here's how fabric does manipulations with imports while loading fab file. Also this issue is relevant.
Hope that helps.