Develop Reference

How can I integrate doctests with unittest's test discovery?

I wrote a python script to do all my tests automatically for me, and generate a HTML report. I discovered discover for unittests the other day which lets me run all the unittests in a given directory without explicitly naming them, and I'd really like to be able to do my doctests the same way, rather than having to import each module explicitly.
I found some info on how to do this at https://docs.python.org/2/library/doctest.html but didn't really get it. Could you please help me with using discover with my doctests?
Python test discovery with doctests, coverage and parallelism is related, but still doesn't answer my question.
coverage_module
import coverage
import doctest
import unittest
import os
# import test_module
import my_module
cov = coverage.Coverage()
cov.start()
# running doctest by explicity naming the module
doctest.testmod(my_module)
# running unittests by just specifying the folder to look into
testLoad = unittest.TestLoader()
testSuite = testLoad.discover(start_dir=os.getcwd())
runner = unittest.TextTestRunner()
runner.run(testSuite)
cov.stop()
cov.save()
cov.html_report()
print "tests completed"
test_module
import unittest
import doctest
from my_module import My_Class
class My_Class_Tests(unittest.TestCase):
def setUp(self):
# setup variables
def test_1(self):
# test code
# The bit that should load up the doctests? What's loader, tests, and ignore though?
# Is this in the right place?
def load_tests(loader, tests, ignore):
tests.addTests(doctest.DocTestSuite(module_with_doctests))
return tests
if __name__ == '__main__':
unittest.main()

Lets figure out what's happening there
1) unittest.discovery
It has no clue of doctests as doctests is a different framework.
So unittest isn't supposed to discover doctests out of the box.
That means you'll need to glue them together by hand
2) doctest
It's essentially a separate framework although it has some glueing classes to convert doctests into unittest-like TestCases.
https://docs.python.org/2.7/library/doctest.html#doctest.DocTestSuite
3) discover
Didn't get what discover you mean, I suppose it's
python -m unittest discover
If not and you're talking about https://pypi.python.org/pypi/discover then just forget about it - it's a backport for earlier versions of python
4) what to do
either scatter a lot of load_tests hooks across your code as described here https://docs.python.org/2.7/library/doctest.html#unittest-api or code a method to collect all the modules your have in one place and convert them into a DocTestSuite[s] https://docs.python.org/2.7/library/doctest.html#doctest.DocTestSuite
But honestly neither approach makes any sense nowadays as it boils down to:
$ py.test --doctest-modules
or
$ nosetests --with-doctest
Of course coverage and lots of bells & whistles are also supplied by these frameworks and you may keep sticking to unittest.TestCase, and you won't even need to create a coverage_module so I would dig into one of them rather then trying to come up with your own solution

PyCharm - no tests were found?

I've been getting na error in PyCharm and I can't figure out why I'm getting it:
No tests were found
This is what I have for my point_test.py:
import unittest
import sys
import os
sys.path.insert(0, os.path.abspath('..'))
from ..point import Point
class TestPoint(unittest.TestCase):
def setUp(self):
pass
def xyCheck(self,x,y):
point = Point(x,y)
self.assertEqual(x,point.x)
self.assertEqual(y,point.y)
and this point.py, what I'm trying to test:
import unittest
from .utils import check_coincident, shift_point
class Point(object):
def __init__(self,x,y,mark={}):
self.x = x
self.y = y
self.mark = mark
def patched_coincident(self,point2):
point1 = (self.x,self.y)
return check_coincident(point1,point2)
def patched_shift(self,x_shift,y_shift):
point = (self.x,self.y)
self.x,self,y = shift_point(point,x_shift,y_shift)
Is it something wrong with my run configuration? I looked at this SO post but I'm still utterly confused. My run configuration currently looks like this:

In order to recognize test functions, they must be named test_. In your case, rename xyCheck to test_xyCheck :)

I know it's more than a year since the question was asked, but I had the same issue and that post was the first result in search.
As I understood PyCharm (or Intellij Idea Python plugin) needs your test to meet the following criteria if you want it to be launched when you run all the tests in directory.
test functions should start with "test" (underscore is not necessary)
the file, containing the test should also start with "test". "Test" (with capital T doesn't work in my case
I'm using Intellij IDEA 2016.3.5 with Python plugin
If you want to run you tests with command line
python -m unittest
Then you should add __init__.py to test directory. Python still wants your test function names to start with "test", and you test file name to start with "test", but in case of files it doesn't care if the first "t" is capital or not. TestCase and test_case is equally fine.

One thing that can also cause this problem is if you have not selected the right testing framework in your settings:
settings > Python Integrated Tools > Testing > Default test runner
All my tests are in pytest, but it was set to unit test

Don't use dashes ("-") in your filename. These files were being ignored on my machine. renaming them from project-tests.py to project_tests.py solved the problem.

Another gotcha that has just bitten me.
I had a test file within my test package called test_queue.py which followed all of the above advice, however when I selected "Run UnitTests" in PyCharm the console reported no tests found.
The issue in my case was that I had a non-unit test file in the root of the project also called test_queue.py which had been used for some other purpose early on in the project and forgotten about.
Even though I was specifically selecting the test file in my tests folder, and the path was set to absolutely point at the unit test version of the file, it seems the file in the root of the project was being used.
So, one more thing to check, make sure there are no other files in your project with the same name.

Another issue you may want to check is if you see in the console output "## tests deselected by '-k XXXX'". This means you've added a Keyword to the Run/Debug Configuration, and Pycharm will only run tests whose name contains the keyword.

Adding another answer in the hopes that it helps someone down the road. I've been fighting this problem and just figured out the answer (I think). I originally deleted the standard Django file tests.py out of my application folder. I also created a subdirectory of my project called tests, which contains separate test scripts. In this situation, Pycharm failed to find the tests. I corrected this simply by creating an empty file called tests.py in my application folder.
So:
Make sure you have a file called tests.py in your application director (it can be an empty file)
It seems that a folder called tests, in your project, can contain separate test scripts and Pycharm seems to find and run these.
Here's a picture of the directory structure that's working for me:

I had this exception when running individual tests in a Django 1.8 project in PyCharm 2018.1. I could run all the tests together, but individual tests in one file crashed.
The exception was happening in unittest's loader.py
It was getting an ImportError trying to import test_admin_views.py, though the exception was hiding the details of that error.
To see the details of the ImportError, I opened a Python Console and ran:
import my_app.users.tests.test_admin_views
This gave me:
Traceback (most recent call last):
[...]
File "my_app/my_app/users/tests/model_factories.py", line 42, in <module>
from my_app.applications.tests.model_factories import ApplicationFactory
ImportError: cannot import name ApplicationFactory
I noticed that some of the other factories in that file are imported without using the full path, so I tried adding ApplicationFactory to the relative path import list:
from .model_factories import UserFactory, UserGroupFactory, ApplicationFactory
Which worked!

None of these helped me, but I was able to fix the issue by setting the working directory to the project directory.
I debugged this by first creating a new Django test configuration witj a target of one of the app names with tests.
That ran successfully. No idea why working directory should need to be specified.

Test if code is executed from within a py.test session

I'd like to connect to a different database if my code is running under py.test. Is there a function to call or an environment variable that I can test that will tell me if I'm running under a py.test session? What's the best way to handle this?

A simpler solution I came to:
import sys
if "pytest" in sys.modules:
...
Pytest runner will always load the pytest module, making it available in sys.modules.
Of course, this solution only works if the code you're trying to test does not use pytest itself.

There's also another way documented in the manual:
https://docs.pytest.org/en/latest/example/simple.html#pytest-current-test-environment-variable
Pytest will set the following environment variable PYTEST_CURRENT_TEST.
Checking the existence of said variable should reliably allow one to detect if code is being executed from within the umbrella of pytest.
import os
if "PYTEST_CURRENT_TEST" in os.environ:
# We are running under pytest, act accordingly...
Note
This method works only when an actual test is being run.
This detection will not work when modules are imported during pytest collection.

A solution came from RTFM, although not in an obvious place. The manual also had an error in code, corrected below.
Detect if running from within a pytest run
Usually it is a bad idea to make application code behave differently
if called from a test. But if you absolutely must find out if your
application code is running from a test you can do something like
this:
# content of conftest.py
def pytest_configure(config):
import sys
sys._called_from_test = True
def pytest_unconfigure(config):
import sys # This was missing from the manual
del sys._called_from_test
and then check for the sys._called_from_test flag:
if hasattr(sys, '_called_from_test'):
# called from within a test run
else:
# called "normally"
accordingly in your application. It’s also a good idea to use your own
application module rather than sys for handling flag.

Working with pytest==4.3.1 the methods above failed, so I just went old school and checked with:
script_name = os.path.basename(sys.argv[0])
if script_name in ['pytest', 'py.test']:
print('Running with pytest!')

While the hack explained in the other answer (http://pytest.org/latest/example/simple.html#detect-if-running-from-within-a-pytest-run) does indeed work, you could probably design the code in such a way you would not need to do this.
If you design the code to take the database to connect to as an argument somehow, via a connection or something else, then you can simply inject a different argument when you're running the tests then when the application drives this. Your code will end up with less global state and more modulare and reusable. So to me it sounds like an example where testing drives you to design the code better.

This could be done by setting an environment variable inside the testing code. For example, given a project
conftest.py
mypkg/
__init__.py
app.py
tests/
test_app.py
In test_app.py you can add
import os
os.environ['PYTEST_RUNNING'] = 'true'
And then you can check inside app.py:
import os
if os.environ.get('PYTEST_RUNNING', '') == 'true':
print('pytest is running')

Way to access resource files in python

What is the proper way to access resources in python programs.
Basically in many of my python modules I end up writing code like that:
DIRNAME = os.path.split(__file__)[0]
(...)
template_file = os.path.join(DIRNAME, "template.foo")
Which is OK but:
It will break if I will start to use python zip packages
It is boilerplate code
In Java I had a function that did exactly the same --- but worked both when code was lying in bunch of folders and when it was packaged in .jar file.
Is there such function in Python, or is there any other pattern that I might use.

You'll want to look at using either get_data in the stdlib or pkg_resources from setuptools/distribute. Which one you use probably depends on whether you're already using distribute to package your code as an egg.

Since version 3.7 of Python, the proper way to access a file in resources is to use the importlib.resources library.
One can, for example, use the path function to access a particular file in a Python package:
import importlib.resources
with importlib.resources.path("your.package.templates", "template.foo") as template_file:
...
Starting with Python 3.9, this package introduced the files() API, to be preferred over the legacy API.
One can, use the files function to access a particular file in a Python package:
template_res = importlib.resources.files("your.package.templates").joinpath("template.foo")
with importlib.resources.as_file(template_res) as template_file:
...
For older versions, I recommend to install and use the importlib-resources library. The documentation also explains in detail how to migrate your old implementation using pkg_resources to importlib-resources.

Trying to understand how we could combine the two aspect togather
Loading for resources in native filesystem
Packaged in zipped files
Reading through the quick tutorial on zipimport : http://www.doughellmann.com/PyMOTW/zipimport/
I see the following example:
import sys
sys.path.insert(0, 'zipimport_example.zip')
import os
import zipimport
importer = zipimport.zipimporter('zipimport_example.zip')
module = importer.load_module('example_package')
print module.__file__
print module.__loader__.get_data('example_package/README.txt')
I think that output of __file__ is "zipimport_example.zip/example_package/__init__.pyc"
Need to check how it looks from inside.
But then we could always do something like this:
if ".zip" in example_package.__file__:
...
load using get_data
else:
load by building the correct file path
[Edit:] I have tried to work out the example a bit better.
If the the package gets imported as zipped file then, two things happen
__file__ contains ".zip" in it's path.
__loader__ is available in the name space
If these two conditions are met then within the package you could do:
print __loader__.get_data(os.path.join('package_name','README.txt'))
else the module was loaded normally and you can follow the regular approach to loading the file.

I guess the zipimport standard python module could be an answer...
EDIT: well, not the use of the module directly, but using sys.path as shown in the example could be a good way:
I have a zip file test.zip with one python module test and a file test.foo inside
to test that for the zipped python module test can be aware of of test.foo, it contains this code:
c
import os
DIRNAME = os.path.dirname(__file__)
if os.path.exists(os.path.join(DIRNAME, 'test.foo')):
print 'OK'
else:
print 'KO'
Test looks ok:
>>> import sys
>>> sys.path.insert(0, r'D:\DATA\FP12210\My Documents\Outils\SVN\05_impl\2_tools\test.zip')
>>> import test
OK
>>>
So a solution could be to loop in your zip file to retrieve all python modules, and add them in sys.path; this piece of code would be ideally the 1st one loaded by your application.

Imports in Python project with doctests

I have a Python project with following directory structure:
/(some files)
/model/(python files)
/tools/(more python files)
...
So, I have Python files in couple subdirectories and there are some
dependencies between directories as well: tools are used by model, etc. Now
my problem is that I want to make doctests for both models and tools,
and I want be able to run tests from command line like this: ./model/car.py .
I can make this work, but only with messy boilerplate code. I would like
to know what is the correct way, or is there any?
Question: How should I write my imports?
Thanx. Here is an example...
Content of tools/tool.py:
#!/usr/bin/env python
"""
>>> is_four(21)
False
>>> is_four(4)
True
"""
def is_four(val):
return val == 4
if __name__ == '__main__':
import doctest
doctest.testmod()
... and model/car.py:
#!/usr/bin/env python
"""
>>> car = Car()
>>> car.ok()
True
"""
from tools.tool import *
class Car(object):
def __init__(self):
self.tire_count = 4
def ok(self):
return is_four(self.tire_count)
if __name__ == '__main__':
import doctest
doctest.testmod()
By adding following lines in the begin of car.py it works, but doesn't look nice. :(
if __name__ == '__main__':
import sys
import os
sys.path.append(os.path.abspath(os.path.dirname('..')))

What you are trying to do is a relative import. It works fine in Python, but on the module level, not on the file system level. I know, this is confusing.
It means that if you run a script in a subdir, it doesn't see the upper dirs because for the running script, the root of the module is the current dir: there is no upper module.
So what are relative imports for?
Well, module in subdirs car import module in upper dirs as long as they are themself imported from a upperdir.
In your case it means you must run your scripts from "/" so it becomes the root of the module, and the submodules are allowed to use relative import.
A possible solution to your problem is to remove your if __name__ == "__main__" block and create /tests.py:
import doctest
from model import car
from tools import tool
doctest.testmod(car)
doctest.testmod(tool)
Then run in too launch all the tests.
Ultimately you will want to automatize the process, a simple solution is to use unittest so you can create test suites and just add the module names you want to test:
import unittest
import doctest
modules = ("model.car",
"tools.tool")
suite = unittest.TestSuite()
for mod in modules:
suite.addTest(doctest.DocTestSuite(mod))
runner = unittest.TextTestRunner()
runner.run(suite)
Another solution (recommended) is to use a tool such as nose that automates this for you.
easy_install nose
nosetests --with-doctest # done :-)
And by the way, avoid from x import *. This works for quick scripts, but when your program will grow, you really will need to explicitly name what you import. Either import x or from x import y

Use packages. Add an __init__.py file to your working directory and all subfolders then your imports will search the parent directories if it doesn't find the module in the current directory.
See http://www.network-theory.co.uk/docs/pytut/Packages.html
Also this question is a duplicate of:
Import a module from a relative path

Don't frob sys.path in this manner. Instead either use $PYTHONPATH to force the base directory in when invoking python, or use python -m model.car from that directory.