I'm trying to use nosetests to run my tests in a directory structure like this
src
- file1.py
- ...
test
- helper.py
- test_file1.py
As you can see, test_file1.py has some functions that test file1.py, so it imports file1.py like this:
# In file1.py
import file1
import helper
# Tests go here...
I also use a helper.py file that has some neat functionality built in so that I can create tests more easily. This functionality is achieved by extending a couple of classes in my actual code and overriding some methods. So helper.py looks something like this:
# In helper.py
import file1
# Use stuff in file1.py
I'm having trouble understanding how nose goes about importing these things with its custom importer. I was able to get my test file to import file1.py by running nosetest ../tests within the src directory, but I'm currently getting an error akin to:
File helper.py:
ImportError: cannot import name file1
How does nose do its imports and is there a way I can essentially get it to lump all my tests/src files together so they can all import one another while I keep them in separate folders?
Seeing that you execute tests with nosetests ../tests I assume they are executed from the tests folder itself. Therefore, files from the src directory are not added to sys.path, hence the error.
To fix this one could:
run tests from the parent directory - nosetests will be able to identify src and test (or tests) directory by himself and will add them to the sys.path before running tests
add src directory path to the PYTHONPATH before running nosetests (export PYTHONPATH=../src; nosetests)
Note that you can as well omit the last argument to the nosetests as by default it runs the tests from current directory. Otherwise, if the tests are not in the directory you launch nosetests from, you can define its location with --where=<path-to-tests> parameter (or, simply -w). So for example you can execute tests from src direcotory and without even setting the PYTHONPATH (because current directory will be added to sys.path by default) like this: nosetests -w ../tests.
Lastly, even though this is very questionable by itself, and yet: the most common way to organize a Python source code is having python files and packages starting directly in the project directory, and having tests in "test" sub-packages of the packages they test. So, in your case it would be:
/file1.py
/test/helper.py
/test/test_file1.py
or better:
/myproject/__init__.py
/myproject/file1.py
/myproject/test/__init__.py
/myproject/test/helper.py
/myproject/test/test_file1.py
(latter, provided you also use correct imports in your test sources, e.g. from .. import file1).
In which case one runs tests from the project's root directory simply with nosetests without any argument.
Anyway, nosetests is flexible enough to work with any structure - use whatever seems more suitable for you and the project.
More on project structure in What is the best project structure for a Python application?
This seems generally like an issue I had with nose tests:
Importing with Python and Nose Tests
The work around I found was to insert a try..except block so that BOTH python and nosetest commands will work on the same directory as follows:
(1) In your main file, at the very top before anything else add:
# In file1.py
try:
# This will allow you to do python file1.py inside the src directory
from file2 import *
from helper import *
except:
# This will allow you to run nosetests in the directory just above
# the src and test directories.
from src.file1 import *
from src.helper import *
(2) Inside your test.py file add:
from src.file2 import *
from src.helper import *
Related
I encountered a strange issue with unit tests in a namespaced package. Here's an example I built on GitHub. Here's the basic structure:
$ tree -P '*.py' src
src
└── namespace
└── testcase
├── __init__.py
├── a.py
├── sub
│ ├── __init__.py
│ └── b.py
└── tests
├── __init__.py
└── test_imports.py
4 directories, 6 files
I would expect that relative imports within a namespaced package would maintain the namespace. Normally, that seems to be true:
$ cat src/namespace/testcase/a.py
print(__name__)
$ cat src/namespace/testcase/sub/b.py
print(__name__)
from ..a import *
$ python -c 'from namespace.testcase.sub import b'
namespace.testcase.sub.b
namespace.testcase.a
But if I involve a test, I get a surprise:
$ cat src/namespace/testcase/tests/test_imports.py
from namespace.testcase import a
from ..sub import b
$ python -m unittest discover src/namespace/
namespace.testcase.a
testcase.sub.b
testcase.a
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
The code in src/namespace/testcase/a.py is getting run twice! In my case, this caused a singleton I had stubbed to be re-initialized as a real object, subsequently causing test failures.
Is this expected behavior? What is the correct usage here? Should I always avoid relative imports (and have to do global search-and-replace if my company decides to rename something?)
Problem: Overlapping sys.path entries
The duplicate imports with different module names happen when you have overlapping sys.path entries: that is, when sys.path contains both a parent and child directory as separate entries. This situation is almost always an error: it will make Python see the child directory as a separate, unrelated root for imports, which leads surprising behaviour.
In your example:
$ python -m unittest discover src/namespace/
namespace.testcase.a
testcase.sub.b
testcase.a
This means that both src and src/namespace ended up in sys.path, so that:
namespace.testcase.a was imported relative to src
testcase.sub.b and testcase.a were imported relative to src/namespace
Why?
In this case, the overlapping sys.path entries happen because unittest discover is trying to be helpful: it defaults to assuming that the start directory for test discovery is also the top-level directory that your imports are relative to, and it will insert that top-level directory into sys.path if it's not already there, as a convenience. (…not so convenient, it turns out. 😔️)
Solution: Explicitly specify the correct top-level directory
You can explicitly specify the correct top-level directory with -t (--top-level-directory):
python -m unittest discover -t src -s src/namespace/
This will work as before, but won't treat src/namespace as a top-level directory to insert into sys.path.
Side note: The -s option prefix for src/namespace/ was implicit in the previous example: the above just makes it explicit.
(unittest discover has weird positional argument handling: it treats its first three positional arguments as values for -s, -p, and -t, in that order.)
Details
The code responsible for this lives in unittest/loader.py:
class TestLoader(object):
def discover(self, start_dir, pattern='test*.py', top_level_dir=None):
...
if top_level_dir is None:
set_implicit_top = True
top_level_dir = start_dir
top_level_dir = os.path.abspath(top_level_dir)
if not top_level_dir in sys.path:
# all test modules must be importable from the top level directory
# should we *unconditionally* put the start directory in first
# in sys.path to minimise likelihood of conflicts between installed
# modules and development versions?
sys.path.insert(0, top_level_dir)
...
Not sure exactly why unittest wouldn't respect your setup.py, but indeed often it does not (maybe a bug, or a difficulty in doing so for the implementers). Or perhaps unittest is by design very "low level" and does not come with any bells or whistles you'd expect from something like pytest.
What you need to do is help unittest out and tell it where your package starts, use the --top-level-directory option for that (or -t for short).
This should work as you expect:
python -m unittest discover -t src/ src/namespace/
The issue is that you probably have something like this in your setup.py:
package_dir={"": "src"},
And unfortunately unittest is not "smart enough" to figure that out.
This is one example detail why I strongly prefer pytest to std-lib's unittest :)
pytest will go to greater lengths to "do the right thing", while not forcing you to be verbose in your test run invocation (for example: it auto-discovers recursively by default etc).
If you want to learn more about how unittest imports things, you can add this line to your a.py file:
assert __package__ == "namespace.testcase"
Then, run your test without the -t src/ as you originally did -> you will see exactly where unittest is crashing. If you open that code, you will see that all it does is try to simply __import__(name), where name is simply the thing it just found that could look like a test.
Tests are usually NOT in a package, a more strict project layout would be like:
src/namespace/ # -> your project or lib
tests/ # -> your tests
The above is "more strict" because it makes it harder to confuse your tests with your actual shipped code (ie: no oopsie import ..tests.foo from the actual code).
Now, given this, a lot of testing tools like unittest and pytest, will kind of assume that your tests don't really have a package, so they will import them as-if the package doesn't matter at all...
Ie: they won't necessarily try and import test_foo.py as-if it was under your main top-level name.
So, in theory you should (from my experience writing tests):
use relative imports from within your actual code only (ie: any non-test submodule)
use full absolute import from the tests (that simplifies quite a few things for testing tools + it allows to treat your code "less intimately" from the tests -> kinda forces you to import stuff from your namespace project like any other user would do)
Hope that helps. I don't have handy links to docs on this (and maybe it would be worth a good book). But consider this: if you write this from your test:
from ..sub import b
You are taking shortcuts a user of your library cannot do. Anyone who would pip install namespace for example would have to import b with an absolute import:
from namespace.sub import b
It is helpful I find to isolate tests from the code itself. I know many projects do just add a tests/ subfolder to their main code tree, but I do find that odd, since that ships the tests together with the published package, and one could technically import the tests just like the rest of the code... for example:
from namespace.testcase.tests import test_imports
An example of tests/ outside the main code tree is the requests package.
Followed the code, as this got me curious.
unittest discover looks for test cases, it finds testcase/ which looks like a test folder to it.
So it simply does a "standalone" (ie: regardless of any "top-level" context) import testcase.
Then your test does this (all of these imports are simply cached in sys.modules, by name):
from namespace.testcase import a, which triggers the import of a as a submodule of namespace.testcase as expected
but then it calls from ..sub import b, now in unittest's context, this expands to testcase.sub.b, which then leads to the confusion.
I am trying to improve the project structure while adding to a code base. I found a sample structure here which looks like this:
README.rst
LICENSE
setup.py
requirements.txt
sample/__init__.py
sample/core.py
sample/helpers.py
docs/conf.py
docs/index.rst
tests/test_basic.py
tests/test_advanced.py
I notice in particular that requirements.txt and setup.py are on a higher level than tests/ and sample/
If I add sample/classes.py you need only write from classes import MyClass in sample/core.py to get it in there. It cannot however so easily be imported into tests/test_basic.py, does not seem like python 'looks around the corner' like that when importing.
In my case, there is also a MANIFEST.in on the same level with requirements.txt and some files which are not really python but just set things up for the platform on which this runs.
If classes.py were on the same level as requirements.txt I think it would be easily importable by everything in tests/ and in sample/ and their subdirectories, but it may need a __init__.py That doesn't feel right somehow.
So where should it go if both tests/ and sample/ need to be able to use it?
Let's make it easy.
If I understand correctly, the problem is How to import simple module in test. Which means you want to use something like from simple.classes import MyClass.
That's easy, just add your root path to PYTHONPATH before executing python test/test_basic.py.
That's also what an IDE does for you when you execute tests through it.
Assuming you use a Python >= 3.3, you can simply turn the test folder in a package by adding a __init__.py module in it. Then in that __init__.py (and only there) you add the path of the parent package to sys.path. That if enough for unittest discover to use it for all the modules in tests.
My one is just:
import os
import sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
Then if you need to access classes.py from one of the test modules, you can just use:
from sample import classes
or to directly import MyClass:
from sample.classes import MyClass
It just works because sample is already a package, and its parent folder has been added to sys.path when python unittest has loaded the test package.
Of course, this only works in you can have your tests in a package. If for any reason it is not an option, for example because you need to run individually the test modules, then you should put the sys.path modification directly in all the test files.
Write a path_helper.py file in the tests folder:
import os
import sys
core_path = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
if core_path not in sys.path: # don't add it if it is already here
sys.path.append(core_path)
You can then import it in all test files:
import path_helper
...
I have a directory structure as follows:
DirA
__init__.py
MyClass.py
unittests <------------------directory
MyClassTest.py
MyClassTest.py is executable:
import unittest
from . import MyClass
class MyClassTestCase(unittest.TestCase):
""" Testcase """
...
.....
if __name__ == '__main__':
unittest.main()
I get an error "Parent module '' not loaded, cannot perform relative import" at the line:
from . import MyClass
I would like to place unittests in a 'unittests' directory beside the modules being tested. Is there a way to do this and have access to all the modules in the parent directory which I am testing?
Have you tried running the tests like so:
cd DirA
python -m unittest discover unittests "*Test.py"
This should find your modules correctly. See Test Discovery
Use whatever layout you want, depending on your own preferences and the way you want your module to be imported:
http://python-notes.curiousefficiency.org/en/latest/python_concepts/import_traps.html#the-double-import-trap
https://pytest.org/latest/goodpractises.html
To find your unittests folder, since the name is not the conventional one (unit test scripts by default look for a test folder), you can use the discover option of the unittest module to tell how to find your test scripts:
python -m unittest discover unittests
Note that the first unittest is the Python module, and the second unittests (with an s) is your directory where you have placed your testing scripts.
Another alternative is to use the nosetest module (or other new unit testing modules like pytest or tox) which should automatically find your testing script, wherever you place them:
nosetests -vv
And to fix your import error, you should use the full relative (or absolute) path:
from ..MyClass import MyClass # Relative path from the unittests folder
from MyClass import MyClass # Absolute path from the root folder, which will only work for some unit test modules or if you configure your unit test module to run the tests from the root
A suggested structure, would be to look at your structure like this:
my_app
my_pkg
__init__.py
module_foo.py
test
__init__.py
test_module_foo.py
main.py
Run everything from within my_app, this way you will use all the same module references between your test code and core code.
I have a bunch of unittest test cases in separate directories. There is also a directory which just contains helper scripts for the tests. So my file tree looks like this
test_dir1
test_dir2
test_dir3
helper_scripts
Each python file in test_dir* will have these lines:
import sys
sys.path.append('../helper_scripts')
import helper_script
This all works fine, as long as I run the tests from within their directory. However, I would like to be at the project root and just run:
py.test
and have it traverse all the directories and run each test it finds. The problem is that the tests are being run from the wrong directory, so the sys.path.append doesn't append the helper_scripts directory, it appends the parent of the project root. This makes all the imports fail with an Import Error.
Is there a way to tell py.test to run the test scripts from their directory? ie. change the cwd before executing them? If not, is there another test runner I can use that will?
What I usually do is structure my project like this:
myproject/
setup.py
myproject/
__init__.py
mymodule.py
tests/
__init__.py
test_dir1/
test_mymodule.py
helper_scripts/
__init__.py
helper_script.py
For running tests, I use a virtualenv with myproject installed in development mode using one of the following commands in the myproject root directory:
pip install -e .
python setup.py develop
Now in test_mymodule.py I can just say
from myproject.tests.helper_scripts import helper_script
I can then just run pytest and there's no need to change the working directory in tests at all.
See Pytest's Good Integration Practices for a great summary of pros and cons for different project directory structures.
os.chdir("newdir")
will change your current working directory
I would suggest that you instead configure your environment so that import helper_scripts will work regardless of the current directory. This is the recommended approach.
If you absolutely must though, you can use relative imports instead:
from .. import helper_script
I just got set up to use pytest with Python 2.6. It has worked well so far with the exception of handling "import" statements: I can't seem to get pytest to respond to imports in the same way that my program does.
My directory structure is as follows:
src/
main.py
util.py
test/
test_util.py
geom/
vector.py
region.py
test/
test_vector.py
test_region.py
To run, I call python main.py from src/.
In main.py, I import both vector and region with
from geom.region import Region
from geom.vector import Vector
In vector.py, I import region with
from geom.region import Region
These all work fine when I run the code in a standard run. However, when I call "py.test" from src/, it consistently exits with import errors.
Some Problems and My Solution Attempts
My first problem was that, when running "test/test_foo.py", py.test could not "import foo.py" directly. I solved this by using the "imp" tool. In "test_util.py":
import imp
util = imp.load_source("util", "util.py")
This works great for many files. It also seems to imply that when pytest is running "path/test/test_foo.py" to test "path/foo.py", it is based in the directory "path".
However, this fails for "test_vector.py". Pytest can find and import the vector module, but it cannot locate any of vector's imports. The following imports (from "vector.py") both fail when using pytest:
from geom.region import *
from region import *
These both give errors of the form
ImportError: No module named [geom.region / region]
I don't know what to do next to solve this problem; my understanding of imports in Python is limited.
What is the proper way to handle imports when using pytest?
Edit: Extremely Hacky Solution
In vector.py, I changed the import statement from
from geom.region import Region
to simply
from region import Region
This makes the import relative to the directory of "vector.py".
Next, in "test/test_vector.py", I add the directory of "vector.py" to the path as follows:
import sys, os
sys.path.append(os.path.realpath(os.path.dirname(__file__)+"/.."))
This enables Python to find "../region.py" from "geom/test/test_vector.py".
This works, but it seems extremely problematic because I am adding a ton of new directories to the path. What I'm looking for is either
1) An import strategy that is compatible with pytest, or
2) An option in pytest that makes it compatible with my import strategy
So I am leaving this question open for answers of these kinds.
The issue here is that Pytest walks the filesystem to discover files that contain tests, but then needs to generate a module name that will cause import to load that file. (Remember, files are not modules.)
Pytest comes up with this test package name by finding the first directory at or above the level of the file that does not include an __init__.py file and declaring that the "basedir" for the module tree containing a module generated from this file. It then adds the basedir to sys.path and imports using the module name that will find that file relative to the basedir.
There are some implications of this of which you should beware:
The basepath may not match your intended basepath in which case the module will have a name that doesn't match what you would normally use. E.g., what you think of as geom.test.test_vector will actually be named just test_vector during the Pytest run because it found no __init__.py in src/geom/test/ and so added that directory to sys.path.
You may run into module naming collisions if two files in different directories have the same name. For example, lacking __init__.py files anywhere, adding geom/test/test_util.py will conflict with test/test_util.py because both are loaded as import test_util.py, with both test/ and geom/test/ in the path.
The system you're using here, without explicit __init__.py modules, is having Python create implicit namespace packages for your directories. (A package is a module with submodules.) Ideally we'd configure Pytest with a path from which it would also generate this, but it doesn't seem to know how to do that.
The easiest solution here is simply to add empty __init__.py files to all of the subdirectories under src/; this will cause Pytest to import everything using package/module names that start with directory names under src/.
The question How do I Pytest a project using PEP 420 namespace packages? discusses other solutions to this.
import looks in the following directories to find a module:
The home directory of the program. This is the directory of your root script. When you are running pytest your home directory is where it is installed (/usr/local/bin probably). No matter that you are running it from your src directory because the location of your pytest determines your home directory. That is the reason why it doesn't find the modules.
PYTHONPATH. This is an environment variable. You can set it from the command line of your operating system. In Linux/Unix systems you can do this by executing: 'export PYTHONPATH=/your/custom/path' If you wanted Python to find your modules from the test directory you should include the src path in this variable.
The standard libraries directory. This is the directory where all your libraries are installed.
There is a less common option using a pth file.
sys.path is the result of combining the home directory, PYTHONPATH and the standard libraries directory. What you are doing, modifying sys.path is correct. It is something I do regularly. You could try using PYTHONPATH if you don't like messing with sys.path
If you include an __init__.py file inside your tests directory, then when the program is looking to set a home directory it will walk 'upwards' until it finds one that does not contain an init file. In this case src/.
From here you can import by saying :
from geom.region import *
you must also make sure that you have an init file in any other subdirectories, such as the other nested test directory
I was wondering what to do about this problem too. After reading this post, and playing around a bit, I figured out an elegant solution. I created a file called "test_setup.py" and put the following code in it:
import sys, os
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
I put this file in the top-level directory (such as src). When pytest is run from the top-level directory, it will run all test files including this one since the file is prefixed with "test". There are no tests in the file, but it is still run since it begins with "test".
The code will append the current directory name of the test_setup.py file to the system path within the test environment. This will be done only once, so there are not a bunch of things added to the path.
Then, from within any test function, you can import modules relative to that top-level folder (such as import geom.region) and it knows where to find it since the src directory was added to the path.
If you want to run a single test file (such as test_util.py) instead of all the files, you would use:
pytest test_setup.py test\test_util.py
This runs both the test_setup and test_util code so that the test_setup code can still be used.
Are so late to answer that question but usining python 3.9 or 3.10 u just need to add __init__.py folder in tests folders.
When u add this file python interprets this folders as a module.
Wold be like this
src/
main.py
util.py
test/
__init__.py
test_util.py
geom/
vector.py
region.py
test/
__init__.py
test_vector.py
test_region.py
so u just run pytest.
Sorry my poor english
Not the best solution, but maybe the fastest one:
cd path/python_folder
python -m pytest python_file.py