I'm trying to run my unit tests for a python module from a separate test script. Here is my file structure
- root
|- modules
|- a_module.py
|- test
|- test_a_module.py
|- main.py
The main.py looks like this:
import unittest
loader = unittest.TestLoader()
suite = loader.discover(start_dir='./test', pattern='test_*.py')
runner = unittest.TextTestRunner()
runner.run(suite)
And here are a_module.py and test_a_module.py:
# a_module.py
def something():
return True
# test_a_module.py
import unittest
from ..modules.a_module import something
class TestSomething(unittest.TestCase):
def test_something(self):
self.assertTrue(something)
When running python3 main.py I get the following error.
======================================================================
ERROR: test_a_module (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_a_module
Traceback (most recent call last):
File "/usr/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
module = self._get_module_from_name(name)
File "/usr/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
__import__(name)
File "/home/schu_max/root/test/test_a_module.py", line 3, in <module>
from ..modules.a_module import something
ImportError: attempted relative import with no known parent package
----------------------------------------------------------------------
I'm relatively new to python and don't have a clue on how to solve this issue. And other questions/answers on SO weren't that helpful either. So how can I get this to work?
There are several ways to make your tests run. Now that they are import packages try
python -m unittest -v in the root package folder, after removing the two dots from the relative import path in test/test_a_module.py. Will work too with python main.py from the same folder. If you issue python -m unittest -v outside root and you prefix the relative path with root it will work too. There are probably other ways.
Just beaware that the relative import paths in the test files are affected by the folder from where you run your test script.
To stay coherent, run your tests always from the same place, and put all your tests inside test/ where you can refer all you root subpackages by relative path, without the need to mention the `root* package name, which is nice because you might want to rename it in the future.
Related
Setup
test/
main.py
pkg/
a.py
__init__.py
main.py contains:
import pkg
pkg.a
__init__.py contains:
from . import a
main.py can be run without errors.
Question
Changing the content of __init__.py to
import a
gives the following error when running main.py:
Traceback (most recent call last):
File "C:/Users/me/PycharmProjects/test/main.py", line 1, in <module>
import pkg
File "C:\Users\me\PycharmProjects\test\pkg\__init__.py", line 1, in <module>
import a
ModuleNotFoundError: No module named 'a'
Interestingly, __init__.py can be executed directly with python __init__.py without errors.
What's going on?
When you run a python script, it's parent folder is added to sys.path
run main.py: sys.path[0] = '../test'
run init.py: sys.path[0] = '../test/pkg'
Your case: You try to "absolute-like" import a in __init__.py but the parent folder of a.py - which is '../test/pkg' - is not in the sys.path when you run main.py. This is why you get an error. However, your absolute import is incomplete as it should always start at the top level folder, e.g.
from test.pkg import a
Final answer to your question: You don't have to use relative imports!
See: PEP-8: Absolute imports are recommended, as they are usually more readable and tend to be better behaved (or at least give better error messages) if the import system is incorrectly configured (such as when a directory inside a package ends up on sys.path).
And keep in mind that relative imports don't work in a top-level-script when __name__ = "__main__", but from imported modules only.
You can learn more about absolute and relative imports here:
Absolute vs. explicit relative import of Python module
https://realpython.com/absolute-vs-relative-python-imports/
I suppose you are using Pycharm? Then that's one of the confusion cause.
For example, let's say your directory looks like this
project1
p1.py
test/
__init__.py
main.py
pkg/
a.py
__init__.py
If you run (F10) the main.py your default working directory will be project1/test, which does not contain the a.py so import a will not find anything.
But if you run (F10) the pkg/__init__.py your working directory will be project1/test/pkg which has the a.py, and it works like what you tested.
So in these situation, if you use from . import a it will look for the directory that file is, project1/test/pkg in this case, which will always work regardless your working directory.
I have the following layout:
/spamalot
/spam
__init__.py
spam.py
/spam_on_eggs
__init__.py
spam_on_eggs.py
/tests
test_spam.py
Spam just so happens to be a flask application.
Within spam.py I have
import spam_on_eggs.spam_on_eggs as eggs
# Other Flask setup & application code here.
And this works fine - from the spamalot directory I'm able to run python spam/spam.py
However, when I start to throw tests into the mix, it's not as awesome.
In my test_spam.py file I have:
import spam.spam
test_client = spam.spam.app.test_client()
def test_it_fails():
assert False
However, rather than failing where I would expect, it fails on the import line:
/spamalot/ $ py.test
# some output
E ImportError
I can fix this by putting __init__.py in my /tests folder, but then I get a different ImportError:
spam/spam.py:1: in <module>
> import spam_on_eggs.spam_on_eggs as eggs
E ImportError: No module named 'spam_on_eggs'
I can solve that one by changing the line to:
from spam.spam_on_eggs import spam_on_eggs
Which allows me to test but then I break my ability to run $ python spam/spam.py - because I get
ImportError: no module named 'spam'
Obviously I have a gap in my understanding of how module imports work and how py.test works with this system.
What am I missing?
Is it even possible to have the layout I've described - and be able to run both py.test and my server from the spamalot directory?
py.test will always use the shortest directory path with an __init__.py in it.
Put a __init__.py into spamalot and you can import spamalot.spam.spam.
Reference: choosing a test layout
My package structure is:
main.py
mapp/
__init__.py
core/
__init__.py
tobeimported.py
test/
__init__.py
(test modules)
utils/
__init__.py
blasttofasta.py
The file blasttofasta.py is executed as script.
blasttofasta.py looks like:
import mapp.core.tobeimported
def somefunc():
pass
if __name__ == '__main__':
pass
But Exception occurs:
Traceback (most recent call last):
File "utils/blasttofasta.py", line 5, in <module>
import mapp.core.tobeimported
ImportError: No module named mapp.core.analyzers
How to import tobeimported module? I run the blasttofasta.py from top directory (where main.py is)
EDIT: Maybe better question is: How to get mapp package to the sys.path? Because script file only see its own directory but not the package directory.
Thank you
If I want to including blasttofasta.py or run it as script simultaneously mos important is to have directory containing mapp package in sys.path.
This worked for me:
Before importing mapp (or other module from this package) I wrote into blasttofasta.py:
import os
os.sys.path.append(os.path.dirname(os.path.realpath(__file__))+ '/../../')
This append mapp package path and I can run it as script. On the other side is no problem with is included in another package.
Follow the absolute structure to import.
To import blasttofasta.py in tobeimport.py
ToBeimport contents
from myapp.utils import blasttofasta
Your structure is good.
Two things need to happen:
Your map directory needs a __init__.py file.
You can simply do this (naively):
$ touch /path/to/map/__init__.py
/path/to/map needs to be in sys.path
Please read: http://docs.python.org/2/tutorial/modules.html for more details.
I reviewed the Python 2.7.5 documentation. I am having issues with my real project, but created a small test project here to concisely reproduce the issue. Imagine a package with the following layout stored at ~/Development/Test
Here is the structure:
Test/
__init__.py
foo.py
sub/
__init__.py
test_foo.py
And the code (__init__.py files are empy):
foo.py
def bar():
print("hello world")
test_foo.py
import Test.foo
# also tried from Test import foo
def main():
foo.bar()
if __name__ == "__main__":
main()
When trying to run test_foo.py from the terminal (i.e. python test_foo.py) I'm getting:
Traceback (most recent call last):
File "test_foo.py", line 1, in <module>
import Test.foo
ImportError: No module named Test.foo
I'm trying to import the main file in the package (foo.py) from the test file in the sub module (in my real project the sub module is the unit testing code). Oddly using Sublime text 2 editor and the plugin python test runner, I can run my individual tests just fine, but I cannot build the test file. It gives me the above error.
Module names are case-sensitive. Use:
import Test.foo as foo
(The as foo is so that you can call foo.bar in main.)
You must also have ~/Development listed in PYTHONPATH.
If using Unix and your login shell is bash, to add ~/Development to PYTHONPATH edit ~/.profile to include
export PYTHONPATH=$PYTHONPATH:$HOME/Development
Here are instructions for Windows.
Further suggestions for debugging:
Place
import sys
print(sys.path)
import Test
print(Test)
import Test.foo
at the top of test_foo.py. Please post the output.
Runnable scripts should always be put outside the module. So if you have a module and a script that runs some code from that module, the structure should look like:
foo/
__init__.py
bar.py
your_script.py
And the code in your_script.py should be something like:
from foo.bar import your_func
your_func()
In case of unittesting it is a good idea (this is opinionated and everyone has his way of doing things) to place the tests inside the module so the structure should look like:
foo/
__init__.py
bar.py
tests/
test_bar.py
But in that case you shouldn't run the script directly. You should either use one of the testing frameworks like nose and run:
nosetests foo
in the directory where you placed your foo module.
Or if you used the standard unittest library, create a TestSuite and what not, run it with:
python -m unittest foo.tests.test_bar
again in the directory where you placed your foo module.
I am using pycharm at one of my university projects and I wanted to integrated it with unittest module, but I have a problem with structuring my project
Part of this project involves generating abstract syntax trees, so I created AST directory and put __init__.py there, then I created expression module. I wanted to put my tests in test/ subdirectory, so it would look like this:
AST/
__init__.py
expression.py
test/
some_test.py
utils.py
now I have also module in my AST called symbol_table and module called utils, example test class looks like
import unittest
from ...AST import expression
from ...AST import utils
class ConstantExpressionTest(unittest.TestCase):
def testConstantExpressionCheck(self):
constantExpression = expression.ConstantExpression(17, 5, utils.TYPES.INT)
self.assertTrue(constantExpression.check())
when I right click on this file and select Run Unittest in ... I am getting errors:
/usr/bin/python2.7 /home/xubuntu/Downloads/pycharm-2.7.2/helpers/pycharm/utrunner.py /home/xubuntu/Przedmioty/VI/kompilatory/tk-projekt/src/AST/test/test_constant_expression.py true
Testing started at 12:06 PM ...
Traceback (most recent call last):
File "/home/xubuntu/Downloads/pycharm-2.7.2/helpers/pycharm/utrunner.py", line 110, in <module>
modules = [loadSource(a[0])]
File "/home/xubuntu/Downloads/pycharm-2.7.2/helpers/pycharm/utrunner.py", line 34, in loadSource
module = imp.load_source(moduleName, fileName)
File "/home/xubuntu/Przedmioty/VI/kompilatory/tk-projekt/src/AST/test/test_constant_expression.py", line 2, in <module>
from ...AST import utils
ValueError: Attempted relative import in non-package
Process finished with exit code 1
I have read about this problem and if I understand this right, this file is treated as it would be in top-level package so I can't use any relative imports.
But if that is the case, how can I run unit-tests from pycharm and also keep my current project strcture?
If I am not mistaken, putting tests in sub-package is pretty popular (http://as.ynchrono.us/2007/12/filesystem-structure-of-python-project_21.html) so there must be some kind of solution
Well, that is a bit silly, I found out that pycharm adds the root of the project to path so I can just use normal imports from the root of my project.
So for example I can write
from AST import expression in my some_test file