I have a package and a test package. According to advice from Where do the Python unit tests go?, the tests should be in a different directory. The project's directory tree is as follows:
project\
kernel\
__init__.py
file1.py
file2.py
tests\
__init__.py
test1.py
test2.py
test3.py
I would like to import the kernel package to the tests package, because that is where file1.py and file2.py are being tested. Also, I would like to use one import statement in the __init__.py instead of importing kernel again and again in each test.
I tried adding the following to the __init__.py file in tests and to test2.py , test2.py (together and separately), with no success (the first does no harm, the second gives a syntax error):
import kernel
import ../kernel
I'm using python2.6. From command line all the above happens. When I use Eclipse PyDev, everything magically works.
The relative imports you're using will only work if the "project" directory is a python package (i.e. it has an __init__.py file in it). Try that first and see if that works for you.
If the kernel directory is acting as the "package" that would be distributed then you could put the tests directory inside of that and do the relative imports that way. So it would look like this:
project/
kernel/
__init__.py
file1.py
file2.py
tests/
__init__.py
test1.py ...
And you would import the kernel modules from the tests directory either as:
from kernel import file1 # if it's installed in the python path/environment
Or:
from .. import file1
# 'import ..file1' might work, but I'm not sure that's syntactically correct
Related
For Python 3.
I just started learning Python. I have PHP and Ruby background.
Currently very confused with modules, __init__ and python -m.
At the moment I have the following:
modules/practice.py
tests/test_practice.py
In practice.py
class First:
def attempt(self):
return 'attempted'
In test_practice.py
from modules.practice import First
class TestMain:
def test_attempt(self):
first = First()
attempted = first.attempt()
assert attempted is 'attempted'
When I run pytest I get an error ModuleNotFoundError: No module named 'modules
When I run python -m pytest test is green.
However, if I add __init__.py files in modules and tests, both are green.
After trying to find out answers on my own I confess I am not sure I am getting it.
Why pytest does not work without __init__?
When working on a project, when people assume python -m will be used and when people add the __init__.py files instead?
Do not add an __init__.py file into the tests directory.
You should add an __init__.py file into the modules directory.
Ideally it should be possible to run the test suite with either pytest or python -m pytest. For this to work, the parent directory of modules needs to be present on sys.path. Usually you would do this by writing a setup.py file for the package and installing your code in "editable" mode. The tests run against the installed code linked in site-packages.
If you don't want to write a setup.py file at this stage, you can either export an environment variable:
export PYTHONPATH=/path/to/parent_of_modules
Or you can inject to sys.path directly from conftest.py file, which gets imported first:
.
├── modules
│ ├── __init__.py
│ └── practice.py
└── tests
├── conftest.py
└── test_practice.py
Example:
# conftest.py
import os
import sys
here = os.path.dirname(os.path.abspath(__file__))
sys.path.append(os.path.dirname(here))
Yes I read the answer to this Relative imports for the billionth time. No I did not find it really useful as I tried both solutions and it did not work for me. See below.
I have the below folder structure.
(I do not have any __init__.py, as I understood they're not required anymore. I have Python 3.7)
project
├── subpackage1
└── subpackage2
├── launch
| └── launchFunc.py
| └── launchFunc.sh
└── src
└── somemodule.py
launchFunc.py:
from somemodule import func
func()
launchFunc.sh:
#!/bin/bash
python -m launch.launchFunc
Then I open command line and I cd into project/subpackage2/ and I run the below command:
./launch/launchFunc.sh
and I get the below error:
from somemodule import func
ModuleNotFoundError: No module named 'somemodule'
If I replace from somemodule import func with from ..src.somemodule import func, I get:
ValueError: attempted relative import beyond top-level package
What is the correct way to import and/or run scripts in python ?
Isn't there a way to have all functions automatically imported to all required modules ?
Ok I have found out that by:
always using the full path to import: import func from subpackage2.somemodule
always running things from the project folder
it will work fine.
I think this is a rule I should adopt for all my projects.
I have a python app that I’m attempting to unit test. The code is sat in a src directory the tests in a test directory.
The problem comes when trying to test the class containing my main method. This class sits in the src directory and has references to classes in other modules that all sit in the src directory. In order to execute my app I have the references in this format from <module> import <class>
When I come to test this main class I write a test class and place it in the test directory. Running the test I find I have problems with the imports in the main class. To resolve I have to change the imports to be from from src.<module> import <class> Having done this my tests now pass but the app itself fails.
How can I resolve the issue so that my imports are valid for both my unit tests and normal execution?
It may fail due to the way import works.
I would recommend always using absolute imports. This means, that when creating a package, all imports within that package import all the way from the top level down: from package.module import thing. And more importantly, installing your package - not as a normal Python package, but with a symbolic link allows you to edit your source code while it still is considered an installed package. This way you can position your tests and scripts however you want them and the thing will always work.
Installing your package is done with pip install -e . from root. See a packaging tutorial for an example of packaging if you're not familiar with it.
An example package structure might be:
root/
setup.py
MANIFEST.in
...
package/
__init__.py
app.py
tests.py
module/
__init__.py
component.py
module_test.py
# Or have a tests folder under root
tests/
test_module.py
tests.py
And for example in app.py, and everywhere really:
from package.module.component import Class
from package.module import thing # that was declared in module/__init__.py
So to your case, if I understood it correctly, you'd have:
root/
setup.py
MANIFEST.in
...
src/
__init__.py
app.py
module.py
tests/
test_module.py
tests.py
In my project, have a subdirectory containing a python application I'd like to unit-test.
The structure looks as follows:
my-project/
python-app/
sometool/
__init__.py
foo/
__init__.py
aaa.py
bar/
__init__.py
bbb.py
test/
sometool-tests/
foo-tests/
aaa_test.py
Now, aaa.py contains imports like import sometool.bar.bbb, which assumes the application's base directory is python-app, which is indeed the case in my build setup.
aaa_test.py obviously imports aaa for testing it.
However, when running nosetests ./python-app/sometool/test from the main project directory, imports fail because my-project is now the base directory for imports, i.e. sometool.bar.bbb is not found from there.
If I cd into python-app first and run nosetests ./sometool/test from there, everything works as expected.
But I would like to configure Visual Studio Code to run those tests using a shortcut, and those command always seem to be executed from the project root.
Is there a way to pass the "base directory" as argument to Nose?
You can use a context providing module in the nosetests environment:
my-project/
python-app/
...
test/
context.py
context.py:
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
import python-app
Then you can import python-app in your tests import from the context module, e.g. in aaa_test.py:
from tests.context import python-app
from python-app.sometool import foo, bar
This should make nosetests always find the python app, no matter where it is being executed.
I have a script script.py and multiple modules in a different directory that all depend on each other. For simplicity, we just look at two of them, module1.py and module2.py, where the first imports the latter. All these should be used by script.py.
Now I added a symlink to module1.py into the directory of script.py, so my directory tree looks like this:
.
├── mymodules
│ ├── module1.py
│ └── module2.py
└── myscript
├── module1.py -> ../mymodules/module1.py
└── script.py
Just running script.py now doesn't work, because PYTHONPATH does not contain the mymodules directory, and module1 is therefore unable to import module2. Now, there is an easy workaround for this; Appending the path of module1.py to PYTHONPATH:
sys.path.append(os.path.abspath(os.path.join(os.path.realpath(__file__),os.path.pardir)))
And this is where problems emerge: This works, but only once!
The first run works fine, all modules are imported without any problems.
But every subsequent execution of $ ./script.py fails with the exception ImportError: no module named module2 and sys.path contains the directory of the symlink, not the file! Why? And how do I fix this?
All of the code:
I thought you could need this if you wanted to try this yourself.
myscript/script.py:
#!/usr/bin/env python
import module1
mymodules/module1.py
#!/usr/bin/env python
import sys, os
#append directory of this file to PYTHONPATH
sys.path.append(os.path.abspath(os.path.join(os.path.realpath(__file__),os.path.pardir)))
#print for control reasons
print sys.path
import module2
mymodules/module2.py
#!/usr/bin/env python
print "import successful!"
Simply, append the real dirname to sys.path
sys.path.append(os.path.dirname(os.path.realpath(__file__)))
This output is
import successful
However, after symbolic module1.py is compiled to module1.pyc, modeul1.pyc will be located at myscript. The above code does not work.
Solution
So solution is to modify mymodules/module1.py as
import os
import sys
srcfile = __file__
if srcfile.endswith('.pyc'):
srcfile = srcfile[:-1] # pyc to py
sys.path.append(os.path.dirname(os.path.realpath(srcfile)))
import module2