I am trying to get my unit test scripts to work no matter where they are invoked from and have looked into a dozen questions on relative import issues now. But relative paths in combination with the unittest module seem to add some extra complications.
My directory structure is this:
importtest
├── importtest
│ ├── importtest.py
│ └── __init__.py
└── test
├── data.txt
├── importtest_test.py
└── __init__.py
the __init__.py files are empty, importtest.py contains
def func(filename):
with open(filename) as f:
return True
and importtest_test.py contains
import unittest
from importtest import importtest
class Test(unittest.TestCase):
def test(self):
self.assertEqual(importtest.func('data.txt'), True)
This works when I execute py -m unittest test/importtest_test.py from the root folder of the project (the top most importtest folder). But it does not work from any other directory.
Why is that and how can I fix this?
Related
I am creating a Python package and want to make the example files in the example folder runnable with something like python example_file.py. How do I do it? My example_file.py and __init__.py look like below:
# example_file.py
from demandforecast import DemandForecast # module coded in demandforecast.py
if __name__ == '__main__':
# Example code here
# __init__.py
import os, sys; sys.path.append(os.path.dirname(os.path.realpath(__file__)))
When I navigate to the examples directory and run python example_file.py I get the following error:
Exception has occurred: ModuleNotFoundError
No module named 'demandforecast'
This can be solved by adding using the old sys.path.insert() trick mentioned in Importing files from different folder, but I would rather not include the absolute path because this will vary from user to user.
Here is my directory structure:
├───demandforecast
│ demandforecast.py
│ __init__.py
│
└───examples
example_file.py
I have tried a few posts here, such as Relative imports in Python 3, but without luck. Adding sys.path.append('../') to the example file before the demandforecast import also did not work.
I think your import is simply incorrect, and should be:
from demandforecast.demandforecast import DemandForecast
Since demandforecast is both the package name (the directory is definitely a package, with the __init__.py file) and a module name within that package. (I assume DemandForecast is a class within the demandforecast module.)
from src.pro.demandforecast.demandforecast import foo
if __name__ == '__main__':
foo()
output:
foo
create a folder src and inside of it create a package pro and now inside pro create two packages demandforecast and examples.
Directory structure:
/src$ tree
.
└── pro
├── demandforecast
│ ├── demandforecast.py
│ ├── __init__.py
│ └── __pycache__
│ ├── demandforecast.cpython-38.pyc
│ └── __init__.cpython-38.pyc
├── examples
│ ├── example_file.py
│ └── __init__.py
├── __init__.py
└── __pycache__
└── __init__.cpython-38.pyc
IDE:
I have a project that I run from
root_dir> python ./src/main_file.py. When I want to test it, I use root_dir> python -m pytest and it will search for any files of the format test*.py or *test.py.
I have a file structure similar to the following:
root_dir
├── src
│ ├── __init__.py
│ ├── main_file.py
│ ├── file1.py
│ ├── file2.py
└── tests
├── __init__.py
└── file1_test.py
What I'm wanting to do is create a test file to test file1.py, but I'm getting a ModuleNotFoundError for what I believe is the following reason and I'm not fully sure what the proper way to address it is.
In file1_test.py:
./file1_test.py/
import src.file1 as file1
def test_file1():
<do stuff>
The problem occurs inside file1.py, where:
./file1.py/
import file2.py
<blah blah blah>
pytest fails with ModuleNotFoundError: No module named 'file2'. If I change file1.py to:
./file1.py/ (EDITED)
import src.file2.py
<blah blah blah>
then pytest will run successfully. However, if my main_file.py imports file1.py from within the same module (src),
./main_file.py/
import file1
<blah blah blah>
Then python ./src/main_file.py will fail with ModuleNotFoundError: No module named 'src.file2' due to that change. Changing file1 back to the original will work for main_file.py, but not pytest, and I'm stuck in a loop.
I can't figure out what I should be doing differently to get the normal import structure of main_file.py and pytest to cooperate. Should I be changing the directory? Should main_file.py be outside of src? I think I would just run into a similar issue where if file1 is importing file2, the import path a file outside of src expects and one inside of src will expect are always going to conflict.
There are three ways to do what you want.
Prefered way
root_dir
├── my_module
│ ├── __init__.py
│ ├── main_file.py
│ ├── file1.py
│ ├── file2.py
└── tests
├── __init__.py
└── file1_test.py
All files in my_module shoud import like
from my_module.file1 import some_function
Then you don't need to do any hacks
Keepin src and assuming statements like
from src.file1 import some_function
are not what you want you need to modify your tests like
import os
import sys
sys.path.insert(0, os.path.abspath('src')
Not modifying src, not modifying tests: Run tests with
PYTHONPATH=path/to/src pytest
├── ledger
│ ├── __init__.py
│ ├── ledger_data.py
│ └── ledger_model.py
├── main.py
├── sscommon
│ ├── __init__.py
│ └── logging.py
└── tests
└── test_ledger_data.py
I need to import classes from ledger_data module when running test_ledger_data.py. I currently do sys.path.append("../") in test_ledger_data.py or I have to add symbolik links to all modules being used to tests directory. Both options seem incorrect. How to do it correctly?
If I just run the file either from project root or tests directories I get error:
from ledger.ledger_data import LedgerData
ImportError: No module named 'ledger'
You can create an __init__.py file in your folder, and import the parent dir using:
parent_dir = os.path.abspath(os.path.join(os.path.abspath(__file__), os.pardir))
sys.append(parent_dir)
This uses os.path to find out the directory based on your file location.
Update: create the above __init__.py and reside it inside tests/ folder.
Then, in your test_ledge_data.py put at the head of the file from __init__ import *; this will import everything in your init file to your module namespace.
I have the following structure for my package
Project
- __init__.py
- my_mod
-- test
-- __init__.py
-- test_my_mod.py
If I run the nosetests in the project folder it doesn't find my tests, but if I run in the test folder then it picks it up.
Any ideas what could be the problem
Delete the __init__.py file in your Project directory. It doesn't look like you intend for Project to be a Python package, so you don't need it. Then, having this directory structure:
Project
└── my_mod
├── __init__.py
├── some_module.py
├── test
│ ├── __init__.py
│ ├── test_my_mod_again.py
│ └── test_my_mod_yet_again.py
└── test_my_mod.py
Running nosetests inside Project will now run your tests, both inside the my_mod/ directory and inside the test/ directory.
Of course, the contents of the test files should contain something that nosetests can recognize as a test, such as:
import unittest
from my_mod import some_module
class MyTestCase(unittest.TestCase):
def test_the_number(self):
assert some_module.double(10) == 20
I've a project which uses git submodules. In my python file I want to use functions from another python file in the submodule project.
In order to work I had to add the init.py file to all subfolders in the path. My folder tree is the following:
myproj
├── gitmodules
│ ├── __init__.py
│ ├── __init__.pyc
│ └── mygitsubmodule
│ ├── __init__.py
│ ├── __init__.pyc
│ └── file.py
└── myfile.py
Is there any way to make it work without touching mygitsubmodule ?
Thanks
you can add to sys.path in the file you want to be able to access the module, something like:
import sys
sys.path.append("/home/me/myproj/gitmodules")
import mygitsubmodule
This example is adding a path as a raw string to make it clear what's happening. You should really use the more sophisticated, system independent methods described below to determine and assemble the path.
Also, I have found it better, when I used this method, to use sys.path.insert(1, .. as some functionality seems to rely of sys.path[0] being the starting directory of the program.
I am used to avoiding modifying sys.path.
The problem is, when using git submodule, submodule is a project directory, not a Python package. There is a "gap" between your module and that package, so you can't import.
Suppose you have created a submodule named foo_project, and there is a foo package inside.
.
├── foo_project
│ ├── README.rst
│ └── foo
│ └── __init__.py
└── main.py
My solution will be creating a soft link to expose that package to your module:
ln -s foo_project/foo foo
.
├── foo_project
│ ├── README.rst
│ └── foo
│ └── __init__.py
├── foo -> foo_project/foo
└── main.py
Now you can import foo in the main.py.
For reference,
from submodulefolder.file import func_name
or
import submodulefolder.file as lib_name
where file excludes the extension of file.py, seems to work in relative terms without modifying the subfolder / git submodule with a init.py since python 3.3+,
as shown here.
Tested on py3.8.5 linux native and py3.7.8 anaconda Windows, both in Spyder's Ipython-console, as well as natively on linux via terminal.