In my python application, I open mp3 files with relative paths from where the program was started. To keep it simple, I made a minimal reproduction of the problem I have in my project here.
Basically, I have a structure like this:
src
└─ main.py
test
└─ test_main.py
In main.py I have a simple function that prints and returns the current working directory:
def get_cwd() -> str:
directory = os.path.basename(os.getcwd())
print('Current directory =', directory)
return directory
So if I cd into the src folder and run python main.py I see:
Current directory = src
This is the desired behavior, as in my program the file paths to the mp3 files are relative to src.
The problem arises when I try to write tests. I can't seem to get a test like this to pass, no matter what I pass to --start-directory and --top-level-directory:
def test_get_cwd(self):
print('testing get_cwd()')
current_dir = get_cwd()
self.assertIsNotNone(current_dir)
self.assertEqual(current_dir, 'src')
The question: How can I run my tests as if they were running in the context of a specific directory if they are saved to a different directory?
Constraints:
the tests must import using absolute paths, as in my example: from
src.main import get_cwd
There is a os function to change the directory, try adding os.chdir('src') to your test.
import unittest
import os
from src.main import get_cwd
class TestMain(unittest.TestCase):
def test_get_cwd(self):
os.chdir('src')
print('testing get_cwd()')
current_dir = get_cwd()
self.assertIsNotNone(current_dir)
self.assertEqual(current_dir, 'src')
An option is to mock the value given by os.path.basename with a return value of "src"
import unittest
from mock import patch
from src.main import get_cwd
class TestMain(unittest.TestCase):
#patch('os.path.basename')
def test_get_cwd(self, basename):
basename.return_value = "src"
print('testing get_cwd()')
current_dir = get_cwd()
self.assertIsNotNone(current_dir)
self.assertEqual(current_dir, 'src')
Related
I want to write a test for a function using Pytest. The function gets an id by which it finds the path to a video file, and then creates another folder in the video's folder (to put some new files in it). Here is the function:
def foo(id):
root_dir = find_local_dir(id) # finds the path to the video file
video_file = root_dir / 'video.mp4'
out_dir = root_dir / 'foo'
out_dir.mkdir(parents=True, exist_ok=True)
I have a video in the tests directory which I would like to use for testing, and I use monkeypatch to prevent the function from looking for the video according to the id. But I don't know how to make it get the video in tests and create the new folder in a tmp, without changing foo.
I tried to copy the video from tests to tmp_path but that doesn't work:
def test_foo(tmp_path, monkeypatch):
id = '1234'
mock = MagicMock()
mock.return_value = tmp_path
monkeypatch.setattr(pipeline, 'find_local_dir', mock)
copy2('coffee.mp4', tmp_path / 'video.mp4')
var = pipeline.foo(id)
And the error I had was FileNotFoundError: [WinError 2] The system cannot find the file specified
I guess the copying didn't work? (I'm a bit new to Pytest so I don't know if that's even possible.)
I'll be happy with any other way to solve this without changing foo.
Thanks.
Edit:
def find_local_dir(id):
local_dir = Path(f'/tmp/{id}')
local_dir.mkdir(parents=True, exist_ok=True)
return local_dir
I have a video in the tests directory which I would like to use for testing
So I assume you already have a tests directory which we can use as our temporary directory for testing. Now, all that we need to control is the output of find_local_dir() to point to the tests directory because
It is where the video.mp4 is located
It is where the directory foo would be created
File tree
$ tree
.
├── pipeline.py
└── tests
├── coffee.mp4
└── test_pipeline.py
pipeline.py
from pathlib import Path
def find_local_dir(id):
local_dir = Path(f'/tmp/{id}')
local_dir.mkdir(parents=True, exist_ok=True)
return local_dir
def foo(id):
root_dir = find_local_dir(id) # finds the path to the video file
video_file = root_dir / 'video.mp4'
out_dir = root_dir / 'foo'
out_dir.mkdir(parents=True, exist_ok=True)
# Let's add a print to see the location of the video file
print(f"{video_file=}")
test_pipeline.py
import os
from pathlib import Path
from shutil import copy2, rmtree
import time
from unittest.mock import MagicMock
import pytest
import pipeline
#pytest.fixture
def tmp_path(mocker):
local_dir = Path(f'./tests/tmp')
local_dir.mkdir(parents=True, exist_ok=True)
yield local_dir
# Optionally, delete the temporary directory used for testing
rmtree(local_dir)
def test_foo(tmp_path, monkeypatch):
id = '1234'
mock = MagicMock()
mock.return_value = tmp_path
monkeypatch.setattr(pipeline, 'find_local_dir', mock)
copy2('./tests/coffee.mp4', tmp_path / 'video.mp4')
pipeline.foo(id)
# Remove this line. It is just here so that we can see the files before deletion.
time.sleep(10)
Running the test
$ pytest -q -rP
================================================================================================= PASSES ==================================================================================================
________________________________________________________________________________________________ test_foo _________________________________________________________________________________________________
------------------------------------------------------------------------------------------ Captured stdout call -------------------------------------------------------------------------------------------
video_file=PosixPath('tests/tmp/video.mp4')
1 passed in 0.05s
File tree while the test is ongoing
$ tree
.
├── pipeline.py
└── tests
├── coffee.mp4
├── test_pipeline.py
└── tmp
├── foo
└── video.mp4
File tree after the test finished
$ tree
.
├── pipeline.py
└── tests
├── coffee.mp4
└── test_pipeline.py
As you can see, the location of the video file is correct at tests/tmp/video.mp4. The new directory was also correctly created at tests/tmp/foo. Then at the end of the test, the temporary tests/tmp/ was deleted.
I have my target.py file in Starterpack/ModulesAndPackages/target.py
and I have my script file in Starterpack/Scripts/Bad.py
My ModulesAndPackages folder has the __init__.py file but I still get this error No module named ModulesAndPackages when I type from ModulesAndPackages.target import Target in the script file.
I've tried the sys.path.append() and sys.path.insert() but none worked. In my editor there is no error but when I run it, it gives the error.
script file:
mandp_dir = "./ModulesAndPackages"
scripts_dir = "./Scripts"
main_dir = ".."
os.chdir(Path(main_dir))
from ModulesAndPackages.target import Target
target.py file:
import time
import os
import keyboard
class Target():
def __init__(self, ip, port, packetsize, time=None):
self.ip = ip
self.port = port
self.packetsize = packetsize
self.time = time
def attack(self):
pass
I expected it to work if I added the __init__.py file, but it doesn't.
You probably wanted to add the dir to the path python searches, not change your current directory, e.g.:
scripts_dir = "./Scripts"
main_dir = ".."
sys.path.append(Path(main_dir))
Also, check this out: python-best-way-to-add-to-sys-path-relative-to-the-current-running-script
Add the __init__.py file to your StarterPack folder too and use the from StarterPack.ModulesAndPackages.target import yourFunctionName
You can try adding additional path to sys.path:
import os, sys
from pathlib import Path
main_dir = ".."
sys.path.append("..")
from ModulesAndPackages.target import Target
if you're doing this just for some small testing then it's ok.
BUT: I would discourage doing this in most projects as this is generally not very good approach. Instead your script should be at the same level as ModulesAndPackages like this in order for from ModulesAndPackages.target import Target to import properly:
├── ModulesAndPackages
│ ├── __init__.py
│ └── target.py
└── script.py
Here script.py will act as sort of main starting point.
I am trying to get a list of the imported custom modules (modules I created by myself) in my current file but I could not find a proper way to achieve it.
For example:
<test.py>
import sys
import foo
import bar
I would like to get a list of [foo, bar] excluding the sys module.
Lets say, that if file located near curent file, or in some subfolder, when it's "our" module, not system one.
For this example, I've created three files: main.py, other.py and another.py, and placed whem into one folder.
The code of main.py is:
# os and sys are needed to work
import sys
import os
import shutil
import io
import datetime
import other
import another
def get_my_modules():
# Get list of modules loaded
modules = list(sys.modules.keys())
mymodules = []
# Get current dir
curdir = os.path.realpath(os.path.dirname(__file__))
for m in modules:
try:
# if some module's file path located in current folder or in some subfolder
# lets sey, that it's our self-made module
path = sys.modules[m].__file__
if path.startswith(curdir):
mymodules.append(m)
except Exception:
# Exception could happen if module doesn't have any __file__ property
pass
# Return list of our moudles
return mymodules
print(get_my_modules())
And this code actually outputs ["other", "another"]
This approach has a problem: If you import module, that somehow located in upper folder, it woudln't be detected.
There are multiple threads about getting the current Python's script directory, for example:
import os
dir = os.path.dirname(os.path.abspath(__file__))
The question is, what should I do if I want to add this function into some utility file, while I want the returned value to be the directory of the calling file, in this case, the file executed by the user?
What would happen here?
// utils.py
def get_script_dir():
import os
return os.path.dirname(os.path.abspath(__file__))
// main_app.py
from utils import get_script_dir
print(get_script_dir())
// user's shell
python c:\path\to\somewhere\main_app.py
Will it print the directory of main_app.py or the directory of utils.py? What is an elegant solution for placing the function in the utils file while getting the directory of the file that was actually executed by the user?
Please try following and let me know whether it's exactly what you want:
# utils.py
def get_script_dir():
import sys
import os
return os.path.dirname(sys.modules['__main__'].__file__)
# main_app.py
from utils import get_script_dir
print(get_script_dir())
Supposing that your project directory tree is like the following:
Python/
main.py
modules/
utils.py
__init__.py
Being Python/modules/utils.py:
import os
get_script_dir = lambda file: os.path.dirname(os.path.abspath(file))
and Python/main.py:
from modules import utils
import os
script_dir = utils.get_script_dir(__file__)
print("[i] Currently executing file {} located at {}".format(os.path.basename(__file__), script_dir))
Executing Python/main.py is going to output something like:
[i] Currently executing file main.py located at C:\Users\BlackVirusScript\Desktop\Python
I have a directory which has all the files:
myDirectory/
directory1/
importantFile.py
Output.py
How can I import Output.py from importantFile.py without having to put in the same directory?
importantFile.py
import Output
Output.write('This worked!')
Output.py
class Output():
def writeOutput(s):
print s
if "call" is import, in Output.py
import sys
import os.path
# change how import path is resolved by adding the subdirectory
sys.path.append(os.path.abspath(os.getcwd()+'/directory1'))
import importantFile
importantFile.f()
sys.path contains the list of path where to look for modules, details in https://docs.python.org/2/library/sys.html
The other way is to use the relative notation, for which the python file you want to import should be in a package.
You have to make the directory a python package by putting an init.py file.
Look for the packages section in this link.
https://docs.python.org/2/tutorial/modules.html
import sys
sys.path.append('/full/path/to/use')
global exist_importedname
exist_importedname = True
try:
import myimport
except ImportError as e:
exist_importedname = False
print (e.message)