I am trying to read a file in a custom module I have created in Python. But it is showing error when I try to do so.
Directory structure
base/
|
|_ app.py
|_ cmod/
|
|_ __init__.py
|_ util.py
|_ db.csv
util.py snippet
from csvhandler import CSVFile
def get_db():
with open("db.csv", "r+") as db:
data = CSVFile(db)
return data
app.py snippet
from cmod import util
data = util.get_db()
It throws the following error
FileNotFoundError: [Errno 2] No such file or directory: 'db.csv'
Is there any issue with imports of placement?
Note: The db.csv has to be put there only.
The current working directory when you run this is likely the one app.py sits in. open() looks for files in the current working directory, not the directory of the currently executing file.
The most portable way to open a file stored relatively would be to get the full path to the python file, isolate the directory, and then join it.
util.py
import os
def get_db():
directory = os.path.dirname(__file__)
with open(os.path.join(directory, "db.csv"), "r+") as db:
data = CSVFile(db)
return data
Related
so my program import a utils that reads a file in the same directory as the utils. However, this utils function can be called from different files from different directory.
Project
|
|-module_1:
|__ init __.py
| file.py <--- calls util.load_file()
|module_2:
| __ init __.py
| utils.py <---- load_file() path used 'file.txt'
| file.txt
what is this thing called ? I couldn't even search for it. tried package managment, expanding path ...etc
__file__ contains the path to the current file. Check it with print(__file__).
pathlib from Pythons standard library can be used to construct an absolute path to the data file.
import pathlib
print(pathlib.Path(__file__))
print(pathlib.Path(__file__).parent)
print(pathlib.Path(__file__).parent / 'file.txt')
You can now open your file like this:
filepath = pathlib.Path(__file__).parent / 'file.txt'
with open(filepath) as f:
for line in f:
print(line)
I have this code in my_program.py:
from my_module import do_stuff_with_file
do_stuff_with_file("hi.txt")
And this is my_module.py:
def do_stuff_with_file(fileName):
print(fileName)
# do stuff with the file
A file not found error arises when my_module.py is not in the same directory as my_program.py. This problem was solved using this code (my_module.py).
def do_stuff_with_file(fileName):
fileName = os.path.join(os.path.dirname(sys.modules['__main__'].__file__), fileName)
print(fileName)
# do stuff with file
The issue arises when my_program.py is imported by a file in a different directory.
How can I fix this?
Given the following file hierarchy :
stack_overflow/
├─ q67993523/
├─ my_module/
│ ├─ __init__.py
├─ 67993523.py
├─ hi.txt
With the following file content :
# 67993523.py
from my_module import do_stuff_with_file
do_stuff_with_file("hi.txt")
# my_module/__init__.py
def do_stuff_with_file(filename):
print(f"{filename!s} content is :")
with open(filename, "rt") as file:
print(file.read())
and the file hi.txt :
Hello !
When I run C:\path\to\python.exe C:/stack_overflow/q67993523/67993523.py (with the path to q67993523 included in my PYTHONPATH), with my current directory being q67993523/, I get :
hi.txt content is :
Hello !
But if I change my current dir to q67993523/my_module/ and execute the exact same command, I get :
hi.txt content is :
Traceback:
[...]
FileNotFoundError: [Errno 2] No such file or directory: 'hi.txt'
because relative to the current working directory q67993523/my_module/ there is no file hi.txt, the file would be ../hi.txt.
I think what you are doing is an instance of the XY problem.
What you are trying to achieve is to find a file given its name but not the location. It is very difficult to do, prone to error, and would include lots of hacks to work.
I don't think it is actually what you want to do. What you want to do, I presume, is not to search for files but just to use them. So you should not lose the precious information of their location.
For example, in your main script (mine is 67993523.py), you know that the file is right there, in the same directory. But if you just send hi.txt, because the function don't know the file location of the code that called her, it does not know where to search for the file.
Instead, give the complete file location, namely the absolute path.
If I change my main script to :
# 67993523.py
from pathlib import Path
from my_module import do_stuff_with_file
the_directory_of_this_pyfile = Path(__file__).parent
do_stuff_with_file((the_directory_of_this_pyfile / "hi.txt").absolute())
And run it with my current directory being q67993523/, I get :
C:\stack_overflow\q67993523\hi.txt content is :
Hello !
And when I change my current directory to q67993523/my_module/, I get the same thing :
C:\stack_overflow\q67993523\hi.txt content is :
Hello !
The difference is that in your script, the hi.txt filename assumes that your current working directory is q67993523/. If you have a different current working directory (because Pytest, because running the script for anywhere you want, ... see the comment from #tdelaney) then there is no ./hi.txt file, so it will fail.
I encourage you to learn on the topic of current working directory
and how to express the current Python file directory.
Let's consider this directory structure:
package/
rooster.py
files/
rooster.txt
app/
main.py
and the content of the two Python files:
rooster.py:
def roost():
file = open('files/rooster.txt')
return file.read()
main.py:
from package.rooster import roost
print(roost())
Running main.py returns an error:
FileNotFoundError: [Errno 2] No such file or directory: 'files/rooster.txt'
That obviously happens because the working directory is app/ and there's no app/files/rooster.txt file. Thus the error. What solution does Python provide for such a scenario?
Maybe you can use the full path of the file, this should work, if it is always the same
Solution:
I am not sure if I understood the issue correctly, but this code might solve your issue.
rooster.py
def roost(path):
import os
file = open(os.path.join(path, 'files/rooster.txt'))
return file.read()
main.py
import os
from pathlib import Path
from rooster import roost
p = Path(__file__).parents[1]
package_dir = os.path.join(p, 'package')
print(roost(package_dir))
OUTPUT:
Hello World1
Hello World2
Hello World3
Directory Structure:
testCode/
package/
rooster.py
files/
rooster.txt
app/
main.py
Let me know in case there is any issue.
I have a folder structure like so:
main/
__init__.py
files/
__init__.py
a.csv
b.yml
subfolder/
__init__.py
code.py
In code.py, I want to use the files in the files folder as so:
# HOW DO I IMPORT a.csv and b.yml
# from main.files import a -- this doesn't work
class Code():
def func(file_a, file_b):
# do something
Is this something I can do? How do I do this?
If code.py is in the same folder as your files (.csv and .yml) you don't have to import them instead you can open them and read them within the function like so:
import csv
import yaml
file_a = 'a.csv'
file_b = 'b.yaml'
class Code():
def func(file_a, file_b):
with open(file_a) as f_obj:
read = csv.reader(f_object)
# do something with opened file
with open("example.yaml", 'r') as stream:
another_read = yaml.safe_load(stream))
# do something with opened file
I think your second file is supposed to be a .yaml file not a .yml file.
I have the following directory structure.
data
│ ├── 2019-12-11
│ └── 2019-12-12
│ ├── ADAZ19.json
│ ├── BCHZ19.json
I want to go in, run a tar command from os.system, and then go back out to where I started.
I tried something like below:
# Full filepath from root.
import os
path_to_file = 'data/2019-12-12/' # This is generated dynamically.
file = 'ADAZ19.json' # This is generated dynamically.
cur_dir = os.getcwd()
os.system(f'tar -czf {cur_dir}{path_to_file}{file}.tar.gz {cur_dir}{path_to_file}{file}')
Which didnt work. So now I want to go into path_to_file, run tar there directly on the file, then go back up to where I started. Note, path_to_file is set by the user and can't be hardcoded.
I want to go into the subdirectory so that the directory structure doesn't get tarred with the file. What is the most elegant way to do so?
Use "CHange DIRectory" to change folder
os.chdir(folder)
import os
path_to_file = 'data/2019-12-12/'
file = 'ADAZ19.json'
cur_dir = os.getcwd()
os.chdir(path_to_file) # change folder
os.system(f'tar -czf {file}.tar.gz {file}')
os.chdir(cur_dir) # go back
EDIT: But you could use python standard module tarfile to create it.
And then you don't have to chage folder but you have to use second argument in
add(added_file, name_in_tar_file)
to set name without path.
import tarfile
path_to_file = 'data/2019-12-12/'
file = 'ADAZ19.json'
t = tarfile.open(f'{path_to_file}{file}.tar.gz', mode='w:gz')
t.add(f'{path_to_file}{file}', file) # second argument without path to file
t.close()