I have the following structure
main.py
module/
properties.yaml
file.py
file.py relevant code:
def read_properties():
with open('properties.yaml') as file:
properties = yaml.load(file)
main.py relevant code:
from module import file
file.read_properties()
When read_properties() is called within main.py, I get the following error: FileNotFoundError: [Errno 2] No such file or directory: 'properties.yaml'
What is the recommended way of allowing my module to access the properties file even when imported?
Provide the absolute path to properties.yaml:
with open('/Users/You/Some/Path/properties.yaml') as file:
As JacobIRR said in his answer, it is best to use the absolute path to the file. I use the os module to construct the absolute path based on the current working directory. So for your code it would be something like:
import os
working_directory = os.path.dirname(__file__)
properties_file = os.path.join(working_directory, 'module', 'properties.yaml')
Based on answers from #JacobIRR and #BigGerman
I ended up using pathlib instead of os, but the logic is the same.
Here is the syntax with pathlib for those interested:
in file.py:
from pathlib import Path
properties_file = Path(__file__).resolve().parent/"properties.yaml"
with open(properties_file) as file:
properties = yaml.load(file)
Related
My file structure is as follows:
kkg/
builder/test.py
builder/data
api/api.py
__init__py
'kkg' is my package name, and in init.py some function are defined, and implementations of these function are written api.py.
In test.py, I have:
import kkg
kkg.load('builder/data/')
Inside the 'load' of the api.py, I have code:
abspath = os.path.abspath(os.path.dirname(__file__))
...
for file in files:
file_path = os.path.join(abspath, data_path)
The data_path is the parameter 'builder/data/' passed from test.py. The path.join reports an error:
Caused by: java.io.FileNotFoundException: /Users/comin/kkg/kkg/api/data/people.csv
The correct data path, if parsed properly, should be:
/Users/comin/kkg/kkg/data/people.csv
I run the 'test.py' inside the builder/ directory. I think the reason there is an unnecessary 'api' in the path generated is because the code piece where error occurs in the api.py.
Perhaps I shouldn't use the join(abspath, data_path) to get the absolute directory. How to get the path correctly parsed?
EDIT:
I changed the path parameter:
kkg.load('../builder/data/')
but then this code failed:
if not os.path.isdir(data_path):
raise ValueError('data_path must be a directory, not file name!')
Why does it raise an error when I added '..' to the path? It is not considered as a directory due to the '..'?
you want the parent directory I think
abspath = os.path.abspath(os.path.join(os.path.dirname(__file__),".."))
you seem to be struggling with this more than you should...it might be easier to just do
path_to_data_file_folder = os.path.abspath(sys.argv[1])
then call it with python main.py /path/to/data/folder
It is much easier to use pathlib over os library:
# api/api.py
from pathlib import Path
KKG_FOLDER = Path(__file__).parent.parent
DATA_FOLDER = KKG_FOLDER / 'builder/data'
print(DATA_FOLDER)
This is more verbose and easier to understand.
Say I have a Python project that is structured as follows:
project
/data
test.csv
/package
__init__.py
module.py
main.py
__init__.py:
from .module import test
module.py:
import csv
with open("..data/test.csv") as f:
test = [line for line in csv.reader(f)]
main.py:
import package
print(package.test)
When I run main.py I get the following error:
C:\Users\Patrick\Desktop\project>python main.py
Traceback (most recent call last):
File "main.py", line 1, in <module>
import package
File "C:\Users\Patrick\Desktop\project\package\__init__.py", line 1, in <module>
from .module import test
File "C:\Users\Patrick\Desktop\project\package\module.py", line 3, in <module>
with open("../data/test.csv") as f:
FileNotFoundError: [Errno 2] No such file or directory: '../data/test.csv'
However, if I run module.py from the package directory, I don’t get any errors. So it seems that the relative path used in open(...) is only relative to where the originating file is being run from (i.e __name__ == "__main__")? I don't want to use absolute paths. What are some ways to deal with this?
Relative paths are relative to current working directory.
If you do not want your path to be relative, it must be absolute.
But there is an often used trick to build an absolute path from current script: use its __file__ special attribute:
from pathlib import Path
path = Path(__file__).parent / "../data/test.csv"
with path.open() as f:
test = list(csv.reader(f))
This requires python 3.4+ (for the pathlib module).
If you still need to support older versions, you can get the same result with:
import csv
import os.path
my_path = os.path.abspath(os.path.dirname(__file__))
path = os.path.join(my_path, "../data/test.csv")
with open(path) as f:
test = list(csv.reader(f))
[2020 edit: python3.4+ should now be the norm, so I moved the pathlib version inspired by jpyams' comment first]
For Python 3.4+:
import csv
from pathlib import Path
base_path = Path(__file__).parent
file_path = (base_path / "../data/test.csv").resolve()
with open(file_path) as f:
test = [line for line in csv.reader(f)]
This worked for me.
with open('data/test.csv') as f:
My Python version is Python 3.5.2 and the solution proposed in the accepted answer didn't work for me. I've still were given an error
FileNotFoundError: [Errno 2] No such file or directory
when I was running my_script.py from the terminal. Although it worked fine when I run it through Run/Debug Configurations from the PyCharm IDE (PyCharm 2018.3.2 (Community Edition)).
Solution:
instead of using:
my_path = os.path.abspath(os.path.dirname(__file__)) + some_rel_dir_path
as suggested in the accepted answer, I used:
my_path = os.path.abspath(os.path.dirname(os.path.abspath(__file__))) + some_rel_dir_path
Explanation:
Changing os.path.dirname(__file__) to os.path.dirname(os.path.abspath(__file__))
solves the following problem:
When we run our script like that: python3 my_script.py
the __file__ variable has a just a string value of "my_script.py" without path leading to that particular script. That is why method dirname(__file__) returns an empty string "". That is also the reason why my_path = os.path.abspath(os.path.dirname(__file__)) + some_rel_dir_path is actually the same thing as my_path = some_rel_dir_path. Consequently FileNotFoundError: [Errno 2] No such file or directory is given when trying to use open method because there is no directory like "some_rel_dir_path".
Running script from PyCharm IDE Running/Debug Configurations worked because it runs a command python3 /full/path/to/my_script.py (where "/full/path/to" is specified by us in "Working directory" variable in Run/Debug Configurations) instead of justpython3 my_script.py like it is done when we run it from the terminal.
Try
with open(f"{os.path.dirname(sys.argv[0])}/data/test.csv", newline='') as f:
I was surprised when the following code worked.
import os
for file in os.listdir("../FutureBookList"):
if file.endswith(".adoc"):
filename, file_extension = os.path.splitext(file)
print(filename)
print(file_extension)
continue
else:
continue
So, I checked the documentation and it says:
Changed in version 3.6: Accepts a path-like object.
path-like object:
An object representing a file system path. A path-like object is
either a str or...
I did a little more digging and the following also works:
with open("../FutureBookList/file.txt") as file:
data = file.read()
Say I have a Python project that is structured as follows:
project
/data
test.csv
/package
__init__.py
module.py
main.py
__init__.py:
from .module import test
module.py:
import csv
with open("..data/test.csv") as f:
test = [line for line in csv.reader(f)]
main.py:
import package
print(package.test)
When I run main.py I get the following error:
C:\Users\Patrick\Desktop\project>python main.py
Traceback (most recent call last):
File "main.py", line 1, in <module>
import package
File "C:\Users\Patrick\Desktop\project\package\__init__.py", line 1, in <module>
from .module import test
File "C:\Users\Patrick\Desktop\project\package\module.py", line 3, in <module>
with open("../data/test.csv") as f:
FileNotFoundError: [Errno 2] No such file or directory: '../data/test.csv'
However, if I run module.py from the package directory, I don’t get any errors. So it seems that the relative path used in open(...) is only relative to where the originating file is being run from (i.e __name__ == "__main__")? I don't want to use absolute paths. What are some ways to deal with this?
Relative paths are relative to current working directory.
If you do not want your path to be relative, it must be absolute.
But there is an often used trick to build an absolute path from current script: use its __file__ special attribute:
from pathlib import Path
path = Path(__file__).parent / "../data/test.csv"
with path.open() as f:
test = list(csv.reader(f))
This requires python 3.4+ (for the pathlib module).
If you still need to support older versions, you can get the same result with:
import csv
import os.path
my_path = os.path.abspath(os.path.dirname(__file__))
path = os.path.join(my_path, "../data/test.csv")
with open(path) as f:
test = list(csv.reader(f))
[2020 edit: python3.4+ should now be the norm, so I moved the pathlib version inspired by jpyams' comment first]
For Python 3.4+:
import csv
from pathlib import Path
base_path = Path(__file__).parent
file_path = (base_path / "../data/test.csv").resolve()
with open(file_path) as f:
test = [line for line in csv.reader(f)]
This worked for me.
with open('data/test.csv') as f:
My Python version is Python 3.5.2 and the solution proposed in the accepted answer didn't work for me. I've still were given an error
FileNotFoundError: [Errno 2] No such file or directory
when I was running my_script.py from the terminal. Although it worked fine when I run it through Run/Debug Configurations from the PyCharm IDE (PyCharm 2018.3.2 (Community Edition)).
Solution:
instead of using:
my_path = os.path.abspath(os.path.dirname(__file__)) + some_rel_dir_path
as suggested in the accepted answer, I used:
my_path = os.path.abspath(os.path.dirname(os.path.abspath(__file__))) + some_rel_dir_path
Explanation:
Changing os.path.dirname(__file__) to os.path.dirname(os.path.abspath(__file__))
solves the following problem:
When we run our script like that: python3 my_script.py
the __file__ variable has a just a string value of "my_script.py" without path leading to that particular script. That is why method dirname(__file__) returns an empty string "". That is also the reason why my_path = os.path.abspath(os.path.dirname(__file__)) + some_rel_dir_path is actually the same thing as my_path = some_rel_dir_path. Consequently FileNotFoundError: [Errno 2] No such file or directory is given when trying to use open method because there is no directory like "some_rel_dir_path".
Running script from PyCharm IDE Running/Debug Configurations worked because it runs a command python3 /full/path/to/my_script.py (where "/full/path/to" is specified by us in "Working directory" variable in Run/Debug Configurations) instead of justpython3 my_script.py like it is done when we run it from the terminal.
Try
with open(f"{os.path.dirname(sys.argv[0])}/data/test.csv", newline='') as f:
I was surprised when the following code worked.
import os
for file in os.listdir("../FutureBookList"):
if file.endswith(".adoc"):
filename, file_extension = os.path.splitext(file)
print(filename)
print(file_extension)
continue
else:
continue
So, I checked the documentation and it says:
Changed in version 3.6: Accepts a path-like object.
path-like object:
An object representing a file system path. A path-like object is
either a str or...
I did a little more digging and the following also works:
with open("../FutureBookList/file.txt") as file:
data = file.read()
I have a directory which has all the files:
myDirectory/
directory1/
importantFile.py
Output.py
How can I import Output.py from importantFile.py without having to put in the same directory?
importantFile.py
import Output
Output.write('This worked!')
Output.py
class Output():
def writeOutput(s):
print s
if "call" is import, in Output.py
import sys
import os.path
# change how import path is resolved by adding the subdirectory
sys.path.append(os.path.abspath(os.getcwd()+'/directory1'))
import importantFile
importantFile.f()
sys.path contains the list of path where to look for modules, details in https://docs.python.org/2/library/sys.html
The other way is to use the relative notation, for which the python file you want to import should be in a package.
You have to make the directory a python package by putting an init.py file.
Look for the packages section in this link.
https://docs.python.org/2/tutorial/modules.html
import sys
sys.path.append('/full/path/to/use')
global exist_importedname
exist_importedname = True
try:
import myimport
except ImportError as e:
exist_importedname = False
print (e.message)
How do I get the current file's directory path?
I tried:
>>> os.path.abspath(__file__)
'C:\\python27\\test.py'
But I want:
'C:\\python27\\'
The special variable __file__ contains the path to the current file. From that we can get the directory using either pathlib or the os.path module.
Python 3
For the directory of the script being run:
import pathlib
pathlib.Path(__file__).parent.resolve()
For the current working directory:
import pathlib
pathlib.Path().resolve()
Python 2 and 3
For the directory of the script being run:
import os
os.path.dirname(os.path.abspath(__file__))
If you mean the current working directory:
import os
os.path.abspath(os.getcwd())
Note that before and after file is two underscores, not just one.
Also note that if you are running interactively or have loaded code from something other than a file (eg: a database or online resource), __file__ may not be set since there is no notion of "current file". The above answer assumes the most common scenario of running a python script that is in a file.
References
pathlib in the python documentation.
os.path - Python 2.7, os.path - Python 3
os.getcwd - Python 2.7, os.getcwd - Python 3
what does the __file__ variable mean/do?
Using Path from pathlib is the recommended way since Python 3:
from pathlib import Path
print("File Path:", Path(__file__).absolute())
print("Directory Path:", Path().absolute()) # Directory of current working directory, not __file__
Note: If using Jupyter Notebook, __file__ doesn't return expected value, so Path().absolute() has to be used.
In Python 3.x I do:
from pathlib import Path
path = Path(__file__).parent.absolute()
Explanation:
Path(__file__) is the path to the current file.
.parent gives you the directory the file is in.
.absolute() gives you the full absolute path to it.
Using pathlib is the modern way to work with paths. If you need it as a string later for some reason, just do str(path).
Try this:
import os
dir_path = os.path.dirname(os.path.realpath(__file__))
import os
print(os.path.dirname(__file__))
I found the following commands return the full path of the parent directory of a Python 3 script.
Python 3 Script:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from pathlib import Path
#Get the absolute path of a Python3.6 and above script.
dir1 = Path().resolve() #Make the path absolute, resolving any symlinks.
dir2 = Path().absolute() #See #RonKalian answer
dir3 = Path(__file__).parent.absolute() #See #Arminius answer
dir4 = Path(__file__).parent
print(f'dir1={dir1}\ndir2={dir2}\ndir3={dir3}\ndir4={dir4}')
REMARKS !!!!
dir1 and dir2 works only when running a script located in the current working directory, but will break in any other case.
Given that Path(__file__).is_absolute() is True, the use of the .absolute() method in dir3 appears redundant.
The shortest command that works is dir4.
Explanation links: .resolve(), .absolute(), Path(file).parent().absolute()
USEFUL PATH PROPERTIES IN PYTHON:
from pathlib import Path
#Returns the path of the current directory
mypath = Path().absolute()
print('Absolute path : {}'.format(mypath))
#if you want to go to any other file inside the subdirectories of the directory path got from above method
filePath = mypath/'data'/'fuel_econ.csv'
print('File path : {}'.format(filePath))
#To check if file present in that directory or Not
isfileExist = filePath.exists()
print('isfileExist : {}'.format(isfileExist))
#To check if the path is a directory or a File
isadirectory = filePath.is_dir()
print('isadirectory : {}'.format(isadirectory))
#To get the extension of the file
fileExtension = mypath/'data'/'fuel_econ.csv'
print('File extension : {}'.format(filePath.suffix))
OUTPUT:
ABSOLUTE PATH IS THE PATH WHERE YOUR PYTHON FILE IS PLACED
Absolute path : D:\Study\Machine Learning\Jupitor Notebook\JupytorNotebookTest2\Udacity_Scripts\Matplotlib and seaborn Part2
File path : D:\Study\Machine Learning\Jupitor Notebook\JupytorNotebookTest2\Udacity_Scripts\Matplotlib and seaborn Part2\data\fuel_econ.csv
isfileExist : True
isadirectory : False
File extension : .csv
works also if __file__ is not available (jupyter notebooks)
import sys
from pathlib import Path
path_file = Path(sys.path[0])
print(path_file)
Also uses pathlib, which is the object oriented way of handling paths in python 3.
IPython has a magic command %pwd to get the present working directory. It can be used in following way:
from IPython.terminal.embed import InteractiveShellEmbed
ip_shell = InteractiveShellEmbed()
present_working_directory = ip_shell.magic("%pwd")
On IPython Jupyter Notebook %pwd can be used directly as following:
present_working_directory = %pwd
I have made a function to use when running python under IIS in CGI in order to get the current folder:
import os
def getLocalFolder():
path=str(os.path.dirname(os.path.abspath(__file__))).split(os.sep)
return path[len(path)-1]
Python 2 and 3
You can simply also do:
from os import sep
print(__file__.rsplit(sep, 1)[0] + sep)
Which outputs something like:
C:\my_folder\sub_folder\
This can be done without a module.
def get_path():
return (__file__.replace(f"<your script name>.py", ""))
print(get_path())