I have a small text (XML) file that I want a Python function to load. The location of the text file is always in a fixed relative position to the Python function code.
For example, on my local computer, the files text.xml and mycode.py could reside in:
/a/b/text.xml
/a/c/mycode.py
Later at run time, the files could reside in:
/mnt/x/b/text.xml
/mnt/x/c/mycode.py
How do I ensure I can load in the file? Do I need the absolute path? I see that I can use os.path.isfile, but that presumes I have a path.
you can do a call as follows:
import os
BASE_DIR = os.path.dirname(os.path.realpath(__file__))
This will get you the directory of the python file you're calling from mycode.py
then accessing the xml files is as simple as:
xml_file = "{}/../text.xml".format(BASE_DIR)
fin = open(xml_file, 'r+')
If the parent directory of the two directories are always the same this should work:
import os
path_to_script = os.path.realpath(__file__)
parent_directory = os.path.dirname(path_to_script)
for root, dirs, files in os.walk(parent_directory):
for file in files:
if file == 'text.xml':
path_to_xml = os.path.join(root, file)
You can use the special variable __file__ which gives you the current file name (see http://docs.python.org/2/reference/datamodel.html).
So in your first example, you can reference text.xml this way in mycode.py:
xml_path = os.path.join(__file__, '..', '..', 'text.xml')
Related
Essentially what I'm trying to do is loop through a directory that contains multiple sub-directories and within those run code against each file in a for loop.
The only start I managed to make was listing the directories but as I've rarely ever used os I'm not sure if I could potentially loop through os.chdir and a bit of f string formatting to loop through each subdirectory.
The files I want to run code against are just txt files.
Here goes my code, up to the moment:
import os
for folders in os.listdir('../main_directory'):
for something in os.listdir(f'{folders}'):
# run some function of sorts
pass
Any help would be greatly appreciated.
I like using pure os:
import os
for fname in os.listdir(src):
# build the path to the folder
folder_path = os.path.join(src, fname)
if os.path.isdir(folder_path):
# we are sure this is a folder; now lets iterate it
for file_name in os.listdir(folder_path):
file_path = os.path.join(folder_path, file_name)
# now you can apply any function assuming it is a file
# or double check it if needed as `os.path.isfile(file_path)`
Note that this function just iterate over the folder given at src and one more level:
src/foo.txt # this file is ignored
src/foo/a.txt # this file is processed
src/foo/foo_2/b.txt # this file is ignored; too deep.
src/foo/foo_2/foo_3/c.txt # this file is ignored; too deep.
In case you need to go as deep as possible, you can write a recursive function and apply it to every single file, as follows:
import os
def function_over_files(path):
if os.path.isfile(path):
# do whatever you need with file at path
else:
# this is a dir: we will list all files on it and call recursively
for fname in os.listdir(path):
f_path = os.path.join(path, fname)
# here is the trick: recursive call to the same function
function_over_files(f_path)
src = "path/to/your/dir"
function_over_files(src)
This way you can apply the function to any file under path, don't care how deep it is in the folder, now:
src/foo.txt # this file is processed; as each file under src
src/foo/a.txt # this file is processed
src/foo/foo_2/b.txt # this file is processed
src/foo/foo_2/foo_3/c.txt # this file is processed
You could try something like this:
for subdir, dirs, files in os.walk(rootdir):
Now you have "access" to all subdirs, dirs, and files for your main folder.
Hope it helps
I am currently accessing a script that opens a file in the directory it's located in. I am accessing this file from both the main.py file located in the same directory, as well as a testfile which is located in a "Test" subdirectory. Trying to use a file from the Test subdirectory to call the function that opens the file causes the script to try and open it from the Test directory instead of the super directory, since I am opening the file simply by calling it as following:
with open(filename,"w") as f:
Is there a way to define the location of the file in a way that makes it possible for the script opening it to be called from anywhere?
Use __file__ to get the path to the current script file, then find the file relative to that:
# In main.py: find the file in the same directory as this script
import os.path
open(os.path.join(os.path.dirname(__file__), 'file.txt'))
# In Test/test.py: find the file one directory up from this script
import os.path
open(os.path.join(os.path.dirname(__file__), '..', 'file.txt'))
just give the absolute file path instead of giving a relative one
for eg
abs_path = '/home/user/project/file'
with open(abs_path, 'r') as f:
f.write(data)
Try specifying the path:
import os
path = 'Your path'
path = os.path.abspath(path)
with open(path, 'w') as f:
f.write(data)
From what I understood your file is in a directory parent_directory/file_name.txt
and in another folder parent_directory/sub_directory/file_name.txt. All you have to do is paste the below code in both parent and sub directories.
import os
file_name = 'your_file_name'
# if the file is in current directory set the path to file_name
if file_name in os.listdir(os.getcwd()):
path = file_name
# if the path is not in current directory go back to parent directory set the path to parent directory
else:
path = os.path.abspath(os.path.join(os.getcwd(), os.pardir))
print('from',os.getcwd())
with open(path, 'r') as filename:
print(filename.read())
I am trying to write a python script to use the linux command wc to input the amount of lines in a file. I am iterating through a directory inputted by the user. However, whenever I get the absolute path of a file in the directory, it skips the directory it is in. So, the path isn't right and when I call wc on it, it doesn't work because it is trying to find the file in the directory above. I have 2 test text files in a directory called "testdirectory" which is located directly under "projectdirectory".
Script file:
import subprocess
import os
directory = raw_input("Enter directory name: ")
for root,dirs,files in os.walk(os.path.abspath(directory)):
for file in files:
file = os.path.abspath(file)
print(path) #Checking to see the path
subprocess.call(['wc','l',file])
This is what I get when running the program:
joe#joe-VirtualBox:~/Documents/projectdirectory$ python project.py
Enter directory name: testdirectory
/home/joe/Documents/projectdirectory/file2
wc: /home/joe/Documents/projectdirectory/file2: No such file or directory
/home/joe/Documents/projectdirectory/file1
wc: /home/joe/Documents/projectdirectory/file1: No such file or directory
I don't know why the path isn't /home/joe/Documents/projectdirectory/testdirectory/file2 since that is where the file is located.
You're using the output of os.walk wrong.
abspath is related to your program's current working directory, whereas your files are in the directory as specified by root. So you want to use
file = os.path.join(root, file)
Your issue is in the use of os.path.abspath(). All that this function does is appends the current working directory onto whatever the argument to the function is. You also need to have a - before the l option for wc. I think this fix might help you:
import os
directory = input("Enter directory name: ")
full_dir_path = os.path.abspath(directory)
for root,dirs,files in os.walk(full_dir_path):
for file in files:
full_file_path = os.path.join(root, file)
print(full_file_path) #Checking to see the path
subprocess.call(['wc','-l',full_file_path])
I have a Python script which opens a specific text file located in a specific directory (working directory) and perform some actions.
(Assume that if there is a text file in the directory then it will always be no more than one such .txt file)
with open('TextFileName.txt', 'r') as f:
for line in f:
# perform some string manipulation and calculations
# write some results to a different text file
with open('results.txt', 'a') as r:
r.write(someResults)
My question is how I can have the script locate the text (.txt) file in the directory and open it without explicitly providing its name (i.e. without giving the 'TextFileName.txt'). So, no arguments for which text file to open would be required for this script to run.
Is there a way to achieve this in Python?
You could use os.listdir to get the files in the current directory, and filter them by their extension:
import os
txt_files = [f for f in os.listdir('.') if f.endswith('.txt')]
if len(txt_files) != 1:
raise ValueError('should be only one txt file in the current directory')
filename = txt_files[0]
You Can Also Use glob Which is easier than os
import glob
text_file = glob.glob('*.txt')
# wild card to catch all the files ending with txt and return as list of files
if len(text_file) != 1:
raise ValueError('should be only one txt file in the current directory')
filename = text_file[0]
glob searches the current directory set by os.curdir
You can change to the working directory by setting
os.chdir(r'cur_working_directory')
Since Python version 3.4, it is possible to use the great pathlib library. It offers a glob method which makes it easy to filter according to extensions:
from pathlib import Path
path = Path(".") # current directory
extension = ".txt"
file_with_extension = next(path.glob(f"*{extension}")) # returns the file with extension or None
if file_with_extension:
with open(file_with_extension):
...
Lets say my python script is in a folder "/main". I have a bunch of text files inside subfolders in main. I want to be able to open a file just by specifying its name, not the subdirectory its in.
So open_file('test1.csv') should open test1.csv even if its full path is /main/test/test1.csv.
I don't have duplicated file names so it should no be a problem.
I using windows.
you could use os.walk to find your filename in a subfolder structure
import os
def find_and_open(filename):
for root_f, folders, files in os.walk('.'):
if filename in files:
# here you can either open the file
# or just return the full path and process file
# somewhere else
with open(root_f + '/' + filename) as f:
f.read()
# do something
if you have a very deep folder structure you might want to limit the depth of the search
import os
def get_file_path(file):
for (root, dirs, files) in os.walk('.'):
if file in files:
return os.path.join(root, file)
This should work. It'll return the path, so you should handle opening the file, in your code.
import os
def open_file(filename):
f = open(os.path.join('/path/to/main/', filename))
return f