Lets say my python script is in a folder "/main". I have a bunch of text files inside subfolders in main. I want to be able to open a file just by specifying its name, not the subdirectory its in.
So open_file('test1.csv') should open test1.csv even if its full path is /main/test/test1.csv.
I don't have duplicated file names so it should no be a problem.
I using windows.
you could use os.walk to find your filename in a subfolder structure
import os
def find_and_open(filename):
for root_f, folders, files in os.walk('.'):
if filename in files:
# here you can either open the file
# or just return the full path and process file
# somewhere else
with open(root_f + '/' + filename) as f:
f.read()
# do something
if you have a very deep folder structure you might want to limit the depth of the search
import os
def get_file_path(file):
for (root, dirs, files) in os.walk('.'):
if file in files:
return os.path.join(root, file)
This should work. It'll return the path, so you should handle opening the file, in your code.
import os
def open_file(filename):
f = open(os.path.join('/path/to/main/', filename))
return f
Related
Essentially what I'm trying to do is loop through a directory that contains multiple sub-directories and within those run code against each file in a for loop.
The only start I managed to make was listing the directories but as I've rarely ever used os I'm not sure if I could potentially loop through os.chdir and a bit of f string formatting to loop through each subdirectory.
The files I want to run code against are just txt files.
Here goes my code, up to the moment:
import os
for folders in os.listdir('../main_directory'):
for something in os.listdir(f'{folders}'):
# run some function of sorts
pass
Any help would be greatly appreciated.
I like using pure os:
import os
for fname in os.listdir(src):
# build the path to the folder
folder_path = os.path.join(src, fname)
if os.path.isdir(folder_path):
# we are sure this is a folder; now lets iterate it
for file_name in os.listdir(folder_path):
file_path = os.path.join(folder_path, file_name)
# now you can apply any function assuming it is a file
# or double check it if needed as `os.path.isfile(file_path)`
Note that this function just iterate over the folder given at src and one more level:
src/foo.txt # this file is ignored
src/foo/a.txt # this file is processed
src/foo/foo_2/b.txt # this file is ignored; too deep.
src/foo/foo_2/foo_3/c.txt # this file is ignored; too deep.
In case you need to go as deep as possible, you can write a recursive function and apply it to every single file, as follows:
import os
def function_over_files(path):
if os.path.isfile(path):
# do whatever you need with file at path
else:
# this is a dir: we will list all files on it and call recursively
for fname in os.listdir(path):
f_path = os.path.join(path, fname)
# here is the trick: recursive call to the same function
function_over_files(f_path)
src = "path/to/your/dir"
function_over_files(src)
This way you can apply the function to any file under path, don't care how deep it is in the folder, now:
src/foo.txt # this file is processed; as each file under src
src/foo/a.txt # this file is processed
src/foo/foo_2/b.txt # this file is processed
src/foo/foo_2/foo_3/c.txt # this file is processed
You could try something like this:
for subdir, dirs, files in os.walk(rootdir):
Now you have "access" to all subdirs, dirs, and files for your main folder.
Hope it helps
I'd like to browse through the current folder and all its subfolders and get all the files with .htm|.html extensions. I have found out that it is possible to find out whether an object is a dir or file like this:
import os
dirList = os.listdir("./") # current directory
for dir in dirList:
if os.path.isdir(dir) == True:
# I don't know how to get into this dir and do the same thing here
else:
# I got file and i can regexp if it is .htm|html
and in the end, I would like to have all the files and their paths in an array. Is something like that possible?
You can use os.walk() to recursively iterate through a directory and all its subdirectories:
for root, dirs, files in os.walk(path):
for name in files:
if name.endswith((".html", ".htm")):
# whatever
To build a list of these names, you can use a list comprehension:
htmlfiles = [os.path.join(root, name)
for root, dirs, files in os.walk(path)
for name in files
if name.endswith((".html", ".htm"))]
I had a similar thing to work on, and this is how I did it.
import os
rootdir = os.getcwd()
for subdir, dirs, files in os.walk(rootdir):
for file in files:
#print os.path.join(subdir, file)
filepath = subdir + os.sep + file
if filepath.endswith(".html"):
print (filepath)
Hope this helps.
In python 3 you can use os.scandir():
def dir_scan(path):
for i in os.scandir(path):
if i.is_file():
print('File: ' + i.path)
elif i.is_dir():
print('Folder: ' + i.path)
dir_scan(i.path)
Use newDirName = os.path.abspath(dir) to create a full directory path name for the subdirectory and then list its contents as you have done with the parent (i.e. newDirList = os.listDir(newDirName))
You can create a separate method of your code snippet and call it recursively through the subdirectory structure. The first parameter is the directory pathname. This will change for each subdirectory.
This answer is based on the 3.1.1 version documentation of the Python Library. There is a good model example of this in action on page 228 of the Python 3.1.1 Library Reference (Chapter 10 - File and Directory Access).
Good Luck!
Slightly altered version of Sven Marnach's solution..
import os
folder_location = 'C:\SomeFolderName'
file_list = create_file_list(folder_location)
def create_file_list(path):
return_list = []
for filenames in os.walk(path):
for file_list in filenames:
for file_name in file_list:
if file_name.endswith((".txt")):
return_list.append(file_name)
return return_list
There are two ways works for me.
1. Work with the `os` package and use `'__file__'` to replace the main
directory when the project locates
import os
script_dir = os.path.dirname(__file__)
path = 'subdirectory/test.txt'
file = os.path.join(script_dir, path)
fileread = open(file,'r')
2. By using '\\' to read or write the file in subfolder
fileread = open('subdirectory\\test.txt','r')
from tkinter import *
import os
root = Tk()
file = filedialog.askdirectory()
changed_dir = os.listdir(file)
print(changed_dir)
root.mainloop()
I'm attempting to write a script that will save a file in the given directory, but I'm getting a NotADirecotryError[WinError 267] whenever I run it. Any ideas or tips on what I may have done incorrectly?
import shutil
import os
src = 'C:\\Users\\SpecificUsername\\Pictures\\test.txt\'
dest = 'C:\\Users\\SpecificUsername\\Desktop'
files = os.listdir(src)
for file in files:
shutil.copy(file, dest)
for file in files:
if os.path.isfile(file):
shutil.copy(file,dest) ```
There are a couple of things going on here:
You can just use forward slashes in the paths.
Your src is the test.txt file, and not a directory, so you cannot iterate over it using os.listdir().
You can also merge the two loops together since they are looping over the same set of data.
shutil.copy() takes a file path as input, while what you are passing is a filename.
The following code should work and it also copies directories as is:
import shutil
import os
basepath = "C:/Users/SpecificUsername/"
src = "Pictures/"
dest = "Desktop/"
files = os.listdir(os.path.join(basepath, src))
for filename in files:
filepath = os.path.join(basepath, src, filename)
if (os.path.isfile(filepath)):
print("File: " + filename)
shutil.copy(filepath,dest)
else:
print("Dir: " + filename)
shutil.copytree(filepath, os.path.join(dest, filename))
Hope it helps!
I am a total Python Newb
I need to loop through a directory looking for .txt files, and then read and process them individually. I would like to set this up so that whatever directory the script is in is treated as the root of this action. For example if the script is in /bsepath/workDir, then it would loop over all of the files in workDir and its children.
What I have so far is:
#!/usr/bin/env python
import os
scrptPth = os.path.realpath(__file__)
for file in os.listdir(scrptPth)
with open(file) as f:
head,sub,auth = [f.readline().strip() for i in range(3)]
data=f.read()
#data.encode('utf-8')
pth = os.getcwd()
print head,sub,auth,data,pth
This code is giving me an invalid syntax error and I suspect that is because os.listdir does not like file paths in standard string format. Also I dont think that I am doing the looped action right. How do I reference a specific file in the looped action? Is it packaged as a variable?
Any help is appriciated
import os, fnmatch
def findFiles (path, filter):
for root, dirs, files in os.walk(path):
for file in fnmatch.filter(files, filter):
yield os.path.join(root, file)
Use it like this, and it will find all text files somewhere within the given path (recursively):
for textFile in findFiles(r'C:\Users\poke\Documents', '*.txt'):
print(textFile)
os.listdir expects a directory as input. So, to get the directory in which the script resides use:
scrptPth = os.path.dirname(os.path.realpath(__file__))
Also, os.listdir returns just the filenames, not the full path.
So open(file) will not work unless the current working directory happens to be the directory where the script resides. To fix this, use os.path.join:
import os
scrptPth = os.path.dirname(os.path.realpath(__file__))
for file in os.listdir(scrptPth):
with open(os.path.join(scrptPth, file)) as f:
Finally, if you want to recurse through subdirectories, use os.walk:
import os
scrptPth = os.path.dirname(os.path.realpath(__file__))
for root, dirs, files in os.walk(scrptPth):
for filename in files:
filename = os.path.join(root, filename)
with open(filename, 'r') as f:
head,sub,auth = [f.readline().strip() for i in range(3)]
data=f.read()
#data.encode('utf-8')
I'd like to browse through the current folder and all its subfolders and get all the files with .htm|.html extensions. I have found out that it is possible to find out whether an object is a dir or file like this:
import os
dirList = os.listdir("./") # current directory
for dir in dirList:
if os.path.isdir(dir) == True:
# I don't know how to get into this dir and do the same thing here
else:
# I got file and i can regexp if it is .htm|html
and in the end, I would like to have all the files and their paths in an array. Is something like that possible?
You can use os.walk() to recursively iterate through a directory and all its subdirectories:
for root, dirs, files in os.walk(path):
for name in files:
if name.endswith((".html", ".htm")):
# whatever
To build a list of these names, you can use a list comprehension:
htmlfiles = [os.path.join(root, name)
for root, dirs, files in os.walk(path)
for name in files
if name.endswith((".html", ".htm"))]
I had a similar thing to work on, and this is how I did it.
import os
rootdir = os.getcwd()
for subdir, dirs, files in os.walk(rootdir):
for file in files:
#print os.path.join(subdir, file)
filepath = subdir + os.sep + file
if filepath.endswith(".html"):
print (filepath)
Hope this helps.
In python 3 you can use os.scandir():
def dir_scan(path):
for i in os.scandir(path):
if i.is_file():
print('File: ' + i.path)
elif i.is_dir():
print('Folder: ' + i.path)
dir_scan(i.path)
Use newDirName = os.path.abspath(dir) to create a full directory path name for the subdirectory and then list its contents as you have done with the parent (i.e. newDirList = os.listDir(newDirName))
You can create a separate method of your code snippet and call it recursively through the subdirectory structure. The first parameter is the directory pathname. This will change for each subdirectory.
This answer is based on the 3.1.1 version documentation of the Python Library. There is a good model example of this in action on page 228 of the Python 3.1.1 Library Reference (Chapter 10 - File and Directory Access).
Good Luck!
Slightly altered version of Sven Marnach's solution..
import os
folder_location = 'C:\SomeFolderName'
file_list = create_file_list(folder_location)
def create_file_list(path):
return_list = []
for filenames in os.walk(path):
for file_list in filenames:
for file_name in file_list:
if file_name.endswith((".txt")):
return_list.append(file_name)
return return_list
There are two ways works for me.
1. Work with the `os` package and use `'__file__'` to replace the main
directory when the project locates
import os
script_dir = os.path.dirname(__file__)
path = 'subdirectory/test.txt'
file = os.path.join(script_dir, path)
fileread = open(file,'r')
2. By using '\\' to read or write the file in subfolder
fileread = open('subdirectory\\test.txt','r')
from tkinter import *
import os
root = Tk()
file = filedialog.askdirectory()
changed_dir = os.listdir(file)
print(changed_dir)
root.mainloop()