I have a program that I want to make more dynamic. The current setup is literally typing everything out.
I would like the program to work with a for loop (any other suggestions would be great). My goal is to loop through a specific file that has sub-directories and get the name of each folder (sub-directory), then get the name of the files within the sub-directory.
To put this in a file string:
C:\Folder 1\Folder 2\File name
From the above, I would like to get the value of Folder 2 and File name.
My code so far:
for sub_dir in os.listdir(r"C:\Folder 1\"):
DIR = r'' + sub_dir
files_found = len([name for name in os.listdir(DIR) if os.path.isfile(os.path.join(DIR, name))])
if(files_found > 0):
for files in os.listdir(sub_dir):
file_name = os.path.splitext(files)[0]
I get the error --> FileNotFoundError: [WinError 3] The system cannot find the path specified: Folder 2
I appreciate your efforts to help.
You should take a look at os.walk
Have a look at os.walk()
It recursively traverses a file tree, returning a list of all directories and files within a directory at each step.
Related
So my program search_file.py is trying to look for .log files in the directory it is currently placed in. I used the following code to do so:
import os
# This is to get the directory that the program is currently running in
dir_path = os.path.dirname(os.path.realpath(__file__))
# for loop is meant to scan through the current directory the program is in
for root, dirs, files in os.walk(dir_path):
for file in files:
# Check if file ends with .log, if so print file name
if file.endswith('.log')
print(file)
My current directory is as follows:
search_file.py
sample_1.log
sample_2.log
extra_file (this is a folder)
And within the extra_file folder we have:
extra_sample_1.log
extra_sample_2.log
Now, when the program runs and prints the files out it also takes into account the .log files in the extra_file folder. But I do not want this. I only want it to print out sample_1.log and sample_2.log. How would I approach this?
Try this:
import os
files = os.listdir()
for file in files:
if file.endswith('.log'):
print(file)
The problem in your code is os.walk traverses the whole directory tree and not just your current directory. os.listdir returns a list of all filenames in a directory with the default being your current directory which is what you are looking for.
os.walk documentation
os.listdir documentation
By default, os.walk does a root-first traversal of the tree, so you know the first emitted data is the good stuff. So, just ask for the first one. And since you don't really care about root or dirs, use _ as the "don't care" variable name
# get root files list.
_, _, files = next(os.walk(dir_path))
for file in files:
# Check if file ends with .log, if so print file name
if file.endswith('.log')
print(file)
Its also common to use glob:
from glob import glob
dir_path = os.path.dirname(os.path.realpath(__file__))
for file in glob(os.path.join(dir_path, "*.log")):
print(file)
This runs the risk that there is a directory that ends in ".log", so you could also add a testing using os.path.isfile(file).
so I have a directory with a few subdirectories in it and I am trying to iterate through all the subdirectories I have (each subdirectory has a bunch of files in it that I'm splitting up into smaller files). i've been trying to use os.listdir but I keep getting this error FileNotFoundError: [Errno 2] No such file or directory: 'mFAPA'
This subdirectory definitely exists, so I'm not sure why this keeps happening
for dir in os.listdir('../conv_files'):
for filename in os.listdir(dir):
I was trying to use the for loops to go through each directory and then in each directory go through each file. The error is on the second line of code, once it's in the parent directory it for some reason can't do the for filename in os.listdir(dir) part. Any suggestions?
You can use os.walk() which traverses into each subdirectory and files inside a given directory. Refer https://www.geeksforgeeks.org/os-walk-python/ for more details
for (root,dirs,files) in os.walk('../conv_files'):
#add your code here
syntax: os.listdir(path)
Parameters:
path (optional) : path of the directory
Return Type: This method returns the list of all files and directories in the specified path. The return type of this method is list.
in your first nested loop, it consists of the file names but os.listdir(path) requires path in it.
I hope I don't duplicate here, but I didn't find a solution until now since the answers don't include subfolders. I have a zipfile that contains a folder which contains files and subfolders.
I want to extract the files within the folder (my_folder) and the subfolder to a specific path: Users/myuser/Desktop/another . I want only files and subfolders in the another dir. With my current code what happens it that a directory my_folder is created in which my files and subfolders are placed. But I don't want that directory created. This is what I am doing:
with zipfile.ZipFile("Users/myuser/Desktop/another/my_file.zip", "r") as zip_ref:
zip_ref.extractall(Users/myuser/Desktop/another)
I tried listing all the zipfiles within the folder and extracting them manually:
with ZipFile('Users/myuser/Desktop/another/myfile.zip', 'r') as zipObj:
# Get a list of all archived file names from the zip
listOfFileNames = zipObj.namelist()
for fileName in new_list_of_fn:
print(fileName)
zipObj.extract(fileName, 'Users/myuser/Desktop/another/')
This yields the same result. I the tried create a new list, stripping the names so that they don't include the name of the folder anymore but then it tells me that there is no item named xyz in the archive.
Finally I leveraged those two questions/code (extract zip file without folder python and Extract files from zip without keeping the structure using python ZipFile?) and this works, but only if there are no subfolders involved. If there are subfolders it throws me the error FileNotFoundError: [Errno 2] No such file or directory: ''. What I want though is that the files in the subdirectory get extracted to the subdirectory.
I can only use this code if I skip all directories:
my_zip = Users/myuser/Desktop/another/myfile.zip
my_dir = Users/myuser/Desktop/another/
with zipfile.ZipFile(my_zip, 'r') as zip_file:
for member in zip_file.namelist():
filename = os.path.basename(member)
print(filename)
# skip directories
if not filename:
continue
# copy file (taken from zipfile's extract)
source = zip_file.open(member)
target = open(os.path.join(my_dir, filename), "wb")
with source, target:
shutil.copyfileobj(source, target)
So I am looking for a way to do this which would also extract subdirs to their respective dir. That means I want a structure within /Users/myuser/Desktop/another:
-file1
-file2
-file3
...
- subfolder
-file1
-file2
-file3
...
I have the feeling this must be doable with shututil but don't really know how....
Is there a way I can do this? Thanks so much for any help. Very much appreciated.
I am trying to rename all files and folders in a given directory. Id like to replace a space with a hyphen and then rename all to lowercase. I am stuck with the code below. When os.rename is commented out, the print function returns all files as expected, however when I uncomment os.rename I get an error stating that file XYZ -> x-y-z cant be found.
import os
folder = r"C:\Users\Tim\Documents\storage"
space = " "
hyphen = "-"
for root, dirs, files in os.walk(folder):
for file in files:
if space in file:
newFilename = filename.replace(space, hyphen).lower()
os.rename(file, newFilename)
print(newFilename)
Obvioulsy this is just for the files but I'd like to apply the same logic to folders too. Any help would be greately appreciated. Pretty new at Python so this is a little beyond me! Thanks so much.
os.rename() resolves relative file paths (paths that doesn't start with a / in Linux/Mac or a drive letter in Windows) relative to the current working directory.
You'll want to os.path.join() the names with root before passing it to os.rename(), as otherwise rename would look for the file with that name in the current working directory instead of in the original folder.
So it should be:
os.rename(os.path.join(root, file), os.path.join(root, newFilename))
I am having issues reading the contents of the files I am trying to open due to the fact that python believes there is:
"No such file or directory: 'Filename.xrf'"
Here is an outline of my code and what I think the problem may be:
The user's input defines the path to where the files are.
direct = str(raw_input("Enter directory name where your data is: ))
path = "/Users/myname/Desktop/heasoft/XRF_data/%s/40_keV" \
%(direct)
print os.listdir(path)
# This lists the correct contents of the directory that I wanted it to.
So here I essentially let the user decide which directory they want to manipulate and then I choose one more directory path named "40_keV".
Within a defined function I use the OS module to navigate to the corresponding directory and then append every file within the 40_keV directory to a list, named dataFiles.
def Spectrumdivide():
dataFiles = []
for root, dirs, files in os.walk(path):
for file in files:
if file.endswith('.xrf'):
dataFiles.append(file)
Here, the correct files were appended to the list 'dataFiles', but I think this may be where the problem is occurring. I'm not sure whether or not Python is adding the NAME of the file to my list instead of the actual file object.
The code breaks because python believes there is no such file or directory.
for filename in dataFiles:
print filename
f = open(filename,'r') # <- THE CODE BREAKS HERE
print "Opening file: " + filename
line_num = f.readlines()
Again, the correct file is printed from dataFiles[0] in the first iteration of the loop but then this common error is produced:
IOError: [Errno 2] No such file or directory: '40keV_1.xrf'
I'm using an Anaconda launcher to run Spyder (Python 2.7) and the files are text files containing two columns of equal length. The goal is to assign each column to a list and the manipulate them accordingly.
You need to append the path name not just the file's name using the os.path.join function.
for root, dirs, files in os.walk(path):
for file in files:
if file.endswith('.xrf'):
dataFiles.append(os.path.join(root, file))