Last week I wrote a simple code that would find "XXXX" in a folder's subfolders and files and replace it with a four digit project number (my office has template folders for new projects with many subfolders/files that all are prenamed with XXXX to be replaced with the project number, and I was tired of seeing many of these remaining unchanged throughout a project...)
Anyway, I used it on several project folders last week after writing it and tried running it again today for a new project folder, and the Shell returns
====== RESTART: ======
Python Shell return
I hadn't modified the code nor its file location so am quite confused and related threads did not lead me to resolving this.
Here is the code:
import os
basedir = 'K:\Projects\1702 HANH Stanley Justice'
find = "XXXX"
replace = "1702"
for root, dirs, filenames in os.walk(basedir, topdown=False):
dirs[:] = [d for d in dirs]
for filename in filenames:
filename_split = os.path.splitext(filename)
filename_zero = filename_split[0]
extension = filename_split[1]
if find in filename_zero:
path1 = os.path.join(root, filename)
path2 = os.path.join(root, filename_zero.replace(find, replace) + extension)
os.rename(path1, path2)
print ("file: " + path1 + " renamed to: " + path2)
for root, dirs, filenames in os.walk(basedir, topdown=False):
dirs[:] = [d for d in dirs]
for thedir in dirs:
if find in thedir:
path1 = os.path.join(root, thedir)
path2 = os.path.join(root, thedir.replace(find, replace))
os.rename(path1, path2)
print ("dir: " + path1 + " renamed to: " + path2)
Thank you in advance to whoever can advise me on how to fix this!
Haven't had enough coffee yet today and had copied and pasted the new project folder path without replacing the back slashes with forward slashes. Worked properly when replacing those:
basedir = 'K:/Projects/1702 HANH Stanley Justice'
I'll leave this up for anyone else who experiences this caffeine deprived lapse in memory or for someone searching for a code that will find and replace a string in subfolder and file names (mine was created from combining similarly related codes I found on other threads here, was unable to find an exact one. I'm fairly new to Python so stackoverflow is very helpful!)
Related
I would like some help to loop through some directories and subdirectories and extracting data. I have a directory with three levels, with the third level containing several .csv.gz files. The structure is like this
I need to access level 2 (where subfolders are) of each folder and check the existence of a specific folder (in my example, this will be subfolder 3; I left the other folders empty for this example, but in real cases they will have data). If checking returns True, then I want to change the name of files within the target subfolder3 and transfer all files to another folder.
Bellow is my code. It is quite cumbersome and there is probably better ways of doing it. I tried using os.walk() and this is the closest I got to a solution but it won't move the files.
import os
import shutil
def organizer(parent_dir, target_dir, destination_dir):
for root, dirs, files in os.walk(parent_dir):
if root.endswith(target_dir):
target = root
for files in os.listdir(target):
if not files.startswith("."):
# this is to change the name of the file
fullname = files.split(".")
just_name = fullname[0]
csv_extension = fullname[1]
gz_extension = fullname[2]
subject_id = target
#make a new name
origin = subject_id + "/"+ just_name + "." + csv_extension + "." + gz_extension
#make a path based on this new name
new_name = os.path.join(destination_dir, origin)
#move file from origin folder to destination folder and rename the file
shutil.move(origin, new_name)
Any suggestions on how to make this work and / or more eficient?
simply enough, you can use the built-in os module, with os.walk(path) returns you root directories and files found
import os
for root, _, files in os.walk(path):
#your code here
for your problem, do this
import os
for root, dirs, files in os.walk(parent_directory);
for file in files:
#exctract the data from the "file"
check this for more information os.walk()
and if you want to get the name of the file, you can use os.path.basename(path)
you can even check for just the gzipped csv files you're looking for using built-in fnmatch module
import fnmathch, os
def find_csv_files(path):
result = []
for root, _, files in os.walk(path):
for name in files:
if fnmatch.fnmatch(name, "*.csv.gz"): # find csv.gz using regex paterns
result.append(os.path.join(root, name))
return list(set(results)) #to get the unique paths if for some reason duplicated
Ok, guys, I was finally able to find a solution. Here it is. Not the cleanest one, but it works in my case. Thanks for the help.
def organizer(parent_dir, target_dir, destination_dir):
for root, dirs, files in os.walk(parent_dir):
if root.endswith(target_dir):
target = root
for files in os.listdir(target):
#this one because I have several .DS store files in the folder which I don't want to extract
if not files.startswith("."):
fullname = files.split(".")
just_name = fullname[0]
csv_extension = fullname[1]
gz_extension = fullname[2]
origin = target + "/" + files
full_folder_name = origin.split("/")
#make a new name
new_name = full_folder_name[5] + "_"+ just_name + "." + csv_extension + "." + gz_extension
#make a path based on this new name
new_path = os.path.join(destination_dir, new_name)
#move file from origin folder to destination folder and rename the file
shutil.move(origin, new_path)
The guess the problem was that was passing a variable that was a renamed file (in my example, I wrongly called this variable origin) as the origin path to shutil.move(). Since this path does not exist, then the files weren't moved.
I have a directory which contains different file names. The other directory I have is a directory of directories, such that each directory has the same name of the filename in the first directory.
What I want to do is that I would like to check if a file exists in the directory of directories through its name.
While working on this I had to make different for-loops and was a bit confusing. Is there a simpler way to do that in Python?
Well, here's what I did so far:
import os
directory_of_files_path = '/home/user/directory_of_files'
directory_of_directories_path = '/home/user/directory_of_directories'
i = 0
for root_pairs, dirs_pairs, files_pairs in os.walk(directory_of_files_path):
for root_aligned, dirs_aligned, files_aligned in os.walk(directory_of_directories_path):
for file in files_pairs:
for directory in dirs_aligned:
filename, file_extension = os.path.splitext(file)
if filename == directory:
i = i + 1
As you can see, in the above code I was able to return the number of files included in the directory of directories (based on name). But, couldn't figure out to check those that are not included in the directory of directories.
Thanks.
Don't know if I understood what you wanted to do but... here is my guess:
This is how I made the files and dirs
import os
dir_files = 'dir_files'
dir_dir = 'dir_dir'
i = 0
for rf, df, ff in os.walk(dir_files):
ff2 = ff
for rd, dd, fd in os.walk(dir_dir):
for dirs in dd:
if dirs + ".txt" in ff:
i += 1
ff2.remove(dirs + ".txt")
print("There are ", i, "dirs that matches with the files")
print("not found dir corresponding to {}".format(ff2))
Output
I am doing a small project using a book about Python. One of the examples they show is how to rename all of the files and directories right underneath the root folder, however I want to go a step further and change files within sub directories.
I figured using os.walk would be my best bet but I can only get it to change file names not subdirectories. I would like to also change subdirectory names as I go through os.walk.
I am using a book and online resources to accomplish this, any help would be awesome.
Here is what I have right now based on the book. I have tried to include os.listdir but the regex won't work like that.
import shutil, os, re
#regex that matches files with dates
wrongFormat = re.compile(r"""^(.*?)
((0|1)?\d).
((0|1|2|3)?\d).
((19|20)\d\d)
(.*?)$
""", re.VERBOSE)
#Loop over the files in the working directory
path = '.'
for path, dirs, files in os.walk(path, topdown=True):
for incorrectDt in files:
dt = wrongFormat.search(incorrectDt)
#Skip file without a date
if dt == None:
continue
#Split filename into parts
beforePart = dt.group(1)
monthPart = dt.group(2)
dayPart = dt.group(4)
yearPart = dt.group(6)
afterPart = dt.group(8)
#Form the new date format
correctFormat = beforePart + monthPart + "_" + dayPart + "_" + yearPart + afterPart
#Get full, absolute file paths.
absWorkingDir = os.path.abspath(path)
incorrectDt= os.path.join(absWorkingDir, incorrectDt)
correctFormat = os.path.join(absWorkingDir, correctFormat)
#rename files
print ('Renaming "%s" to "%s"...' % (wrongFormat, correctFormat))
shutil.move(incorrectDt, correctFormat)
I have a directory called /user/local/ inside which i have several files of the form, jenjar.dat_1 and jenmis.dat_1. There is another directory /user/data inside which there are two subdirectories of the form, jenjar and jenmis. I need a Python code that would move the jenjar.dat_1 into the jenjar directory of /user/data and similarly, jenmis.dat_1 into jenmis directory of '/user/data.
I guess the os module would work for thus but I'm confused. Most of the questions here do not show a Pythonic way to do this.
EDIT: I have found the solution to this
destination = '/user/local'
target = '/user/data'
destination_list = os.listdir(destination)
data_dir_list = os.listdir(target)
for fileName in destination_list:
if not os.path.isdir(os.path.join(destination, fileName)):
for prefix in data_dir_list:
if fileName.startswith(prefix):
shutil.copy(os.path.join(destination, fileName), os.path.join(target, prefix, fileName))
This should do the trick
srcDir = '/user/local'
targetDir = '/user/data'
for fname in os.listdir(srcDir):
if not os.path.isdir(os.path.join(srcDir, fname)):
for prefix in ['jenjar.dat', 'jenmis.dat']:
if fname.startswith(prefix):
if not os.path.isdir(os.path.join(targetDir, prefix)):
os.mkdir(os.path.join(targetDir, prefix))
shutil.move(os.path.join(srcDir, fnmae), targetDir)
I have some files that I'm working with in a python script. The latest requirement is that I go into a directory that the files will be placed in and rename all files by adding a datestamp and project name to the beginning of the filename while keeping the original name.
i.e. foo.txt becomes 2011-12-28_projectname_foo.txt
Building the new tag was easy enough, it's just the renaming process that's tripping me up.
Can you post what you have tried?
I think you should just need to use os.walk with os.rename.
Something like this:
import os
from os.path import join
for root, dirs, files in os.walk('path/to/dir'):
for name in files:
newname = foo + name
os.rename(join(root,name),join(root,newname))
I know this is an older post of mine, but seeing as how it's been viewed quite a few times I figure I'll post what I did to resolve this.
import os
sv_name="(whatever it's named)"
today=datetime.date.today()
survey=sv_name.replace(" ","_")
date=str(today).replace(" ","_")
namedate=survey+str(date)
[os.rename(f,str(namedate+"_"+f)) for f in os.listdir('.') if not f.startswith('.')]
import os
dir_name = os.path.realpath('ur directory')
cnt=0 for root, dirs, files in os.walk(dir_name, topdown=False):
for file in files:
cnt=cnt+1
file_name = os.path.splitext(file)[0]#file name no ext
extension = os.path.splitext(file)[1]
dir_name = os.path.basename(root)
try:
os.rename(root+"/"+file,root+"/"+dir_name+extension)
except FileExistsError:
os.rename(root+"/"+file,root+""+dir_name+str(cnt)+extension)
to care if more files are there in single folder and if we need to give incremental value for the files