Develop Reference

Python - move all files from one folder to another if their file names contain specified words

I have a folder with many files named like homeXXX_roomXXX_high.csv or homeXXX_roomXXX_low.csv, where the XXX part is replaced with a three-digit number.
I want to use some code to move the files into separate folders based on the number next to "home" in the filename. For example, I want to specify that files with names starting home101, home103, home320, home553, etc. should all be moved into folder A whereas those starting with home555, home431, home105 should go to FolderB.
I have this code so far:
import shutil
import os
source = '/path/to/source_folder'
dest1 = '/path/to/FolderA'
dest2 = '/path/to/FolderB'
files = os.listdir(source)
for f in files:
if (f.startswith("home101") or f.startswith("home103")):
shutil.move(f, dest1)
elif (f.startswith("home431") or f.startswith("home555")):
shutil.move(f, dest2)
However, it's tedious to specify all the if and else cases. I'd like to use some kind of structured data, such as a list, to specify groups of "home" numbers and the corresponding folder paths. How can I do this in Python?

it seems like you can use another for, it would look something like this:
import shutil
import os
source = '/path/to/source_folder'
dest1 = '/path/to/FolderA'
dest2 = '/path/to/FolderB'
list1 = ["home101", "home103"]
list2 = ["home431", "home555"]
files = os.listdir(source)
for f in files:
for home1 in list1:
if f.startswith(home1):
shutil.move(f, dest1)
break
for home2 in list2:
if f.startswith(home2):
shutil.move(f, dest2)
break
You can also create a function:
def check_and_move(file, list_of_patterns, destination):
for pattern in list_of_patterns:
if file.startswith(pattern):
shutil.move(file, destination)
and the code will get cleaner because you avoid repetition :)
for f in files:
check_and_move(f, list1, dest1)
check_and_move(f, list2, dest2)
# etc...

You can make an array for folderA that contains the "home+number"
FolderAGroup = ['home101', 'home103', 'homeXXX', 'homeXXX']
And if they get split like you say with a "_" use this code to filter them
Won't work if they are not split like that.
files = os.listdir(source)
for f in files:
parts = f.split('_')
# Get the first part of the filename before the _
home_number = parts[0]
# Check if the home number is in the FolderA group array
if home_number in FolderAGroup:
shutil.move(f, dest1)
else:
shutil.move(f, dest2)
You can expand with more elif statements if you would want more folders.

If the names homexxx are incremental, you could try something like this:
home_names_list_1 = []
home_names_list_2 = []
for i in range(100):
home_names_list_1.append("home" + str(i))
for i in range(100,200):
home_names_list_2.append("home" + str(i))
for file in files:
moved = False
for name in home_names_list_1:
if file.startswith(name):
print("move somewhere")
moved = True
break
if moved:
break
for name in home_names_list_2:
if file.startswith(name):
print("move somewhere else")
break
print(" did not move because did not match anything")

Change only a part of filename

I have these images in my folder:
area11.tif
area12.tif
area14.tif
area21.tif
area22.tif
area25.tif
How can I change only the last digit so they became ordered and "more incremental" ?
Instead if area14.tif it should be area13.tif and the same thing for area22/area25.
I have a code but it's a bit broken because it delete some files (it's strange, I know...).
EDIT: added (maybe broken..) code
try:
path = (os.path.expanduser('~\\FOLDER\\'))
files = os.listdir(path)
idx = 0
for file in files:
idx =+ 1
i = 'ex_area'
if file.endswith('.tif'):
i = i + str(idx)
os.rename(os.path.join(path, file), os.path.join(path, str(i) + '.tif'))
except OSError as e:
if e.errno != errno.EEXIST:
raise

1) Read the files names in the directory into array (of strings).
2) Iterate over the array of filenames
3) For each filename, slice the string and insert the index
4) Rename
For example:
import os
import glob
[os.rename(n, "{}{}.tif".format(n[:5], i)) for i, n in enumerate(glob.glob("area*"))]

First you get the list of the images pathes with the glob module :
images = glob.glob("/sample/*.tif")
then you just rename all of them with the os module :
for i in range(len(images)): os.rename(images[i], ‘area’+i+’.tif’)

First rename all filename to temp name and then add whatever name you prefer
import glob,os
images = glob.glob("*.tif")
for i in range(len(images)):
os.rename(images[i], 'temp_'+str(i)+'.tif')
tempImages = glob.glob("temp*.tif")
for i in range(len(tempImages)):
os.rename(tempImages[i], 'area'+str(i+1)+'.tif')

Found also this other solution. But there is a small difference in this one, and a better way of do the job in the end (at least for me): create a folder for each area. So simple I didn't think of it before...
BTW, here is the code, commented. I am using this one just because I achieved what I want. Thanks to all who answered, made me learn new things.
path = (os.path.expanduser('~\\FOLDER\\AREA1\\')) #select folder
files = os.listdir(path)
i = 1 #counter
name = 'area' #variable which the file will take as name
for file in files:
if file.endswith('.tif'): #search only for .tif. Can change with any supported format
os.rename(os.path.join(path, file), os.path.join(path, name + str(i)+'.tif')) #name + str(i)+'.tif' will take the name and concatenate to first number in counter. #If you put as name "area1" the str(i) will add another number near the name so, here is the second digit.
i += 1 #do this for every .tif file in the folder
It's a little bit simple, but because I put the files in two separate folders. If you keep the files in the same folder, this will not work properly.
EDIT: now that I see, it's the same as my code above....

Rename multiple files inside multiple folders

So I have a lot of folders with a certain name. In each folder I have +200 items. The items inside the folders has names like:
CT.34562346.246.dcm
RD.34562346.dcm
RN.34562346.LAO.dcm
And some along that style.
I now wish to rename all files inside all folders so that the number (34562346) is replaced with the name of the folder. So for example in the folder named "1" the files inside should become:
CT.1.246.dcm
RD.1.dcm
RN.1.LAO.dcm
So only the large number is replaced. And yes, all files are similar like this. It would be the number after the first . that should be renamed.
So far I have:
import os
base_dir = "foo/bar/" #In this dir I have all my folders
dir_list = []
for dirname in os.walk(base_dir):
dir_list.append(dirname[0])
This one just lists the entire paths of all folders.
dir_list_split = []
for name in dir_list[1:]: #The 1 is because it lists the base_dir as well
x = name.split('/')[2]
dir_list_split.append(x)
This one extracts the name of each folder.
And then the next thing would be to enter the folders and rename them. And I'm kind of stuck here ?

The pathlib module, which was new in Python 3.4, is often overlooked. I find that it often makes code simpler than it would otherwise be with os.walk.
In this case, .glob('**/*.*') looks recursively through all of the folders and subfolders that I created in a sample folder called example. The *.* part means that it considers all files.
I put path.parts in the loop to show you that pathlib arranges to parse pathnames for you.
I check that the string constant '34562346' is in its correct position in each filename first. If it is then I simply replace it with the items from .parts that is the next level of folder 'up' the folders tree.
Then I can replace the rightmost element of .parts with the newly altered filename to create the new pathname and then do the rename. In each case I display the new pathname, if it was appropriate to create one.
>>> from pathlib import Path
>>> from os import rename
>>> for path in Path('example').glob('**/*.*'):
... path.parts
... if path.parts[-1][3:11]=='34562346':
... new_name = path.parts[-1].replace('34562346', path.parts[-2])
... new_path = '/'.join(list(path.parts[:-1])+[new_name])
... new_path
... ## rename(str(path), new_path)
... else:
... 'no change'
...
('example', 'folder_1', 'id.34562346.6.a.txt')
'example/folder_1/id.folder_1.6.a.txt'
('example', 'folder_1', 'id.34562346.wax.txt')
'example/folder_1/id.folder_1.wax.txt'
('example', 'folder_2', 'subfolder_1', 'ty.34562346.90.py')
'example/folder_2/subfolder_1/ty.subfolder_1.90.py'
('example', 'folder_2', 'subfolder_1', 'tz.34562346.98.py')
'example/folder_2/subfolder_1/tz.subfolder_1.98.py'
('example', 'folder_2', 'subfolder_2', 'doc.34.34562346.implication.rtf')
'no change'

This will rename files in subdirectories too:
import os
rootdir = "foo" + os.sep + "bar"
for subdir, dirs, files in os.walk(rootdir):
for file in files:
filepath = subdir + os.sep + file
foldername = subdir.split(os.sep)[-1]
number = ""
foundnumber = False
for c in filepath:
if c.isdigit():
foundnumber = True
number = number + c
elif foundnumber:
break
if foundnumber:
newfilepath = filepath.replace(number,foldername)
os.rename(filepath, newfilepath)

Split each file name on the . and replace the second item with the file name, then join on .'s again for the new file name. Here's some sample code that demonstrates the concept.
folder_name = ['1', '2']
file_names = ['CT.2345.234.dcm', 'BG.234234.222.dcm', "RA.3342.221.dcm"]
for folder in folder_name:
new_names = []
for x in file_names:
file_name = x.split('.')
file_name[1] = folder
back_together = '.'.join(file_name)
new_names.append(back_together)
print(new_names)
Output
['CT.1.234.dcm', 'BG.1.222.dcm', 'RA.1.221.dcm']
['CT.2.234.dcm', 'BG.2.222.dcm', 'RA.2.221.dcm']

Rename many files sequentially Python

My training on Python is ongoing and I'm currently trying to rename sequentially many files that have this kind of root and extension:
Ite_1_0001.eps
Ite_2_0001.eps
Ite_3_0001.eps
Ite_4_0001.eps
However, I'm trying to rename all these files as follows:
Ite_0001.eps
Ite_0002.eps
Ite_0003.eps
Ite_0004.eps
So I'm proceeding in this way:
for path, subdirs, files in os.walk(newpath):
num = len(os.listdir(newpath))
for filename in files:
basename, extension = os.path.splitext(filename)
for x in range(1, num+1):
new_filename = '_%04d' % x + extension
os.rename(os.path.join(newpath, filename), os.path.join(newpath, new_filename))
It's not working at all because all the files are erased from the directory and when running the script once at a time I have this:
First run: _00004
Second run: _00005
.... and so on.
Could any one have some tips that could help me to achieve this task :).
Thank you very much for your help.

You could test the approach with a list of strings. So you do not run the risk of deleting the files. ;-)
files = ["Ite_1_0001.eps", "Ite_2_0001.eps", "Ite_3_0001.eps", "Ite_4_0001.eps",]
for f in files:
# Get the value between underscores. This is the index.
index = int(f[4:f.index('_', 4)])
new_name = '_%04d' % index
# Join the prefix, index and sufix file
print ''.join([f[:3], new_name, f[-4:]])
Ite_0001.eps
Ite_0002.eps
Ite_0003.eps
Ite_0004.eps

You can dynamically change the thing you're substituting in within your loop, like so
import os, re
n = 1
for i in os.listdir('.'):
os.rename(i, re.sub(r'\(\d{4}\)', '(Ite_) ({n})'.format(n=n), i))
n += 1

I write a function that if you give in input your basename it returns the correct name.
def newname(old_name):
num = old_name[4]
return (old_name[0:3] + old_name[5:-1] + num)

Count and print number of files in subfolders using Python

My folder structure is as follows
Folder A
Folder B1
Folder B2
....
Folder Bn
How can I count the number of files in each of the folders (Folder B1 - Folder Bn), check if the number of files is larger than a given limit and print the folder name and number of files in it on the screen?
Like this:
Folders with too many files:
Folder B3 101
Folder B7 256
Here's what I've tried so far. It goes through every subfolder in each of my Folder B1 etc. I just need file count in one level.
import os, sys ,csv
path = '/Folder A/'
outwriter = csv.writer(open("numFiles.csv", 'w')
dir_count = []
for root, dirs, files in os.walk(path):
for d in dirs:
a = str(d)
count = 0
for fi in files:
count += 1
y = (a, count)
dir_count.append(y)
for i in dir_count:
outwriter.writerow(i)
And then I just printed numFiles.csv. Not quite how I'd like to do it.
Thanks in advance!

As the are all contained in that single folder, you only need to search that directory:
import os
path = '/Folder A/'
mn = 20
folders = ([name for name in os.listdir(path)
if os.path.isdir(os.path.join(path, name)) and name.startswith("B")]) # get all directories
for folder in folders:
contents = os.listdir(os.path.join(path,folder)) # get list of contents
if len(contents) > mn: # if greater than the limit, print folder and number of contents
print(folder,len(contents)

os.walk(path) gives you three tuple for a directory, ie (directory,subdirectory,files).
directory -> list of all directory in current dir, list of subdirectory in current dir, list of files in current dir.
so you can code likes this:
import os
for dir,subdir,files in os.walk(path):
if len(files) > your_limit:
print dir + " has crossed limit, " + "total files: " + len(files)
for x in files:
print x
if you want to walk only one level, you need to code like this:
for x in os.listdir(path):
if os.path.isdir(x):
count = len([ y for y in os.listdir(x) if os.path.isfile(os.path.join(x,y)) ])
if count > your_limit:
print x + " has crossed limit: ", +count