Develop Reference

Finding the latest file in each subdirectory

I have the a similar folder structure for which I need to pull the latest .jpg from each subdirectory:
+ C:\\myfiles
+ parentdir1
+ subdir1
+ somename1.jpg
+ somename2.jpg
+ ...
+ subdir2
+ somename3.jpg
+ somename4.jpg
+ ...
+ ...
+ parentdir2
+ subdir1
+ somename5.jpg
+ somename6.jpg
+ ...
+ subdir2
+ somename7.jpg
+ somename8.jpg
+ ...
+ ...
+ parentdir3
+ subdir1
+ somename9.jpg
+ somename10.jpg
+ ...
+ subdir2
+ somename11.jpg
+ somename12.jpg
+ ...
+ ...
+ ...
I don't know any of the names of the folders or files but I need to access the last 2 .jpg files in each subdir.
For the sake of making this simple, let's just assume I need to print the last 2 files created in the subdir.
I wrote a script that will search all subdir's in a given parentdir, but I actually need to go iterate through all parentdir's as well
import os
path = 'C:\\myfiles'
filelist = []
for i in range(len(os.listdir(path))):
subpath = path + '\\' + os.listdir(path)[i]
for root, dirs, files in os.walk(subpath):
for file in os.listdir(subpath):
filelist.append(os.path.join(root, file))
sorted_filelist = sorted(filelist, key=os.path.getctime)
print('the latest jpg file in ' + root + ' is: ' + sorted_filelist[-1])
print('the 2nd last jpg file in ' + root + ' is: ' + sorted_filelist[-2])
filelist.clear()

I think this will do what you want. Note that I sort the files by their last modification times, rather than their creation time because I think that's the way to determine which are "most recent".
import glob
import os
N_MOST_RECENT = 2
path = 'C:\\myfiles'
for entry in os.listdir(path):
subpath = os.path.join(path, entry)
if os.path.isdir(subpath):
for subentry in os.listdir(subpath):
subentrypath = os.path.abspath(os.path.join(subpath, subentry))
if os.path.isdir(subentrypath):
jpg_files = glob.iglob(os.path.join(subentrypath, '*.jpg'))
sorted_filenames = sorted(jpg_files, key=os.path.getmtime)
# Create list of filenames of the N most recent files.
most_recent = [os.path.split(name)[-1] # Extract filename from path.
for name in sorted_filenames[-N_MOST_RECENT:]]
print(f'{N_MOST_RECENT} most recent .jpg files in "{subentrypath}":\n'
f' {most_recent}')

Try iterating through the parent directory, and then through all the sub-directories, using os.listdir().
import os
parent_dir = 'path/to/parent/dir'
for subdir in os.listdir(parent_dir):
if not os.path.isdir(subdir):
continue
sorted_filelist = sorted(
[os.path.join(parent_dir, subdir, f) for f in os.listdir(subdir)
if os.path.splitext(f)[1] == '.jpg'],
key=os.path.getctime, reverse=True)
print(sorted_filelist[:2])

os.rename deleting files python 3

Python novice, my simple script gets a given directory and renames all files sequentially, however it is deleting the files but the print is showing the files names getting renamed, not sure where its going wrong here.
Also, in what order does it retrieve these files?
import os
path = os.path.abspath("D:\Desktop\gp")
i = 0
for file_name in os.listdir(path):
try:
print (file_name + " - " + str(i))
os.rename(os.path.join(path,file_name), str(i))
except WindowsError:
os.remove(str(i))
os.rename(os.path.join(path,file_name), str(i))
i += 1
print(str(i) + " files.")
Edit
Below is the solution with working code, retrieves all files in a dir by creation date and assigns them a iterated number while retaining file extension.
import os
def sorted_dir(folder):
def getctime(name):
path = os.path.join(folder, name)
return os.path.getctime(path)
return sorted(os.listdir(path), key=getctime)
path = os.path.abspath("D:\Path\Here")
i = 0
for file_name in sorted_dir(path):
_, ext = os.path.splitext(file_name)
print (file_name + " - " + str(i)+ext)
os.rename(os.path.join(path,file_name), os.path.join(path, str(i) + ext))
i += 1
print(str(i-1) + " files.")

The problem is that you're using an absolute path for the source, but a relative path for the destination. So the files aren't getting deleted, they're just getting moved into the current working directory.
To fix it so they get renamed into the same directory they were already in, you can do the same thing on the destination you do on the source:
os.rename(os.path.join(path,file_name), os.path.join(path, str(i)))
From a comment, it sounds like you may want to preserve the extensions on these files. To do that:
_, ext = os.path.splitext(file_name)
os.rename(os.path.join(path,file_name), os.path.join(path, str(i) + ext))

How to check if folder has suboflders and then list the directories with listdir()?

I want to write a file searching code where I don't know if the directory I'm searching in has subdirectories and I want to check that so I don't get an error like this:
[Error 267]The directory name is invalid: 'C:/Path/To/Directory'.
I wrote a code like this where if it finds the file it breaks and stopps the program but if not it goes down a layer and so on.
filename = raw_input('> ')
path = 'C:/Path/Of/Directory/You/Want/To/Search/In'
fldr = os.listdir(path)
for f in fldr:
p = path + '/' + f
sfldr = os.listdir(p)
if os.path.exists(p + '/' + filename):
print 'Found file!!', p + '/' + filename
break
else:
for sf in sfldr:
pp = p + '/' + sf
ssfldr = os.listdir(pp)
if os.path.exists(pp + '/' + filename):
print 'Found file!!', pp + '/' + filename
break
else:
for ssf in ssfldr:
ppp = pp + '/' + ssf
sssfldr = os.listdir(ppp)
if os.path.exists(ppp + '/' + filename):
print 'Found file!!', ppp + '/' + filename
break
The easy to notice error is that when the directory doesn't have 3 layers of subfolders the program just breaks and gives an error message.So if I could check if the folder has subfolders before entering it,that would be neat.

Use os.scandir(). Provides better speed over os.walk()
Link to docs here!
Alternatively use, glob
>>> from glob import glob
>>> paths = glob('*/')
>>> paths
['bin/', 'content/', 'include/', 'lib/', 'output/']
>>>

How to move files in multiple directories into another directory with same sub directoies

I have
import shutil
for sub_dir in os.listdir(path + "train"):
src_files = os.listdir(path+"train/" + sub_dir)
...
for file in src_files[0:split_index]:
original = path+"train/" + sub_dir+ "/" + file
distutils.dir_util.mkpath(path + "valid/" + sub_dir)
destination = path + "valid/" + sub_dir+"/"
shutil.move(original, destination)
I have two these directory structures:
train/abc
train/def
train/ghi
valid/
I need to move some portion of files in train/ to valid/ and retain the original directory structure. After the move, valid/ should look the same as train/:
valid/abc
valid/def
valid/ghi
The above code isn't quite right. How can I modify it to make it right?

distutils.dir_util.mkpath(path + "valid/" + sub_dir)
Added this part and it works now.

python can't find file it just made

My program cannot find the path that it just made, the program is for sorting files in the download folder.
If it finds a new type of file it should make a folder for that file type.
import os
FileList = os.listdir("/sdcard/Download/")
for File in FileList:
#print File
extension = ''.join(os.path.splitext(File)[1])
ext = extension.strip('.')
if os.path.exists("/mnt/external_sd/Download/" + ext):
Data = open("/sdcard/Download/" + File, "r").read()
file("/mnt/external_sd/" + ext + "/" + File, "w").write(Data)
elif os.path.exists("/mnt/external_sd/Download/" + ext) != True:
os.makedirs("/mnt/external_sd/Download/" + ext)
Data = open("/sdcard/Download/" + File, "r").read()
file("/mnt/external_sd/" + ext + "/" + File, "w").write(Data)

You create the directory
"/mnt/external_sd/Download/" + ext
but then you are trying to write to
"/mnt/external_sd/" + ext + "/" + File
You dropped the Download folder in that path. Change the last line to:
file("/mnt/external_sd/Download/" + ext + "/" + File, "w").write(Data)
Incidentally, it would be a bit shorter and clearer to write your last seven lines by taking the shared lines out of the if else statement and using shutil.copy instead of reading in the whole file then writing it out again:
import shutil
if not os.path.exists("/mnt/external_sd/Download/" + ext):
os.makedirs("/mnt/external_sd/Download/" + ext)
shutil.copy("/sdcard/Download/" + File, "/mnt/external_sd/Download/" + ext + "/" + File)
(Using shutil will also generally be faster and use less memory, especially if your files are large).