Develop Reference

Skip the renamed Image and rename the new Image coming into the folder only

I copy the images from the source folder to destination folder using shutil.copy() Then I rename the folder using os.rename. However, there is a problem. So if the destination folder is empty, then I will get the image that I copy from source folder and os.rename will rename the images. But once the new images from source folder comes in, it will rename again the image that already rename. For example, after I rename the image, it will be 1_01_411.jpeg. But once it runs again, it will be 1_01_411_412.jpeg as it is in while loop.
I will attach the image that shows the issue.
While True:
src = "/home/pi/Pictures/JB_CAM/"
dest = "/home/pi/windowshare/"
par ="*"
i=1
d = []
for file in glob.glob(os.path.join(src,par)):
f = str(file).split('\\')[-1]
for n in glob.glob(os.path.join(dest,par)):
d.append(str(n).split('\\')[-1])
if f not in d:
print("copied",f," to ",dest)
shutil.copy(file,dest)
else:
f1 = str(f).split(".")
f1 = f1[0]+"_"+str(i)+"."+f1[1]
while f1 in d:
f1 = str(f).split(".")
f1 = f1[0]+"_"+str(i)+"."+f1[1]
print("{} already exists in {}".format(f1,dest))
i =i + 1
shutil.copy(file,os.path.join(dest,f1))
print("renamed and copied ",f1 ,"to",dest)
i = 1
os.getcwd()
collection = "/home/pi/windowshare"
dest = "/home/pi/windowshare/"
for i, filename in enumerate(os.listdir(collection)):
os.rename(dest + filename, dest + filename.split(".")[0] +"_"+ db_sn+ ".jpeg)
I also use OpenCV to test.
src ="/home/pi/Pictures/JB_CAM/"
for img in os.listdir(src):
name,ext = os.path.splitext(img)
if ext ==".jpeg":
f1 = cv2.imread('/home/pi/Pictures/JB_CAM/'+img)
print(f1)
#status =cv2.imwrite("/home/pi/windowshare/"+img.split(".")[0]+"_"+"%d.jpeg" % db_sn,f1)
cv2.imwrite("/home/pi/windowshare/"+img.split(".")[0]+"_"+"%d.jpeg" % db_sn,f1)
However, the problem with this approach is it overwrites the db_sn value to the all the images. The goal is to copy or write(save) the files from source to destination folder and rename according to db_sn(serial number).

python file name change

I am trying to change file names like below:
the 000000 are the same number.
000000_ABC.png --->000000+1_ABC.png
000000_DEF.png --->000000+2_DEF.png
000000_GHI.png --->000000+3_GHI.png
000000_JKL.png --->000000+4_JKL.png
In order to do so, I wrote code like below.
img_files = os.listdir(PATH_TO_PNG_FILES)
for img_file, i in zip(img_files, range(len(img_files))):
new_img_file = img_file.replace("_", "+"+str(i)+"_")
os.rename(path + img_file, path + new_img_file)
There are more than just four files and more of similar lines.
The problem is that immediately after running pycharm, it successfully produces the desired results, but after I run another page related to the result directories, the results continue to be changed like below even after the process finished. I do not understand why.
000000+1+1_ABC.png
000000+2+2_DEF.png
000000+3+3_GHI.png
000000+4+4_JKL.png
or
otherwise "+unexpected number"

This is because the other directory may already contain file in the format of "000000+1_ABC.png" and your script is changing _ to "+1_" resulting in "000000+1+1_ABC.png". To solve this you can add a if statement to check it should not contain "+" symbol.
img_files = os.listdir(path inside of which the png files are saved)
for img_file, i in zip(img_files, range(len(img_files))):
if not ("+" in img_file):
new_img_file = img_file.replace("_", "+"+str(i)+"_")
os.rename(path + img_file, path + new_img_file)

A simple and naive way would be to add a verification to check whether there is a '+' in the filename. If you have other files which may contain a +, you may have to check for a stricter pattern.

I made a YouTube video https://youtu.be/K9jhAPZLZLc on how to rename multiple files like the one you have assuming all your files are in the same directory.
To answer your question. assuming all image files are in the same folder.
path = 'C:\\Users\\USER\\Desktop\\rename_images\\images\\' # path to your images
files = os.listdir(path)
for count, filename in enumerate(files):
# Get the file extension
file, file_extension = os.path.splitext(filename)
# check if the current file is a folder or not
full_path = f'{path}{filename}'
if os.path.isdir(full_path):
print('This is a directory')
elif os.path.isfile(full_path):
print('This is a normal file')
# Rename
if not '+' in file:
try:
file_split = file.split('_')
zeros = file_split[0]
alpha = file_split[-1]
current_file_name = os.path.join(path, filename)
new_file_name = os.path.join(path, ''.join([f'{zeros}+{count}_{alpha}', file_extension]))
os.rename(current_file_name, new_file_name)
except:
pass
else:
pass
else:
print('This is a special file')

I would imagine that the problem comes from modifying the name insted of overwriting.
import os
DIR_PATH = 'files'
def rename_files(dir_name):
img_files = os.listdir(dir_name)
for i in range(len(img_files)):
file_name = img_files[i].split('_')[-1]
file_name = '000000+{0}_{1}'.format(i, file_name)
os.rename(
os.path.join(dir_name, img_files[i]),
os.path.join(dir_name, file_name)
)
if __name__ == '__main__':
rename_files(DIR_PATH)

Sorting corresponding files into the same directory

I've got 6 directories (A, B, C, D, E, F) containing .mov files.
The structure is:
A
-0001_01.mov
-0002_01.mov
-...
B
-0001_02.mov
-0002_02.mov
-...
And so on.
First, I want to create as many directories as there are files in one of the directories mentioned above.
Let's say A contains 35 .mov files (B, C .. contain the same amount of .mov files).
I now got 35 folders starting from "01" up to "35".
Now I want to copy each corresponding .mov file into the same directory, which means 0001_01.mov - 0001_06.mov go into "01", 0002_01.mov - 0002_06.mov go into "02" and so on.
I've got the creation of the directories working, but I just can't wrap my head around the copying part.
import os
pathA = ("./A/")
pathB = ("./B/")
pathC = ("./C/")
pathD = ("./D/")
pathE = ("./E/")
pathF = ("./F/")
path, dirs, filesA = next(os.walk(pathA))
file_countA = len(filesA)
path, dirs, filesB = next(os.walk(pathB))
file_countB = len(filesB)
path, dirs, filesC = next(os.walk(pathC))
file_countC = len(filesC)
path, dirs, filesD = next(os.walk(pathD))
file_countD = len(filesD)
path, dirs, filesE = next(os.walk(pathE))
file_countE = len(filesE)
path, dirs, filesF = next(os.walk(pathF))
file_countF = len(filesF)
path2 = ("./")
if file_countA == file_countB == file_countC == file_countD == file_countE == file_countF:
print("true")
else:
print ("false")
for i in range(file_countA):
try:
if i < 9:
os.mkdir(path2 + "0" + str(i + 1))
path3 = ("./" + "0" + str(i + 1))
print (path3)
elif i >= 9:
os.mkdir(path2 + str(i + 1))
path3 = ("./" + str(i + 1))
print (path3)
except OSError:
print ("Creation of the directory %s failed" % path2)
else:
print ("Successfully created the directory %s " % path2)
This is my first time using python, I think the code reflects that.
I've now wasted countless hours on this, so any help is appreciated.

So I changed your code quite a bit and tested it quickly on my system and it seemed to do what you wanted. Can you try and let me know if this gave you idea of how it can be done?
Disclaimer: I'm not Python expert by any means but I find my way around it and this is most likely not the prettiest solution but it deos work on my machine exactly as you wanted it. Just make sure you run it from inside your folder and if you are not running it from outside your folder then change cwd = os.getcwd() to cwd = "path-to-your-folder"
import os
import shutil
import glob
paths = ["/A/","/B/","/C/","/D/","/E/","/F/"]
cwd = os.getcwd()
num_folders = 0
for path in paths:
num_files = len([f for f in os.listdir(cwd+path)if os.path.isfile(os.path.join(cwd+path, f))])
if num_files>num_folders:
num_folders = num_files
for i in range(num_folders):
try:
if i < 9:
fname = cwd + "/0" + str(i + 1)
os.mkdir(fname)
for path in paths:
source = cwd + "/" + path
filename = "000{}_*.mov".format(i+1)
for file in glob.glob(os.path.join(source,filename)):
shutil.copy2(file,fname)
elif i >= 9:
fname = cwd + "/" + str(i + 1)
os.mkdir(fname)
for path in paths:
source = cwd + "/" + path
filename = "00{}_*.mov".format(i+1)
for file in glob.glob(os.path.join(source,filename)):
shutil.copy2(file,fname)
except OSError:
pass

I'm no python expert either (look at my scores too, hi), but I've tried to keep your original coding order as much as possible. I would recommend to look at different codes for real expert-tier code but it seems to do what you're asking for :
import os
import shutil
mov_pathes = ["./a/", "./b/"]
all_files = []
lengths = []
for mov_path in mov_pathes :
# listdir gives you all files in the direcetory
files_in_dir = os.listdir(mov_path)
# we'll save those in a list along with where it's from ,
# ex : ('./patha/',['0001_01.mov','0002_01.mov'])
all_files.append((mov_path, files_in_dir))
# also length info for "all items are equal length" comparison in the future
lengths.append(len(files_in_dir))
if lengths.count(lengths[0]) == len(lengths) :
print ("true")
else :
print ("false")
base_dir = "./"
for i in range (1,lengths[0]+1) :
try :
# zfill(n) fills rest of your string to 0, (ex. "7".zfill(5) gives you 00007), probably helpful for future
path_name = base_dir + str(i).zfill(2)
os.mkdir(path_name)
except OSError :
print ("Creation of the directory {path_name} failed".format(path_name = path_name))
else :
print ("Successfully created the directory {path_name}".format(path_name = path_name))
Does exactly the same thing but it would probably make maintaining your code easier laster on.
for your real question, IF we're sure that your inputs are gonna look like 00XX_NN.mov, adding
for files in all_files :
# Remember we saved as (original dir, list of files in the dir?)
# This is a original dir
source_dir = files[0]
# This is list of files in that directory
source_files = files[1]
for file in source_files :
# so original file is located in source_dir + file
source_file = source_dir + file
# and your target directory is 00XX, so getting file[2:4] gives the target directory
target_dir = base_dir + file[2:4]
#shutil.copy (source file, target directory) copies your files.
shutil.copy (source_file , target_dir)
seems to do what you're asking for, at least for me. Once again I'm no expert so let me know if it's not working!
tested with :
./a
- 0001_01
- 0002_01
- 0003_01
./b
- 0001_02
- 0002_02
- 0003_02
result :
./01 :
- 0001_01
- 0001_02
./02 :
- 0002_01
- 0002_02
./03 :
- 0003_01
- 0003_02

Need to upload sub-dirs and their contents, not just files in current dir

A script was supplied to me in order to upload files to a cloud bucket. You input the dir where the files you want to upload are and bingo bango, done.
What needs to happen is that there are additional sub dirs with their own files in them that I would like to transfer as well based on the input of the root dir. They would need to retain their tree structure relative to the root dir input.
Using the current code I get a write error/access denied fail. I know this is because the for loop is using os.listdir which can't parse the extra sub dirs and files but I'm not sure how to modify.
I attempted to get all the information I needed using os.walk and parsing that out. I verified with some print tests that it was looking in the right place for everything. However I hit a wall when I got this error when running the script:
folder\folder\lib\ntpath.py", line 76, in join
path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not list
I understand that something is being generated as a list when it shouldn't be but I'm not sure how to go about this...
This is the original script provided to me below. I have added the variable at the top just to be a little less abstract.
local_directory_path = 'C:\folder\folder\sync\FROM_LOCAL_UPLOAD'
def upload_folder_to_cloud(self, mount_id, local_directory_path):
''' This method will list every file at the local_directory_path and then for each,
it will call the api method athera.sync.upload_file for every file in your local directory
'''
_, destination_folder = os.path.split(local_directory_path)
if not destination_folder:
self.logger.error("Make sure the provided 'local_directory_path' does not end with a '/' or a '\\'")
sys.exit(2)
destination_folder = destination_folder + "/"
self.logger.info("Folder = {}".format(destination_folder))
for filename in os.listdir(local_directory_path):
destination_path = destination_folder + filename
filepath = os.path.join(local_directory_path, filename)
with open(filepath, "rb") as f:
_, err = self.client.upload_file(self.group_id, mount_id, f, destination_path=destination_path,)
if err != None:
self.logger.error(err)
sys.exit(4)
return destination_folder
This is what I modified it to as a test:
for root, dirs, files in os.walk(local_directory_path):
srcFile = (os.path.join(files))
srcRoot = (os.path.join(root))
rootSplit = os.path.normpath(srcRoot).split(os.path.sep)
srcDirs = '/'.join(rootSplit[4:])
src = str('fixLocalFolder') + '/' + str(srcDirs) +'/'+ (files)
dst = str(srcDirs) + '/' + (files)
destination_folder = str(srcRoot) + "/"
destination_path = str(destination_folder) + str(srcFile)
filepath = os.path.join((str(srcDirs), str(srcFile)))
with open(filepath, "rb") as f:
_, err = self.client.upload_file(
self.group_id,
mount_id,
f,
destination_path=destination_path,
)
if err != None:
self.logger.error(err)
sys.exit(4)
return destination_folder
I do not code for a living so I am sure I am not going about this the right way. I apologize for any code atrocities in advance. Thank you!

I do see some issues in that code, even without testing it. Something like the following might work for that loop. (Note! Untested!).
for root, dirs, files in os.walk(local_directory_path):
# Iterate through files in the currently processed directory
for current_file in files:
# Full path to file
src_file = os.path.join(root, current_file)
# Get the sub-path relative the original root.
sub_path = os.path.relpath(root, start=destination_folder)
# Get the destination path
destination_path = os.path.join(sub_path, current_file)
with open(src_file, "rb") as f:
_, err = self.client.upload_file(
self.group_id,
mount_id,
f,
destination_path=destination_path,
)
if err != None:
self.logger.error(err)
sys.exit(4)
I believe your central problem was misunderstanding what os.walk gives you. It gives you listing of each directory (and subdirectory), one after another.
Thus the values of one iterations might look like (when listing /mydir):
# First iteration:
root = "/mydir"
dirs = ["subdir", ...]
files = ["something.doc", "something else.txt"]
# Second iteration:
root = "/mydir/subdir"
dirs = ["sub-sub-dir1", ...]
files = ["file1.txt", "file2.txt", ...]

Getting paths of each file of a directory into an Array in python

Im trying to put into an array files[] the paths of each file from the Data folder but when I try to go into subfolders I want it to be able to go down to the end of the Data file, for example I can read files in a subfolder of the main folder Data which im trying to get a list of all the paths of each file into an array but it doesn't go deeper it does not access the subfolder of the subfolder of Data without writing a loop. Want I want is a loop which has infinit depth of view of files in the Data folder so I can get all the file paths.
For example this is what I get:
['Data/DataReader.py', 'Data/DataReader - Copy.py', 'Data/Dat/DataReader.py', 'Data/fge/er.txt']
This is what I want but it can still go into deeper folders:
['Data/DataReader.py', 'Data/DataReader - Copy.py', 'Data/Dat/DataReader.py', 'Data/fge/er.txt', 'Data/fge/Folder/dummy.png', 'Data/fge/Folder/AnotherFolder/data.dat']
This is my current path, what would i need to add or change?
import os
from os import walk
files = []
folders = []
for (dirname, dirpath, filename) in walk('Data'):
folders.extend(dirpath)
files.extend(filename)
break
filecount = 0
for i in files:
i = 'Data/' + i
files[filecount] = i
filecount += 1
foldercount = 0
for i in folders:
i = 'Data/' + i
folders[foldercount] = i
foldercount += 1
subfolders = []
subf_files = []
for i in folders:
for (dirname, dirpath, filename) in walk(i):
subfolders.extend(dirpath)
subf_files.extend(filename)
break
subf_files_count = 0
for a in subf_files:
a = i + '/'+a
files = files
files.append(a)
print files
subf_files = []
print files
print folders
Thanks a lot!

Don't understand what are your trying to do, especially why you break your walk after the first element:
import os
files = []
folders = []
for (path, dirnames, filenames) in os.walk('Data'):
folders.extend(os.path.join(path, name) for name in dirnames)
files.extend(os.path.join(path, name) for name in filenames)
print files
print folders