Develop Reference

Skip the renamed Image and rename the new Image coming into the folder only

I copy the images from the source folder to destination folder using shutil.copy() Then I rename the folder using os.rename. However, there is a problem. So if the destination folder is empty, then I will get the image that I copy from source folder and os.rename will rename the images. But once the new images from source folder comes in, it will rename again the image that already rename. For example, after I rename the image, it will be 1_01_411.jpeg. But once it runs again, it will be 1_01_411_412.jpeg as it is in while loop.
I will attach the image that shows the issue.
While True:
src = "/home/pi/Pictures/JB_CAM/"
dest = "/home/pi/windowshare/"
par ="*"
i=1
d = []
for file in glob.glob(os.path.join(src,par)):
f = str(file).split('\\')[-1]
for n in glob.glob(os.path.join(dest,par)):
d.append(str(n).split('\\')[-1])
if f not in d:
print("copied",f," to ",dest)
shutil.copy(file,dest)
else:
f1 = str(f).split(".")
f1 = f1[0]+"_"+str(i)+"."+f1[1]
while f1 in d:
f1 = str(f).split(".")
f1 = f1[0]+"_"+str(i)+"."+f1[1]
print("{} already exists in {}".format(f1,dest))
i =i + 1
shutil.copy(file,os.path.join(dest,f1))
print("renamed and copied ",f1 ,"to",dest)
i = 1
os.getcwd()
collection = "/home/pi/windowshare"
dest = "/home/pi/windowshare/"
for i, filename in enumerate(os.listdir(collection)):
os.rename(dest + filename, dest + filename.split(".")[0] +"_"+ db_sn+ ".jpeg)
I also use OpenCV to test.
src ="/home/pi/Pictures/JB_CAM/"
for img in os.listdir(src):
name,ext = os.path.splitext(img)
if ext ==".jpeg":
f1 = cv2.imread('/home/pi/Pictures/JB_CAM/'+img)
print(f1)
#status =cv2.imwrite("/home/pi/windowshare/"+img.split(".")[0]+"_"+"%d.jpeg" % db_sn,f1)
cv2.imwrite("/home/pi/windowshare/"+img.split(".")[0]+"_"+"%d.jpeg" % db_sn,f1)
However, the problem with this approach is it overwrites the db_sn value to the all the images. The goal is to copy or write(save) the files from source to destination folder and rename according to db_sn(serial number).

how to get folder name that ends with a number in python

I have a folder that is structured like this
post
-----1
------10am
-----------images
-----2
-------10am
-----------images
-----3
------10am
-----------images
this folder is up to 31 with the same subfolder '10am' and inside that is another folder 'images'
in another folder, I have all the image and .txt files that that I need to copy based on folder name in python
So what I need to do now is copy
'2.jpg' inside 'post\2\10am\images' and
'2.txt' inside ''post\2\10am' and so on
here is my code so far:
import os,shutil
sampleFiles = r"\practice image and text"
destination = r"\posts"
time = '10am'
sample = os.listdir(sampleFiles)
# sample = ['10.jpg', '10.txt', '11.jpg', '11.txt', '13.png', '13.txt', '16.jpg', '16.txt', '17.jpg', '17.txt', '18.jpg', '18.txt', '2.jpg', '2.txt', '20.jpg', '20.txt', '23.jpg', '23.txt', '24.jpg', '24.txt', '25.jpg', '25.txt', '27.jpg', '27.txt', '3.jpg', '3.txt','4.jpg', '4.txt', '5.jpg', '5.txt', '6.jpg', '6.txt', '9.jpg', '9.txt']
for root, dirs, files in os.walk(destination):
for folderName in dirs:
#get root + foldername
rootWithFolder = os.path.join(root, folderName)
#get path to date
pathToDate = rootWithFolder.endswith(int(folderName)) # how to get the number?
# get path to image folders
if rootWithFolder.endswith('images'):
pathToImage = rootWithFolder
#copy .jpg files to pathToImage
shutil.copy(sampleFiles + '\\' + str(pathToDate) + '.jpg' , pathToImage) #not the most elegant way
#copy .txt files to pathToDate
shutil.copy(sampleFiles + '\\' + str(pathToDate) + '.txt' , pathToDate + '\\' + 'time') #not the most elegant way
in my code, I am stuck on how to get pathToDate so I can just copy it based on the name of the folder,
I tried using def like this:
def allfiles(list):
for i in range(len(list)):
return list[i] # returns only the first value of the list
# print(list[i]) #but this one returns all the value of the list
allfiles(sample)
but it only returns 1 instance of the list.
My question is, how can I get the folder that is named as a number and ignore the strings like 10am folder and images folder
or is there a better way to do this? Thank you

Here's a suggestion if you're still looking for a solution... I'd go the other way round:
from pathlib import Path
from shutil import copy
sample_folder = Path("practice image and text")
dest_folder = Path("posts")
for file in sample_folder.glob("*.*"):
number, suffix = file.name.split(".")
if suffix == "txt":
copy(file, dest_folder / number / "10am")
else:
copy(file, dest_folder / number / "10am" / "images")

Need to upload sub-dirs and their contents, not just files in current dir

A script was supplied to me in order to upload files to a cloud bucket. You input the dir where the files you want to upload are and bingo bango, done.
What needs to happen is that there are additional sub dirs with their own files in them that I would like to transfer as well based on the input of the root dir. They would need to retain their tree structure relative to the root dir input.
Using the current code I get a write error/access denied fail. I know this is because the for loop is using os.listdir which can't parse the extra sub dirs and files but I'm not sure how to modify.
I attempted to get all the information I needed using os.walk and parsing that out. I verified with some print tests that it was looking in the right place for everything. However I hit a wall when I got this error when running the script:
folder\folder\lib\ntpath.py", line 76, in join
path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not list
I understand that something is being generated as a list when it shouldn't be but I'm not sure how to go about this...
This is the original script provided to me below. I have added the variable at the top just to be a little less abstract.
local_directory_path = 'C:\folder\folder\sync\FROM_LOCAL_UPLOAD'
def upload_folder_to_cloud(self, mount_id, local_directory_path):
''' This method will list every file at the local_directory_path and then for each,
it will call the api method athera.sync.upload_file for every file in your local directory
'''
_, destination_folder = os.path.split(local_directory_path)
if not destination_folder:
self.logger.error("Make sure the provided 'local_directory_path' does not end with a '/' or a '\\'")
sys.exit(2)
destination_folder = destination_folder + "/"
self.logger.info("Folder = {}".format(destination_folder))
for filename in os.listdir(local_directory_path):
destination_path = destination_folder + filename
filepath = os.path.join(local_directory_path, filename)
with open(filepath, "rb") as f:
_, err = self.client.upload_file(self.group_id, mount_id, f, destination_path=destination_path,)
if err != None:
self.logger.error(err)
sys.exit(4)
return destination_folder
This is what I modified it to as a test:
for root, dirs, files in os.walk(local_directory_path):
srcFile = (os.path.join(files))
srcRoot = (os.path.join(root))
rootSplit = os.path.normpath(srcRoot).split(os.path.sep)
srcDirs = '/'.join(rootSplit[4:])
src = str('fixLocalFolder') + '/' + str(srcDirs) +'/'+ (files)
dst = str(srcDirs) + '/' + (files)
destination_folder = str(srcRoot) + "/"
destination_path = str(destination_folder) + str(srcFile)
filepath = os.path.join((str(srcDirs), str(srcFile)))
with open(filepath, "rb") as f:
_, err = self.client.upload_file(
self.group_id,
mount_id,
f,
destination_path=destination_path,
)
if err != None:
self.logger.error(err)
sys.exit(4)
return destination_folder
I do not code for a living so I am sure I am not going about this the right way. I apologize for any code atrocities in advance. Thank you!

I do see some issues in that code, even without testing it. Something like the following might work for that loop. (Note! Untested!).
for root, dirs, files in os.walk(local_directory_path):
# Iterate through files in the currently processed directory
for current_file in files:
# Full path to file
src_file = os.path.join(root, current_file)
# Get the sub-path relative the original root.
sub_path = os.path.relpath(root, start=destination_folder)
# Get the destination path
destination_path = os.path.join(sub_path, current_file)
with open(src_file, "rb") as f:
_, err = self.client.upload_file(
self.group_id,
mount_id,
f,
destination_path=destination_path,
)
if err != None:
self.logger.error(err)
sys.exit(4)
I believe your central problem was misunderstanding what os.walk gives you. It gives you listing of each directory (and subdirectory), one after another.
Thus the values of one iterations might look like (when listing /mydir):
# First iteration:
root = "/mydir"
dirs = ["subdir", ...]
files = ["something.doc", "something else.txt"]
# Second iteration:
root = "/mydir/subdir"
dirs = ["sub-sub-dir1", ...]
files = ["file1.txt", "file2.txt", ...]

Python Read multiple images from multiple folders

I want to read multiple .jpg images, that are in 3 separate folders. The 3 folders are on the same path. I tried to do it like this:
path1 = os.path.abspath('Type_1')
path2 = os.path.abspath('Type_2')
path3 = os.path.abspath('Type_3')
folder = os.path.join(path1, path2, path3)
def load_images_from_folder(folder):
images = []
for filename in os.listdir(folder):
if filename.endswith(".jpg"):
img = cv2.imread(os.path.join(folder, filename))
if img is not None:
images.append(img)
return images
print(load_images_from_folder(folder))
But it only returns the last path and not all of them. I also tried to use relative paths, such as:
path1 = os.path.relpath('Type_1')
path2 = os.path.relpath('Type_2')
path3 = os.path.relpath('Type_3')
folder = os.path.join(os.path.sep, path1, path2, path3)
but still the same problem. Can someone help with this?

Your return images line is inside your loop, so it will return as soon as it finds any matching result. You only want it to return after the whole loop as finished.
Reduce the indentation of the return statement so that it lies after the loop instead of inside it.
def load_images_from_folder(folder):
images = []
for filename in os.listdir(folder):
if filename.endswith(".jpg"):
img = cv2.imread(os.path.join(folder, filename))
if img is not None:
images.append(img)
return images
[Edit]
If you want to look in multiple adjacent folders, you want something like this:
root_folder = '[whatever]/data/train'
folders = [os.path.join(root_folder, x) for x in ('Type_1', 'Type_2', 'Type_3')]
all_images = [img for folder in folders for img in load_images_from_folder(folder)]
That will call load_images on each folder, and put all the results into one list.

If I understand the problem correctly, your file structure looks as follows:
- Type1
- Image1.jpg
- Image2.jpg
- Type2
- Image1.jpg
- Image2.jpg
- Type3
- Image1.jpg
- Image2.jpg
If this is true, then the os.path.join call is errant (it'll result in a string that reads "Type1/Type2/Type3" which achieves nothing for you).
I think the code you're looking for is as follows:
def load_images_from_folder(folder):
images = []
for filename in os.listdir(folder):
if any([filename.endswith(x) for x in ['.jpeg', '.jpg']]):
img = cv2.imread(os.path.join(folder, filename))
if img is not None:
images.append(img)
return images
folders = [
'Type1',
'Type2',
'Type3',
]
for folder in folders:
images = load_images_from_folder(folder)
# your code that does something with the return images goes here

I know this is really old, but this worked for me recently.
def create_dataset(img_folder):
img_data_array=[]
class_name=[]
for dirl in os.listdir(img_folder):
for file in os.listdir(os.path.join(img_folder,dirl)):
if any([file.endswith(x) for x in ['.jpeg', '.jpg']]):
image_path=os.path.join(img_folder,dirl,file)
image=cv2.imread(image_path,cv2.COLOR_BGR2RGB)
img_data_array.append(image)
class_name.append(dirl)
return img_data_array,class_name
img_data, class_name =create_dataset(train_folder)

How do I fix my error with PIL and numpy images

You have to run it in the folder with a couple images and run shuffle_all_images() and it will create new folder and randomly generate all of the values for each pixel. I think it has to do with not converting to numpy images as opposed to PIL images?, but I can't figure it out.
import random
import os.path
import PIL
import numpy
def image_shuffle(original_image):
for row in len(original_image):
for col in len(original_image[1]):
r,g,b = original_image[row][col]
r = random.randint(1,255)
g = random.randint(1,255)
b = random.randint(1,255)
original_image[row][col] = [r, g, b]
return original_image
def get_images(directory=None):
""" Returns PIL.Image objects for all the images in directory.
If directory is not specified, uses current directory.
Returns a 2-tuple containing
a list with a PIL.Image object for each image file in root_directory, and
a list with a string filename for each image file in root_directory
"""
if directory == None:
directory = os.getcwd() # Use working directory if unspecified
image_list = [] # Initialize aggregaotrs
file_list = []
directory_list = os.listdir(directory) # Get list of files
for entry in directory_list:
absolute_filename = os.path.join(directory, entry)
try:
image = PIL.Image.open(absolute_filename)
file_list += [entry]
image_list += [image]
except IOError:
pass # do nothing with errors tying to open non-images
return image_list, file_list
def shuffle_all_images(directory=None):
""" Saves a modfied version of each image in directory.
Uses current directory if no directory is specified.
Places images in subdirectory 'modified', creating it if it does not exist.
New image files are of type PNG and have transparent rounded corners.
"""
if directory == None:
directory = os.getcwd() # Use working directory if unspecified
# Create a new directory 'modified'
new_directory = os.path.join(directory, 'modified')
try:
os.mkdir(new_directory)
except OSError:
pass # if the directory already exists, proceed
#load all the images
image_list, file_list = get_images(directory)
#go through the images and save modified versions
for n in range(len(image_list)):
# Parse the filename
filename, filetype = file_list[n].split('.')
# Round the corners with radius = 30% of short side
new_image = image_shuffle(image_list[n])
#save the altered image, suing PNG to retain transparency
new_image_filename = os.path.join(new_directory, filename + '.png')
new_image.save(new_image_filename)

The image_shuffle function is wrong.
It should be:
for row in range(original_image.size[0]):
for col in range(original_image.size[1]):
r = random.randint(0,255)
g = random.randint(0,255)
b = random.randint(0,255)
original_image.putpixel((row, col), (r,g,b))
You want to have color values starting from 0 unless you don't want to get all the possible colors.