How can I make directory & put image in the directory?
I wrote the following code in data.py:
import os
import cv2
import argparse
import numpy as np
import math
parser = argparse.ArgumentParser(description='')
parser.add_argument('input_dir' ,help='input directory')
parser.add_argument('output_dir' ,help='out directory')
args = parser.parse_args()
def find_all_files(directory):
for root, dirs, files in os.walk(directory):
for file in files:
p=os.path.join(root, file)
p=p.split("/")[len(p.split("/"))-2]
name, ext = os.path.splitext(p)
yield os.path.join(root, file)
dirname=args.output_dir
if not os.path.exists(dirname):
os.mkdir(dirname)
folder_img = find_all_files(args.input_dir)
im =cv2.imread(folder_img)
cv2.imwrite(args.output_dir ,im)
Now when I run command python data.py ./photo ./copy_photo, find a writer for the specified extension in function imwrite error happens. I want to make a directory copy_photo and put images made by this code in the directory. The copy_photo folder is made, so what is wrong in my code? How should I fix this? Am I wrong to write the way of putting images in the copy_photo folder?
find_all_files is a generator. The variable folder_img will be bound to that generator. Calling cv2.imread() on that generator is not going to work because imread() expects an image file, not a generator.
You need to iterate over the generator to copy each file one by one. Something like this:
folder_img = find_all_files(args.input_dir)
for filename in find_all_files(args.input_dir):
im = cv2.imread(filename)
cv2.imwrite(os.path.join(args.output_dir, os.path.basename(filename)), im)
Note also that args.output_dir is a string that represents the desination directory name. You need to specify the path of the file including the directory. Use os.path.join() and os.path.basename() to do that.
If all you are wanting to do is to copy the files from one directory to another then you should perhaps use shutil.copytree as there is no point in opening the image files and then writing them back out. Also, your find_all_files() generator function will pass back all files, including directories, so you need to add some logic to detect image files and filter out those that are unwanted.
cv2.imread() accepts a single file name, but you are calling it with a (generator returning a) list of files.
Try this:
def find_all_files(directory):
for root, dirs, files in os.walk(directory):
for file in files:
# Comment out dead code
'''
p=os.path.join(root, file)
p=p.split("/")[len(p.split("/"))-2]
name, ext = os.path.splitext(p)
'''
# Changed this to return a tuple
yield root, file
for dirname, imagefile in find_all_files(args.input_dir):
im =cv2.imread(os.path.join(dirname, imagefile))
cv2.imwrite(os.path.join(args.output_dir, imagefile), im)
Because imwrite() needs to receive a filename parameter, I changed find_all_files to return a tuple, so you can use the same filename for the output, and choose which directory name you add in front.
Rewriting the images seems like a very inefficient way to do this, though. Why don't you simply copy the files?
Related
During one of my projects, I faced this challenge: There is a folder named Project, and inside that, there are multiple images (say 100 images), and each has been named sequentially like the first image name is imag_0, 2nd image name is img_2,....imag_99.
Now, based on some conditions, I need to separate out some images say img_5, img_10, img_30, img_88, img_61. My question will be, is there any way to filter out these images and make a folder inside the folder Project named "the odd ones" and store those specified images?
One extra help will be in my case. Suppose I have hundreds of such Projects folders in a sequential way Projects_1, Projects_2, Projects_3,....., Projects_99, and each contains hundreds of pictures. Can it be possible to separate all the specified photos and store them inside a separate folder inside each Projects_n folder, assuming the photos we have to separate out and store differently will be the same for each Projects_n folder?
Please help me with this. Thank you!
For the first problem you can lookup to the below pseudo-code (you have to specify the target function). Instead, for the second problem you should provide more details;
from glob import glob
import itertools
import shutil
import os
# Creating a funtion to check if filename
# is a target file which has to be moved:
def is_target(filename):
if ... return True
else return False
dirname = "some/path/to/project"
# Creating a list of all files in dir which
# could be moved based on type extension:
types = ('*.png', '*.jpeg')
filepaths = list(itertools.chain(*[glob(os.path.join(dirname, f"*.{t}")) for t in types]))
# Finding the files to move:
filepaths_to_move = []
for filepath in filepaths:
if is_target(os.path.basename(filepath)):
filepaths_to_move.append(filepath)
# Creating the new subfolder:
new_folder_name = "odd_images"
new_dir = os.path.join(dirname, new_folder_name)
if not os.path.exists(new_dir): os.makedirs(new_dir)
# Moving files into subfolder:
for filepath in filepaths_to_move:
basename = os.path.basename(filepath)
shutil.move(source, os.path.join(filepath, os.path.join(dirname, basename)))
Here is the logic.make necessary improvements for your use case
project_dir = "project_dir"
move_to_dir = os.path.join(project_dir,"move_to_dir")
files = [os.path.join(project_dir,file) for file in os.listdir(project_dir)]
filenames_to_filter = "test1.txt,test2.txt"
if not os.path.exists(move_to_dir):
os.makedirs(move_to_dir)
for(file in files):
if os.path.basename(file) in filenames_to_filter:
shutil.move(file,move_to_dir)
`
So my program search_file.py is trying to look for .log files in the directory it is currently placed in. I used the following code to do so:
import os
# This is to get the directory that the program is currently running in
dir_path = os.path.dirname(os.path.realpath(__file__))
# for loop is meant to scan through the current directory the program is in
for root, dirs, files in os.walk(dir_path):
for file in files:
# Check if file ends with .log, if so print file name
if file.endswith('.log')
print(file)
My current directory is as follows:
search_file.py
sample_1.log
sample_2.log
extra_file (this is a folder)
And within the extra_file folder we have:
extra_sample_1.log
extra_sample_2.log
Now, when the program runs and prints the files out it also takes into account the .log files in the extra_file folder. But I do not want this. I only want it to print out sample_1.log and sample_2.log. How would I approach this?
Try this:
import os
files = os.listdir()
for file in files:
if file.endswith('.log'):
print(file)
The problem in your code is os.walk traverses the whole directory tree and not just your current directory. os.listdir returns a list of all filenames in a directory with the default being your current directory which is what you are looking for.
os.walk documentation
os.listdir documentation
By default, os.walk does a root-first traversal of the tree, so you know the first emitted data is the good stuff. So, just ask for the first one. And since you don't really care about root or dirs, use _ as the "don't care" variable name
# get root files list.
_, _, files = next(os.walk(dir_path))
for file in files:
# Check if file ends with .log, if so print file name
if file.endswith('.log')
print(file)
Its also common to use glob:
from glob import glob
dir_path = os.path.dirname(os.path.realpath(__file__))
for file in glob(os.path.join(dir_path, "*.log")):
print(file)
This runs the risk that there is a directory that ends in ".log", so you could also add a testing using os.path.isfile(file).
I am trying to write a zip file using Python's zipfile module that starts at a certain subfolder but still maintains the tree structure from that subfolder. For example, if I pass "C:\Users\User1\OneDrive\Documents", the zip file will contain everything from Documents onward, with all of Documents' subfolders maintained within Documents. I have the following code:
import zipfile
import os
import datetime
def backup(src, dest):
"""Backup files from src to dest."""
base = os.path.basename(src)
now = datetime.datetime.now()
newFile = f'{base}_{now.month}-{now.day}-{now.year}.zip'
# Set the current working directory.
os.chdir(dest)
if os.path.exists(newFile):
os.unlink(newFile)
newFile = f'{base}_{now.month}-{now.day}-{now.year}_OVERWRITE.zip'
# Write the zipfile and walk the source directory tree.
with zipfile.ZipFile(newFile, 'w') as zip:
for folder, _ , files in os.walk(src):
print(f'Working in folder {os.path.basename(folder)}')
for file in files:
zip.write(os.path.join(folder, file),
arcname=os.path.join(
folder[len(os.path.dirname(folder)) + 1:], file),
compress_type=zipfile.ZIP_DEFLATED)
print(f'\n---------- Backup of {base} to {dest} successful! ----------\n')
I know I have to use the arcname parameter for zipfile.write(), but I can't figure out how to get it to maintain the tree structure of the original directory. The code as it is now writes every subfolder to the first level of the zip file, if that makes sense. I've read several posts suggesting I use os.path.relname() to chop off the root, but I can't seem to figure out how to do it properly. I am also aware that this post looks similar to others on Stack Overflow. I have read those other posts and cannot figure out how to solve this problem.
The arcname parameter will set the exact path within the zip file for the file you are adding. You issue is when you are building the path for arcname you are using the wrong value to get the length of the prefix to remove. Specifically:
arcname=os.path.join(folder[len(os.path.dirname(folder)) + 1:], file)
Should be changed to:
arcname=os.path.join(folder[len(src):], file)
So my friend and I have been having a problem with the first practice project of the above chapter of Automate the Boring Stuff with Python. The prompt goes: "Write a program that walks through a folder tree and searches for files with a certain file extension (such as .pdf or .jpg). Copy these files from whatever location they are in to a new folder."
To simplify, we are trying to write a program that copies all of the .jpg files out of My Pictures to another directory. Here's our code:
#! python3
# moveFileType looks in My Puctures and copies .jpg files to my Python folder
import os, shutil
def moveFileType(folder):
for folderName, subfolders, filenames in os.walk(folder):
for subfolder in subfolders:
for filename in filenames:
if filename.endswith('.jpg'):
shutil.copy(folder + filename, '<destination>')
moveFileType('<source>')
We keep getting an error along the lines of "FileNotFoundError: [Errno 2] No such file or directory".
Edit: I added a "\" to the end of my source path (I'm not sure if that is what you meant, #Jacob H), and was able to copy all of the .jpg files in that directory, but received an error when it tried to copy a file within a subfolder of that directory. I added a for loop for subfolder in subfolders and I no longer get any errors, but it doesn't actually look in the subfolders for .jpg files.
There is a more fundamental problem with your code. When you use os.walk() it will already loop through every directory for you, so looping manually through the subfolders is going to produce the same results multiple times.
The other, and more immediate, problem is that os.walk() produces relative file names, so you need to glue them back together. Basically you are omitting the directory name and looking in the current directory for files which os.walk() is finding down in a subdirectory somewhere.
Here's a quick attempt at fixing your code:
def moveFileType(folder):
for folderName, subfolders, filenames in os.walk(folder):
for filename in filenames:
if filename.endswith('.jpg'):
shutil.copy(os.path.join(folderName, filename), '<destination>')
Making the function accept a destination parameter as a second argument, instead of hardcoding <destination>, would make it a lot more useful for the future.
Make sure to type the source file destination address correctly. While i tested your code, i wrote
moveFileType('/home/anum/Pictures')
and i got error;
IOError: [Errno 2] No such file or directory:
and when i wrote
moveFileType('/home/anum/Pictures/')
the code worked perfectly...
Try doing that, hope that will do your work. M using Python 2.7
Herez the re defined code for walking into subfolders and copying ,jpg files from there aswell.
import os, shutil
def moveFileType(folder):
for root, dirs, files in os.walk(folder):
for file in files:
if file.endswith('.jpg'):
image_path=os.path.join(root,file) # get the path location of each jpeg image.
print 'location: ',image_path
shutil.copy(image_path, '/home/anum/Documents/Stackoverflow questions')
moveFileType('/home/anum/Pictures/')
I have been having an issue with using the zipfile.Zipfile() function. It zips my files properly, but then has extra folders that I do not want in the output zip file. It does put all my desired files in the .zip but it seems to add the last few directories from the files being written in the .zip file by default. Is there any way to exclude these folders? Here is my code:
import arcpy, os
from os import path as p
import zipfile
arcpy.overwriteOutput = True
def ZipShapes(path, out_path):
arcpy.env.workspace = path
shapes = arcpy.ListFeatureClasses()
# iterate through list of shapefiles
for shape in shapes:
name = p.splitext(shape)[0]
print name
zip_path = p.join(out_path, name + '.zip')
zip = zipfile.ZipFile(zip_path, 'w')
zip.write(p.join(path,shape))
for f in arcpy.ListFiles('%s*' %name):
if not f.endswith('.shp'):
zip.write(p.join(path,f))
print 'All files written to %s' %zip_path
zip.close()
if __name__ == '__main__':
path = r'C:\Shape_test\Census_CedarCo'
out_path = r'C:\Shape_outputs'
ZipShapes(path, out_path)
I tried to post some pictures but I do not have enough reputation points. Basically it is adding 2 extra folders (empty) inside the zip file. So instead of the files being inside the zip like this:
C:\Shape_outputs\Public_Buildings.zip\Public_Buildings.shp
They are showing up like this:
C:\Shape_outputs\Public_Buildings.zip\Shape_test\Census_CedarCo\Public_Buildings.shp
The "Shape_test" and "Census_CedarCo" folders are the directories that the shapefiles I am trying to copy come from, but if I am just writing these files why are the sub directories also being copied into the zip file? I suppose it is not a huge deal since I am getting the files zipped, but it is more of an annoyance than anything.
I assumed that when creating a zip file it would just write the files I specify themselves. Why does it add these extra directories inside the zip file? Is there a way around it? Am I missing something here? I appreciate any input! Thanks
The optional second parameter in ZipFile.write(filename[, arcname[, compress_type]]) is that name used in the archive file. You can strip the offending folders from the front of the path and use the remainder for the archive path name. I'm not sure exactly how arcpy gives you the paths, but something like zip.write(p.join(path,shape), shape) should do it.