I am trying to create a program in which I should be able to overlay one image over another base image using python and opencv and store the output image in an other folder . I am using opencv to achieve this however the code I have written is not giving the desired result.
import cv2
from os import listdir
from os.path import isfile, join
import numpy as np
path_wf = 'wf_flare'
path_fr = 'captured_flare'
files_wf = [ f for f in listdir(path_wf) if isfile(join(path_wf,f))]
files_fr = [ fl for fl in listdir(path_fr) if isfile(join(path_fr,fl))]
img_wf = np.empty(len(files_wf), dtype = object)
img_fr = np.empty(len(files_fr), dtype = object)
img = np.empty(len(files_wf), dtype = object)
k = 0
for n in range(0, len(files_wf)):
img_wf[n] = cv2.imread(join(path_wf, files_wf[n]))
img_fr[k] = cv2.imread(join(path_fr, files_fr[k]))
print("Done Reading"+str(n))
img_wf[n] = cv2.resize(img_wf[n], (1024,1024),interpolation = cv2.INTER_AREA)
img[n] = 0.4*img_fr[k] + img_wf[n]
fn = listdir(path_wf)
name = 'C:\Flare\flare_img'+str(fn[n])
print('Creating...'+ name + str(n+10991))
cv2.imwrite(name,img[n])
k += 1
if(k%255 == 0):
k = 0
else:
continue
the folder organization is pasted below:
I want the output images to come here:
There are two issues in the following line:
name = 'C:\Flare\flare_img'+str(fn[n])
In Python, special characters in strings are escaped with backslashes. Some examples are \n (newline), \t (tab), \f (form feed), etc. In your case, the \f is a special character that leads to a malformed path. One way to fix this is to use raw strings by adding an r before the first quote:
'C:\Flare\flare_img'
Out[12]: 'C:\\Flare\x0clare_img'
r'C:\Flare\flare_img'
Out[13]: 'C:\\Flare\\flare_img'
Do not just concatenate strings when you create filesystem paths. Sooner or later you end up misplacing a path separator. In this case, it is missing because fn[n] does not start with one. Let's say that fn[n] = "spam.png". Then assuming you do
name = r'C:\Flare\flare_img'+str(fn[n])
your value for name will be
C:\\Flare\\flare_imgspam.png
which is not what you intended.
Use os.path.join or the modern pathlib.Path as previously suggested. It is also redundant to wrap fn[n] in the str function because os.listdir already returns a list of strings.
The changes you need to make are as follows:
# add to imports section
from pathlib import Path
# add before for-loop
out_path = Path(r'C:\Flare\flare_img')
# change inside for-loop
name = out_path / fn[n]
Documentation: Python Strings
Related
I'm confused when trying to replace specific text in python
my code is:
Image = "/home/user/Picture/Image-1.jpg"
Image2 = Image.replace("-1", "_s", 1)
print(Image)
print(Image2)
Output:
/home/user/Picture/Image-1.jpg
/home/user/Picture/Image_s.jpg
The output what I want from Image2 is:
/home/user/Picture/Image-1_s.jpg
You are replacing the -1 with _s
If you want to keep the -1 as well, you can just add it in the replacement
Image = "/home/user/Picture/Image-1.jpg"
Image2 = Image.replace("-1", "-1_s", 1)
print(Image)
print(Image2)
Output
/home/user/Picture/Image-1.jpg
/home/user/Picture/Image-1_s.jpg
If the digits can be variable, you can also use a pattern with for example 2 capture groups, and then use those capture groups in the replacement with _s in between
import re
pattern = r"(/home/user/Picture/Image-\d+)(\.jpg)\b"
s = "/home/user/Picture/Image-1.jpg\n/home/user/Picture/Image-2.jpg"
print(re.sub(pattern, r"\1_s\2", s))
Output
/home/user/Picture/Image-1_s.jpg
/home/user/Picture/Image-2_s.jpg
Or for example only taking the /Image- into account and then use the full match in the replacement instead of using capture groups:
import re
pattern = r"/Image-\d+(?=\.jpg)\b"
s = "/home/user/Picture/Image-1.jpg\n/home/user/Picture/Image-2.jpg"
print(re.sub(pattern, r"\g<0>_s", s))
Output
/home/user/Picture/Image-1_s.jpg
/home/user/Picture/Image-2_s.jpg
The behaviour of the code you wrote is working as I would have expected from reading it. Now, as to how to correct it to do what you expected/wanted it to do is a little different. You don't necessarily need to replace here, instead, you can consider appending what you need, as it seems from the behaviour you are looking for is in fact appending something to the end of the path before the extension.
We can try to help the code a bit by making it a little more "generic" by allowing us to simply "append" anything to the end of a string. The steps we can do to achieve this is (for other readers, yes there are more foolproof ways to do this, for now sticking to a simple example) :
split the string at . so that you end up with a list containing:
["/home/user/Picture/Image-1", "jpg"]
Append to the first element what you need to the end of the string so you end up with:
"/home/user/Picture/Image-1_s"
Use join to re-craft your string, but use .:
".".join(["/home/user/Picture/Image-1_s", "jpg"])
You will finally get:
/home/user/Picture/Image-1_s.jpg
Coding the above, we can have it work as follows:
>>> Image1 = "/home/user/Picture/Image-1.jpg"
>>> img_split = Image1.split(".")
>>> img_split
['/home/user/Picture/Image-1', 'jpg']
>>> img_split[0] = img_split[0] + "_s"
>>> img_split
['/home/user/Picture/Image-1_s', 'jpg']
>>> final_path = ".".join(img_split)
>>> final_path
'/home/user/Picture/Image-1_s.jpg'
More idiomatic approach using Python's pathlib module is an interesting solution too.
from pathlib import Path
Image1 = "/home/user/Picture/Image-1.jpg"
p = Path(Image1)
# you have access to all the parts you need. Like the path to the file:
p.parent # outputs PosixPath('/home/user/Picture/')
# The name of the file without extension
p.stem # outputs 'Image-1'
# The extension of the file
p.suffix # outputs '.jpg'
# Finally, we get to now rename it using the rename method!
p.rename(p.parent / f"{p.stem}_s{p.suffix}")
# This will now result in the following object with renamed file!
# PosixPath('/home/user/Picture/Image-1_s.jpg')
The replace function replaces "-1" with "_s".
If you want the output to be: /home/user/Picture/Image-1_s.jpg
You should replace "-1" with "-1_s".
Try:
Image = "/home/user/Picture/Image-1.jpg"
Image2 = Image.replace("-1", "-1_s")
print(Image)
print(Image2)
Try this
i think you should append the string in a certain position not replace
Image = "/home/user/Picture/Image-1.jpg"
Image2 = Image[:26]+ '_s' + Image[26:]
print(Image2)
The output
I want to insert a directory name in the middle of a given file path, like this:
directory_name = 'new_dir'
file_path0 = 'dir1/dir2/dir3/dir4/file.txt'
file_path1 = some_func(file_path0, directory_name, position=2)
print(file_path1)
>>> 'dir1/dir2/new_dir/dir3/dir4/file.txt'
I looked through the os.path and pathlib packages, but it looks like they don't have a function that allows for inserting in the middle of a file path. I tried:
import sys,os
from os.path import join
path_ = file_path0.split(os.sep)
path_.insert(2, 'new_dir')
print(join(path_))
but this results in the error
"expected str, bytes or os.PathLike object, not list"
Does anyone know standard python functions that allow such inserting in the middle of a file path? Alternatively - how can I turn path_ to something that can be processed by os.path. I am new to pathlib, so maybe I missed something out there
Edit: Following the answers to the question I can suggest the following solutions:
1.) As Zach Favakeh suggests and as written in this answer just correct my code above to join(*path_) by using the 'splat' operator * and everything is solved.
2.) As suggested by buran you can use the pathlib package, in very short it results in:
from pathlib import PurePath
path_list = list(PurePath(file_path0).parts)
path_list.insert(2, 'new_dir')
file_path1 = PurePath('').joinpath(*path_list)
print(file_path1)
>>> 'dir1/dir2/new_dir/dir3/dir4/file.txt'
Take a look at pathlib.PurePath.parts. It will return separate components of the path and you can insert at desired position and construct the new path
>>> from pathlib import PurePath
>>> file_path0 = 'dir1/dir2/dir3/dir4/file.txt'
>>> p = PurePath(file_path0)
>>> p.parts
('dir1', 'dir2', 'dir3', 'dir4', 'file.txt')
>>> spam = list(p.parts)
>>> spam.insert(2, 'new_dir')
>>> new_path = PurePath('').joinpath(*spam)
>>> new_path
PurePosixPath('dir1/dir2/new_dir/dir3/dir4/file.txt')
This will work with path as a str as well as with pathlib.Path objects
Since you want to use join on a list to produce the pathname, you should do the following using the "splat" operator: Python os.path.join() on a list
Edit: You could also take your np array and concatenate its elements into a string using np.array2string, using '/' as your separator parameter:https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.array2string.html
Hope this helps.
Solution using regex. The regex will create groups of the following
[^\/]+ - non-'/' characters(i.e. directory names)
\w+\.\w+ - word characters then '.' then word characters (i.e. file name)
import re
directory_name = 'new_dir'
file_path0 = 'dir1/dir2/dir3/dir4/file.txt'
position = 2
regex = re.compile(r'([^\/]+|\w+\.\w+)')
tokens = re.findall(regex, file_path0)
tokens.insert(position, directory_name)
file_path1 = '/'.join(tokens)
Result:
'dir1/dir2/new_dir/dir3/dir4/file.txt'
Your solution has only one flaw. After inserting the new directory in the path list path_.insert(2, 'new_dir')you need to call os.path.join(*path_) to get the new modified path. The error that you get is because you are passing a list as parameter to the join function, but you have to unpack it.
In my case, I knew the portion of path that would precede the insertion point (i.e., "root"). However, the position of the insertion point was not constant due to the possibility of having varying number of path components in the root path. I used Path.relative_to() to break the full path to yield an insertion point for the new_dir.
from pathlib import Path
directory_name = Path('new_dir')
root = Path('dir1/dir2/')
file_path0 = Path('dir1/dir2/dir3/dir4/file.txt')
# non-root component of path
chld = file_path0.relative_to(root)
file_path1 = root / directory_name / chld
print(file_path1)
Result:
'dir1/dir2/new_dir/dir3/dir4/file.txt'
I made a try with your need:
directory_name = '/new_dir'
file_path0 = 'dir1/dir2/dir3/dir4/file.txt'
before_the_newpath = 'dir1/dir2'
position = file_path0.split(before_the_newpath)
new_directory = before_the_newpath + directory_name + position[1]
Hope it helps.
I want to create a list of all the filepath names that match a specific string e.g. "04_DEM" so I can do further processing on the files inside those directories?
e.g.
INPUT
C:\directory\NewZealand\04DEM\DEM_CD23_1232.tif
C:\directory\Australia\04DEM\DEM_CD23_1233.tif
C:\directory\NewZealand\05DSM\DSM_CD23_1232.tif
C:\directory\Australia\05DSM\DSM_CD23_1232.tif
WANTED OUTPUT
C:\directory\NewZealand\04DEM\
C:\directory\Australia\04DEM\
This makes sure that only those files are processed, as some other files in the directories also have the same string "DEM" included in their filename, which I do not want to modify.
This is my bad attempt due to being a rookie with Py code
import os
for dirnames in os.walk('D:\Canterbury_2017Copy'):
print dirnames
if dirnames=='04_DEM' > listofdirectoriestoprocess.txt
print "DONE CHECK TEXT FILE"
You can use os.path for this:
import os
lst = [r'C:\directory\NewZealand\04DEM\DEM_CD23_1232.tif',
r'C:\directory\Australia\04DEM\DEM_CD23_1233.tif',
r'C:\directory\NewZealand\05DSM\DSM_CD23_1232.tif',
r'C:\directory\Australia\05DSM\DSM_CD23_1232.tif']
def filter_paths(lst, x):
return [os.path.split(i)[0] for i in lst if os.path.normpath(i).split(os.sep)[3] == x]
res = list(filter_paths(lst, '04DEM'))
# ['C:\\directory\\NewZealand\\04DEM',
# 'C:\\directory\\Australia\\04DEM']
Use in to check if a required string is in another string.
This is one quick way:
new_list = []
for path in path_list:
if '04DEM' in path:
new_list.append(path)
Demo:
s = 'C:/directory/NewZealand/04DEM/DEM_CD23_1232.tif'
if '04DEM' in s:
print(True)
# True
Make sure you use / or \\ as directory separator instead of \ because the latter escapes characters.
First, you select via regex using re, and then use pathlib:
import re
import pathlib
pattern = re.compile('04DEM')
# You use pattern.search() if s is IN the string
# You use pattern.match() if s COMPLETELY matches the string.
# Apply the correct function to your use case.
files = [s in list_of_files if pattern.search(s)]
all_pruned_paths = set()
for p in files:
total = ""
for d in pathlib.Path(p):
total = os.path.join(total, d)
if pattern.search(s):
break
all_pruned_paths.add(total)
result = list(all_pruned_paths)
This is more robust than using in because you might need to form more complicated queries in the future.
I have searched possible ways but I am unable to mix those up yet. I have a string that is a path to the image.
myString= "D:/Train/16_partitions_annotated/partition1/images/AAAAA/073-1_00191.jpeg"
What I want to do is replace images with IMAGES and cut off the 073-1_00191.jpeg part at the end. Thus, the new string string should be
newString = "D:/Train/16_partitions_annotated/partition1/IMAGES/AAAAA/"
And the chopped part (073-1_00191.jpeg) will be used separately as the name of processed image. The function .replace() doesn't work here as I need to provide path and filename as separate parameters.
The reason why I want to do is that I am accessing images through their paths and doing some stuff on them and when saving them I need to create another directory (in this case IMAGES) and the next directories after that (in this case AAAAA) should remain the same ( together with the name of corresponding image).
Note that images may have different names and extensions
If something is not clear by my side please ask, I will try to clear up
As alluded to in the comments, os.path is useful for manipulating paths represented as strings.
>>> import os
>>> myString= "D:/Train/16_partitions_annotated/partition1/images/AAAAA/073-1_00191.jpeg"
>>> dirname, basename = os.path.split(myString)
>>> dirname
'D:/Train/16_partitions_annotated/partition1/images/AAAAA'
>>> basename
'073-1_00191.jpeg'
At this point, how you want to handle capitalizing "images" is a function of your broader goal. If you want to simply capitalize that specific word, dirname.replace('images', 'IMAGES') should suffice. But you seem to be asking for a more generalized way to capitalize the second to last directory in the absolute path:
>>> def cap_penultimate(dirname):
... h, t = os.path.split(dirname)
... hh, ht = os.path.split(h)
... return os.path.join(hh, ht.upper(), t)
...
>>> cap_penultimate(dirname)
'D:/Train/16_partitions_annotated/partition1/IMAGES/AAAAA'
It's game of slicing , Here you can try this :
myString= "D:/Train/16_partitions_annotated/partition1/images/AAAAA/073-1_00191.jpeg"
myString1=myString.split('/')
pre_data=myString1[:myString1.index('images')]
after_data=myString1[myString1.index('images'):]
after_data=['IMAGE'] + after_data[1:2]
print("/".join(pre_data+after_data))
output:
D:/Train/16_partitions_annotated/partition1/IMAGE/AAAAA
The simple way :
myString= "D:/Train/16_partitions_annotated/partition1/images/AAAAA/073-1_00191.jpeg"
a = myString.rfind('/')
filename = myString[a+1:]
restofstring = myString[0:a]
alteredstring = restofstring.replace('images', 'IMAGES')
print(alteredstring)
output:
D:/Train/16_partitions_annotated/partition1/IMAGE/AAAAA
I have list of numbpy files, I need to change their names, In fact, let's assume that I have this list of files:
AES_Trace=1_key=hexaNumber_Plaintext=hexaNumber_Ciphertext=hexaNumber.npy
AES_Trace=2_key=hexaNumber_Plaintext=hexaNumber_Ciphertext=hexaNumber.npy
AES_Trace=3_key=hexaNumber_Plaintext=hexaNumber_Ciphertext=hexaNumber.npy
What I need to change is the number of files, as a result I must have:
AES_Trace=100001_key=hexaNumber_Plaintext=hexaNumber_Ciphertext=hexaNumber.npy
AES_Trace=100002_key=hexaNumber_Plaintext=hexaNumber_Ciphertext=hexaNumber.npy
AES_Trace=100003_key=hexaNumber_Plaintext=hexaNumber_Ciphertext=hexaNumber.npy
I have tried:
import os
import numpy as np
import struct
path_For_Numpy_Files='C:\\Users\\user\\My_Test_Traces\\1000_Traces_npy'
os.chdir(path_For_Numpy_Files)
list_files_Without_Sort=os.listdir(os.getcwd())
list_files_Sorted=sorted((list_files_Without_Sort),key=os.path.getmtime)
for file in list_files_Sorted:
print (file)
os.rename(file,file[11]+100000)
I think that is not the good solution, firstly It doesn't work, then it gives me this error:
os.rename(file,file[11]+100000)
IndexError: string index out of range
Your file variable is a str, so you can't add an int like 10000 to it.
>>> file = 'Tracenumber=01_Pltx5=23.npy'
>>> '{}=1000{}'.format(file.split('=')[0],file.split('=')[1:])
'Tracenumber=100001_Pltx5=23.npy'
So, you can rather use
os.rename(file,'{}=1000{}'.format(file.split('=')[0],file.split('=')[1:]))
I'm sure that you can do this in one line, or with regex but I think that clarity is more valuable. Try this:
import os
path = 'C:\\Users\\user\\My_Test_Traces\\1000_Traces_npy'
file_names = os.listdir(path)
for file in file_names:
start = file[0:file.index("Trace=")+6]
end = file[file.index("_key"):]
num = file[len(start): file.index(end)]
new_name = start + str(100000+int(num)) + end
os.rename(os.path.join(path, file), os.path.join(path, new_name))
This will work with numbers >9, which the other answer will stick extra zeros onto.