New to python, new to OpenCV, which I'm gonna use for my master-thesis, and already got some problems using the VideoCapture object of OpenCV.
Situation:
I got 2 folders containing corresponding images (taken with RGB and infrared cameras). I want to display them sibe by side in a Window using a while-loop. The problem arises, when there are some images missing from one of the image-sequences (Due to problems while recording or whatever, I don't really know but that should be of no importance). My idea was to use the bool-returnvalue of the .read() function to check wheather there is a frame to be read and if not, replace the image by a black one. This is what I did:
Code:
import cv2
import numpy as np
pathRGB = "Bilder/RGB"
pathIR = "Bilder/IR"
# the paths to the folders containing the images
capRGB = cv2.VideoCapture(pathRGB + "/frame_%06d.jpg")
capIR = cv2.VideoCapture(pathIR + "/frame_%06d.jpg")
# setting up the VideoCapture-elements with the according format
shapeRGB = capRGB.read()[1].shape
shapeIR = capIR.read()[1].shape
# get the shape of the first image in each folder to later create the black
# dummy-image
dtypeRGB = capRGB.read()[1].dtype
dtypeIR = capIR.read()[1].dtype
# get the type of the first image in each folder to later create the black
# dummy-image
if (capRGB.isOpened() is False):
print("Error opening RGB images")
if (capIR.isOpened() is False):
print("Error opening IR images")
cv2.namedWindow("frames", cv2.WINDOW_NORMAL)
while capRGB.isOpened() and capIR.isOpened() is True:
retRGB, imgRGB = capRGB.read()
retIR, imgIR = capIR.read()
# read both images
if retRGB is True and retIR is False:
imgIR = np.zeros(shapeIR, dtype=dtypeIR)
# if there is no IR image, crate a dummy one
if retIR is True and retRGB is False:
imgRGB = np.zeros(shapeRGB, dtype=dtypeRGB)
# if there is no RGB image, crate a dummy one
if retRGB is False and retIR is False:
break
imgCombined = np.hstack((imgRGB, imgIR))
# put both images together
cv2.imshow("frames", imgCombined)
k = cv2.waitKey(1)
if k == ord("q"):
break
capRGB.release()
capIR.release()
cv2.destroyAllWindows()
Problem:
From my understanding, the problem arises as capIR.read() attempts to read a missing image (in my case the 527th) and instead of just returning false/None it attempts to read the same image over and over again. Up to the missing frame, everything works fine, the right "IR" image even turns black but then the videoplayback begins to slow down and while i still can close the window by pressing 'q', spyder IDE freezes and if I wait "too long" i even have to shut it down. Console gives out "[image2 # 000002a7af8f0480] Could not open file : Bilder/IR/frame_000527.jpg" over and over again, so much that i can't scroll to the top.
I guess what I'm asking is: Is there any way to make the .read() function just attempt 1 read and after it fails continue with the next frame?
Best regards and thank you very much in advance!
Simulated for testing with different files and directory names.
Will retrieve the largest frame number from both directories and afterwards iterate over all frame numbers for reading the files from both directories.
import os
import cv2
import re
import glob
image_dir1 = 'test1'
image_dir2 = 'test2'
# retrieve all frames in both directories
frames_cap1 = glob.glob(os.path.join(image_dir1, "frame_*.jpg"))
frames_cap2 = glob.glob(os.path.join(image_dir2, "frame_*.jpg"))
# sort inscending
frames_cap1.sort()
frames_cap2.sort()
# retrieve last frame No for both directories
last_frame_cap1 = frames_cap1[-1]
last_frame_cap2 = frames_cap2[-1]
# extract integer counter as a group
# modify regex to match file name if required
match_cap1 = re.search('frame_(\d+).jpg', last_frame_cap1)
match_cap2 = re.search('frame_(\d+).jpg', last_frame_cap2)
last_frame_no_cap1 = int(match_cap1.group(1))
last_frame_no_cap2 = int(match_cap2.group(1))
# retrieve max frame No
max_frame_no = max(last_frame_no_cap1, last_frame_no_cap2)
for i in range(max_frame_no + 1):
# adapt formatting of frame number to digit count in file name
# here: 6 digits with leading zeros
image_path_cap1 = os.path.join(image_dir1, f"frame_{i:06d}.jpg")
image_path_cap2 = os.path.join(image_dir2, f"frame_{i:06d}.jpg")
if not os.path.isfile(image_path_cap1):
print(f"handle missing file: '{image_path_cap1}'")
# ...
else:
img1 = cv2.imread(image_path_cap1)
# …
if not os.path.isfile(image_path_cap2):
print(f"handle missing file: '{image_path_cap2}'")
# ...
else:
img2 = cv2.imread(image_path_cap2)
# …
# …
Assuming that the images in directory1 have the same names as directory2 images, but we know that some image may not be present in both directories...
import glob,os,cv2
path1 = "folder1/"
path2 = "folder2/"
#change directory to path1
os.chdir(path1)
l1 = glob.glob("*.jpg") #get a list of images names
os.chdir("../") #go one directory up
blackimg = cv2.imread("blackimg.jpg")
for fname in l1:
#check if image1 exists , then read it . otherwise im1 = blackimg
if os.path.isfile(path1+fname):
im1=cv2.imread(path1+fname)
else:
im1=blackimg
#check if image2 exists , then read it . otherwise im2 = blackimg
if os.path.isfile(path2+fname):
im2=cv2.imread(path2+fname)
else:
im2=blackimg
imgCombined = np.hstack((im1, im2))
cv2.imshow("Combined", imgCombined)
print("press any key to continue, q to exit")
k = cv2.waitKey(0)
if k == ord("q"):break
cv2.destroyAllWindows()
Goal:
I am trying to batch process images contained inside a folder to resize and optimize them for use online.
Problem:
The following script works, but I have to run it twice before I get the output I want. This is how I would expect it to work:
function 1: resize_aspect_fit()
Resizes each image in the target folder to a specific size, adds "_small.png" to the file name, and saves it as a new file in the subfolder "optimized_images", created in the same directory as the original group of images.
function2: png_conversion()
Takes the newly made images inside "optimized_images" ("_small.png") and applies a conversion that reduces the size of the original file, adding the "-opt.png" suffix to indicate it has been optimized.
function3: unoptimized_cleanup()
Takes the files built by function 1, which are no longer necessary (since they have been optimized) and deletes them, to reduce clutter.
When I run the script I get the expected response from function1, all files in the target file are resized appropriately and saved in the "optimized_images" folder. But I have to run the script a second time before function 2 and 3 take effect. It does work, but I have never encountered an issue like this before. Any idea why this is happening?
What I tried:
I thought this might be related to file open/close operations, but I think I am closing them all at the appropriate time. I swapped Image.open syntax to use "with Image.open(path) as image:" but that did not solve the problem.
I thought there might be some issue with os.listdir or os.path where it might have to be 'reset' in order to iterate through a directory of files twice, but I cannot find anything.
from PIL import Image
import os, sys
path = "../path/to/images/"
new_folder = '/optimized_images/'
optimized_path = path + new_folder[1:]
dirs = os.listdir( path )
optimized_dirs = os.listdir( optimized_path )
def resize_aspect_fit(final_size=250, dirs=dirs, optimized_path=optimized_path, optimized_dirs=optimized_dirs):
for item in dirs:
if item == '.DS_Store':
continue
if os.path.isfile(path+item):
with Image.open(path+item) as im:
f, e = os.path.splitext(path+item)
size = im.size
ratio = float(final_size) / max(size)
new_image_size = tuple([int(x*ratio) for x in size])
im = im.resize(new_image_size, Image.ANTIALIAS)
new_im = Image.new("RGBA", (final_size, final_size), color=(255,255,255,0))
new_im.paste(im, ((final_size-new_image_size[0])//2, (final_size-new_image_size[1])//2))
new_path, new_filename = f.rsplit('/', 1)
new_im.save(new_path + new_folder + new_filename + '_small.png', 'PNG', quality=10, optimize=True)
new_im.close()
def png_conversion(optimized_dirs=optimized_dirs, optimized_path=optimized_path):
for item in optimized_dirs:
if item == '.DS_Store':
continue
f, e = os.path.splitext(optimized_path+item)
with Image.open(f + e) as im:
im.load()
# Get the alpha band
alpha = im.split()[-1]
im = im.convert('RGB').convert('P', palette=Image.ADAPTIVE, colors=255)
# Set all pixel values below 128 to 255,
# and the rest to 0
mask = Image.eval(alpha, lambda a: 255 if a <=128 else 0)
# Paste the color of index 255 and use alpha as a mask
im.paste(255, mask)
# The transparency index is 255
e = e.split('.png')[0]
im.save(f + e + "-opt.png", transparency=255)
im.close()
def unoptimized_cleanup(optimized_dirs=optimized_dirs, optimized_path=optimized_path):
for item in optimized_dirs:
if item.endswith('small.png'):
os.remove(os.path.join(optimized_path, item))
#functions called in order
resize_aspect_fit(final_size=250, dirs=dirs)
png_conversion(optimized_dirs=optimized_dirs, optimized_path=optimized_path)
unoptimized_cleanup(optimized_dirs=optimized_dirs, optimized_path=optimized_path)
I expect that for the following folder structure:
folder/image1.png
folder/image2.png
the output should look like this, with the appropriately sized and smaller files:
folder/optimized_images/image1_small-opt.png
folder/optimized_images/image2_small-opt.png
Relevant Sources that I pulled from:
Converting PNG32 to PNG8 with PIL while preserving transparency
Python/PIL Resize all images in a folder
Sorry for the long question/code, and thanks in advance for any help!!
The problem is that you create the variable optimized_dirs before you run step 1. So before step 1 is executed, you make a list of files in that directory, which is empty at that point. If you run it a second time, the files are in optimized_dirs, and hence then it works.
A solution would be to read the contents of optimized_dirs inside the function png_compression, i.e. moving os.listdir( optimized_path ) in there.
By the way: I see that you do some magic to build paths where you use [1:] to prevent double slashes. It is more robust to build paths using os.path.join, which will ensure there is always a single slash between directories, regardless of whether you specify them at the start or end of each of them.
I've got an image database with a set of images named [frame01.png, frame02.png, ..., frameN.png].
My directory path is ./img, and iteratively I'd like to read one by one, do some image processing until reaching the last one. Since I'm not familiar with strings concatenation in python, what's the easiest way to do it?
file_names = os.listdir('path_to_folder/')
should give you a list of all you files.
To read them you can have:
for file_name in file_names:
read_and_process_image('path_to_folder/' + file_name)
Then inside read_and_process_image:
import matplotlib.image
def read_and_process_image(path):
read_img = matplotlib.image.imread(path) # or whatever you use to read the image
# process read_img
Alternatively, you could have:
import glob
for image_path in glob.glob("path_to_your_image*.png"):
image = matplotlib.image.imread(image_path) # or whatever you use to read the image
# process your image
If you are just looking for a quick way to create the list with this particular names:
[ 'frame' + "%02d" % (i,) + '.png' for i in range(1, MAX_NUM)]
If your last image is 20 then replace MAX_NUM with 20 + 1 applies for any other number x, x + 1.
How/what you use to read the files depends on you. You can use matplotlib.image as in the examples or whatever works for you.
So I've followed this tutorial but it doesn't seem to do anything. Simply nothing. It waits a few seconds and closes the program. What is wrong with this code?
import cv2
vidcap = cv2.VideoCapture('Compton.mp4')
success,image = vidcap.read()
count = 0
success = True
while success:
success,image = vidcap.read()
cv2.imwrite("frame%d.jpg" % count, image) # save frame as JPEG file
if cv2.waitKey(10) == 27: # exit if Escape is hit
break
count += 1
Also, in the comments it says that this limits the frames to 1000? Why?
EDIT:
I tried doing success = True first but that didn't help. It only created one image that was 0 bytes.
From here download this video so we have the same video file for the test. Make sure to have that mp4 file in the same directory of your python code. Then also make sure to run the python interpreter from the same directory.
Then modify the code, ditch waitKey that's wasting time also without a window it cannot capture the keyboard events. Also we print the success value to make sure it's reading the frames successfully.
import cv2
vidcap = cv2.VideoCapture('big_buck_bunny_720p_5mb.mp4')
success,image = vidcap.read()
count = 0
while success:
cv2.imwrite("frame%d.jpg" % count, image) # save frame as JPEG file
success,image = vidcap.read()
print('Read a new frame: ', success)
count += 1
How does that go?
To extend on this question (& answer by #user2700065) for a slightly different cases, if anyone does not want to extract every frame but wants to extract frame every one second. So a 1-minute video will give 60 frames(images).
import sys
import argparse
import cv2
print(cv2.__version__)
def extractImages(pathIn, pathOut):
count = 0
vidcap = cv2.VideoCapture(pathIn)
success,image = vidcap.read()
success = True
while success:
vidcap.set(cv2.CAP_PROP_POS_MSEC,(count*1000)) # added this line
success,image = vidcap.read()
print ('Read a new frame: ', success)
cv2.imwrite( pathOut + "\\frame%d.jpg" % count, image) # save frame as JPEG file
count = count + 1
if __name__=="__main__":
a = argparse.ArgumentParser()
a.add_argument("--pathIn", help="path to video")
a.add_argument("--pathOut", help="path to images")
args = a.parse_args()
print(args)
extractImages(args.pathIn, args.pathOut)
This is Function which will convert most of the video formats to number of frames there are in the video. It works on Python3 with OpenCV 3+
import cv2
import time
import os
def video_to_frames(input_loc, output_loc):
"""Function to extract frames from input video file
and save them as separate frames in an output directory.
Args:
input_loc: Input video file.
output_loc: Output directory to save the frames.
Returns:
None
"""
try:
os.mkdir(output_loc)
except OSError:
pass
# Log the time
time_start = time.time()
# Start capturing the feed
cap = cv2.VideoCapture(input_loc)
# Find the number of frames
video_length = int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) - 1
print ("Number of frames: ", video_length)
count = 0
print ("Converting video..\n")
# Start converting the video
while cap.isOpened():
# Extract the frame
ret, frame = cap.read()
if not ret:
continue
# Write the results back to output location.
cv2.imwrite(output_loc + "/%#05d.jpg" % (count+1), frame)
count = count + 1
# If there are no more frames left
if (count > (video_length-1)):
# Log the time again
time_end = time.time()
# Release the feed
cap.release()
# Print stats
print ("Done extracting frames.\n%d frames extracted" % count)
print ("It took %d seconds forconversion." % (time_end-time_start))
break
if __name__=="__main__":
input_loc = '/path/to/video/00009.MTS'
output_loc = '/path/to/output/frames/'
video_to_frames(input_loc, output_loc)
It supports .mts and normal files like .mp4 and .avi. Tried and Tested on .mts files. Works like a Charm.
This is a tweak from previous answer for python 3.x from #GShocked, I would post it to the comment, but dont have enough reputation
import sys
import argparse
import cv2
print(cv2.__version__)
def extractImages(pathIn, pathOut):
vidcap = cv2.VideoCapture(pathIn)
success,image = vidcap.read()
count = 0
success = True
while success:
success,image = vidcap.read()
print ('Read a new frame: ', success)
cv2.imwrite( pathOut + "\\frame%d.jpg" % count, image) # save frame as JPEG file
count += 1
if __name__=="__main__":
print("aba")
a = argparse.ArgumentParser()
a.add_argument("--pathIn", help="path to video")
a.add_argument("--pathOut", help="path to images")
args = a.parse_args()
print(args)
extractImages(args.pathIn, args.pathOut)
The previous answers have lost the first frame. And it will be nice to store the images in a folder.
# create a folder to store extracted images
import os
folder = 'test'
os.mkdir(folder)
# use opencv to do the job
import cv2
print(cv2.__version__) # my version is 3.1.0
vidcap = cv2.VideoCapture('test_video.mp4')
count = 0
while True:
success,image = vidcap.read()
if not success:
break
cv2.imwrite(os.path.join(folder,"frame{:d}.jpg".format(count)), image) # save frame as JPEG file
count += 1
print("{} images are extacted in {}.".format(count,folder))
By the way, you can check the frame rate by VLC. Go to windows -> media information -> codec details
After a lot of research on how to convert frames to video I have created this function hope this helps. We require opencv for this:
import cv2
import numpy as np
import os
def frames_to_video(inputpath,outputpath,fps):
image_array = []
files = [f for f in os.listdir(inputpath) if isfile(join(inputpath, f))]
files.sort(key = lambda x: int(x[5:-4]))
for i in range(len(files)):
img = cv2.imread(inputpath + files[i])
size = (img.shape[1],img.shape[0])
img = cv2.resize(img,size)
image_array.append(img)
fourcc = cv2.VideoWriter_fourcc('D', 'I', 'V', 'X')
out = cv2.VideoWriter(outputpath,fourcc, fps, size)
for i in range(len(image_array)):
out.write(image_array[i])
out.release()
inputpath = 'folder path'
outpath = 'video file path/video.mp4'
fps = 29
frames_to_video(inputpath,outpath,fps)
change the value of fps(frames per second),input folder path and output folder path according to your own local locations
This code extract frames from the video and save the frames in .jpg formate
import cv2
import numpy as np
import os
# set video file path of input video with name and extension
vid = cv2.VideoCapture('VideoPath')
if not os.path.exists('images'):
os.makedirs('images')
#for frame identity
index = 0
while(True):
# Extract images
ret, frame = vid.read()
# end of frames
if not ret:
break
# Saves images
name = './images/frame' + str(index) + '.jpg'
print ('Creating...' + name)
cv2.imwrite(name, frame)
# next frame
index += 1
In 2022 you also have the option to use ImageIO to do this, which IMHO is much more hasslefree and readable.
import imageio.v3 as iio
for idx, frame in enumerate(iio.imiter("imageio:cockatoo.mp4")):
iio.imwrite(f"extracted_images/frame{idx:03d}.jpg", frame)
Sidenote 1: "imageio:cockatoo.mp4" is a standard image provided by ImageIO for testing and demonstration purposes. You can simply replace it with "path/to/your/video.mp4".
Sidenote 2: You will have to install one of ImageIO's optional dependencies to read video data, which can be done via pip install imageio-ffmpeg or pip install av.
You can time this against OpenCV and you will find that, there isn't that much to gain from OpenCV on this front either:
Read-Only Timings
=================
OpenCV: 0.453
imageio_ffmpeg: 0.765
imageio_pyav: 0.272
Read + Write Timings
====================
OpenCV: 3.237
imageio_ffmpeg: 1.597
imageio_pyav: 1.506
By default, OpenCV and ImageIO+av are about equally fast when reading. Both direct bind into the FFmpeg libraries under the hood so this is rather unsurprising. However, ImageIO allows you to tweak FFmpeg's default threadding model (thread_type="FRAME") which is much faster when bulk reading.
More importantly, ImageIO is much faster at writing JPEG compared to OpenCV. This is because pillow is faster than OpenCV here which ImageIO capitalizes on. Writing images dominates runtime for this scenario, so you end up with an overall 2x improvement when using ImageIO instead of OpenCV.
Here is the code for reference:
import imageio.v3 as iio
import cv2
import timeit
from pathlib import Path
# create a common local file for benchmarking
video_file = "shared_video.mp4"
if not Path(video_file).exists():
frames = iio.imread("imageio:cockatoo.mp4")
meta = iio.immeta("imageio:cockatoo.mp4", exclude_applied=False)
iio.imwrite(video_file, frames, fps=meta["fps"])
repeats = 10
def read_cv2():
vidcap = cv2.VideoCapture(video_file)
success, image = vidcap.read()
idx = 0
while success:
cv2.imwrite(f"extracted_images/frame{idx:03d}.jpg", image)
success, image = vidcap.read()
idx += 1
def read_imageio_ffmpeg():
for idx, frame in enumerate(iio.imiter(video_file, plugin="FFMPEG")):
iio.imwrite(f"extracted_images/frame{idx:03d}.jpg", frame)
def read_imageio_pyav():
for idx, frame in enumerate(
iio.imiter(video_file, plugin="pyav", format="rgb24", thread_type="FRAME")
):
iio.imwrite(f"extracted_images/frame{idx:03d}.jpg", frame)
time_cv2 = (
timeit.timeit("read_cv2()", setup="from __main__ import read_cv2", number=repeats)
/ repeats
)
time_imageio_ffmpeg = (
timeit.timeit(
"read_imageio_ffmpeg()",
setup="from __main__ import read_imageio_ffmpeg",
number=repeats,
)
/ repeats
)
time_imageio_pyav = (
timeit.timeit(
"read_imageio_pyav()",
setup="from __main__ import read_imageio_pyav",
number=repeats,
)
/ repeats
)
print(
f"""
Timings
=======
OpenCV: {time_cv2:<3.3f}
imageio_ffmpeg: {time_imageio_ffmpeg:<3.3f}
imageio_pyav: {time_imageio_pyav:<3.3f}
"""
)
Following script will extract frames every half a second of all videos in folder. (Works on python 3.7)
import cv2
import os
listing = os.listdir(r'D:/Images/AllVideos')
count=1
for vid in listing:
vid = r"D:/Images/AllVideos/"+vid
vidcap = cv2.VideoCapture(vid)
def getFrame(sec):
vidcap.set(cv2.CAP_PROP_POS_MSEC,sec*1000)
hasFrames,image = vidcap.read()
if hasFrames:
cv2.imwrite("D:/Images/Frames/image"+str(count)+".jpg", image) # Save frame as JPG file
return hasFrames
sec = 0
frameRate = 0.5 # Change this number to 1 for each 1 second
success = getFrame(sec)
while success:
count = count + 1
sec = sec + frameRate
sec = round(sec, 2)
success = getFrame(sec)
This function extracts images from video with 1 fps, IN ADDITION it identifies the last frame and stops reading also:
import cv2
import numpy as np
def extract_image_one_fps(video_source_path):
vidcap = cv2.VideoCapture(video_source_path)
count = 0
success = True
while success:
vidcap.set(cv2.CAP_PROP_POS_MSEC,(count*1000))
success,image = vidcap.read()
## Stop when last frame is identified
image_last = cv2.imread("frame{}.png".format(count-1))
if np.array_equal(image,image_last):
break
cv2.imwrite("frame%d.png" % count, image) # save frame as PNG file
print '{}.sec reading a new frame: {} '.format(count,success)
count += 1
I am using Python via Anaconda's Spyder software. Using the original code listed in the question of this thread by #Gshocked, the code does not work (the python won't read the mp4 file). So I downloaded OpenCV 3.2 and copied "opencv_ffmpeg320.dll" and "opencv_ffmpeg320_64.dll" from the "bin" folder. I pasted both of these dll files to Anaconda's "Dlls" folder.
Anaconda also has a "pckgs" folder...I copied and pasted the entire "OpenCV 3.2" folder that I downloaded to the Anaconda "pckgs" folder.
Finally, Anaconda has a "Library" folder which has a "bin" subfolder. I pasted the "opencv_ffmpeg320.dll" and "opencv_ffmpeg320_64.dll" files to that folder.
After closing and restarting Spyder, the code worked. I'm not sure which of the three methods worked, and I'm too lazy to go back and figure it out. But it works so, cheers!
i might be late here but you can use this pip package to quickly generate images from videos. You can also get images using specific fps.
pip install videoToImages
then type the following command in terminal
videoToimages --videoFolder [pathToVideosFolder]
Example: videoToimages --videoFolder "c:/videos"
for specific output fps , set --fps 10 to any required value. --fps 1 means one image per one second of the video.
Full commands:
videoToimages --videoFolder "c:/videos"
videoToimages --videoFolder "c:/videos" --fps 10 --img_size (512, 512)
This code is simple and guarantees reliable execution.
# path of video file
video_path = "path/to/video.mp4"
# Open video file
video = cv2.VideoCapture(video_path)
# number of frames in video
frame_count = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
# Convert frame to image and save to file
for i in range(frame_count):
ret, frame = video.read()
if ret:
image_path = f"path/to/image_{i}.jpg"
cv2.imwrite(image_path, frame)
# Close video file
video.release()
There are several reasons to extract slides/frames from a video presentation, especially in the case of education or conference related videos. It allows you to access the study notes without watching the whole video.
I have faced this issue several times, so I decided to create a solution for it myself using python. I have made the code open-source, you can easily set up this tool and run it in few simple steps.
Refer to this for youtube video tutorial.
Steps on how to use this tool.
Clone this project video2pdfslides
Set up your environment by running "pip install -r requirements.txt"
Copy your video path
Run "python video2pdfslides.py <video_path>"
Boom! the pdf slides will be available in in output folder
Make notes and enjoy!