Develop Reference

Why is the last frame always excluded, when converting images into a video?

I have a folder with 225 pictures of maps. So, I compiled it to an mp4 file using imageio. Whether it's compiling 10 maps, 150, or all 225, the last picture is always not included in the video.
import os
from natsort import humansorted
import imageio
os.chdir(r'folder/path/')
filenames = humansorted((fn for fn in os.listdir('.') if fn.endswith('.png')))
with imageio.get_writer('earthquake_video.mp4', mode='I', fps=2) as writer:
for filename in filenames:
image = imageio.imread(filename)
writer.append_data(image)
writer.close()

For me, your code works fine for 10, 150, or even 225 images – as long as I open the resulting video in Windows Media Player. If I open the video in VLC media player, I get distorted playback, not only skipping the last frame. (I have numbers counting from 0 to 224, so every mistaken frame is noticed.) So, if you use VLC media player, your problem most likely is the one discussed in this StackOverflow Q&A.
On the imageio GitHub issue tracker, there's also this issue, linking to this other StackOverflow question, which seems to be same issue as you have. But, still, I think it's the afore-mentioned issue with the VLC media player.
Unfortunately, I couldn't get the workaround from the first linked Q&A working using the output_params from imageio, i.e. setting -framerate or -r. So, my workaround here would be to set up a desired fps (here: 2), and a fps for the actual playback (in VLC media player), e.g. 30, and then simply add as many identical frames as needed to fake the desired fps, i.e. 30 // 2 = 15 here.
Here's some code:
import os
import imageio
os.chdir(r'folder/path')
filenames = [fn for fn in os.listdir('.') if fn.endswith('.png')]
fps = 2 # Desired, "real" frames per second
fps_vlc = 30 # Frames per second needed for proper playback in VLC
with imageio.get_writer('earthquake_video.mp4', fps=fps_vlc) as writer:
for filename in filenames:
image = imageio.imread(filename)
for i in range(fps_vlc // fps):
writer.append_data(image)
writer.close()
The resulting video "looks" the same as before (in Windows Media player), but now, it's also properly played in VLC media player.
Even, if that's not YOUR actual problem, I guess this information will help other coming across your question, but actually suffering from the stated issue with the VLC media player.
----------------------------------------
System information
----------------------------------------
Platform: Windows-10-10.0.16299-SP0
Python: 3.9.1
PyCharm: 2021.1.1
imageio: 2.9.0
----------------------------------------

Using MoviePy to fix unfinalized .flv video

While recording my screen with OBS capture, I accumulated large quantity of videos that had been subject to a forced system shutdown, leaving them unfinalized. The videos were created using an .flv format, so when I play them in VLC Player they play flawlessly, however they are missing an end time (video length). Instead, the videos show the running time as they play, but maintain the 00:00 end time, despite the actual video playing for several minutes.
From my understanding, unlike .mp4 formatting, .flv formatted video should be able to be recovered if it has not been finalized (as in the case of my footage stopped by unexpected shutdowns). Since I have a large quantity of unfinalized, I need an automated solution to fix them.
Using MoviePy write_videofile
I attempted to fix the videos by using the MoviePy write_videofile command in the python shell with the directory set to the directory of the bad video:
from moviepy.editor import * #no error
vid = VideoFileClip("oldVideoName.flv") #no error
vid.write_videofile("corrected.mp4") #IndexError
The final line created breifly created a file "correctedTEMP_MPY_wvf_snd.mp3"(only 1KB, unplayable in Audacity), shorty before throwing an exception. I recieved a massive traceback with the final teir reading:
File "\Python37-32\lib\site-packages\moviepy\audio\io\readers.py", line 168, in get_frame
"Accessing time t=%.02f-%.02f seconds, "%(tt[0], tt[-1])+
IndexError: index 0 is out of bounds for axis 0 with size 0
I assumed that this was caused by a problem with an audio reader not accepting the supposed 00:00 timestamp as the length of the video.
Using MoviePy subclip
I attempted to see if there was a way that I could manually feed MoviePy the start and end timestamps, using the subclip method. I know the video is at least 4 seconds long, so I used that as a control test:
clip = vid.subclip("00:00:00", "00:00:05") #no error
clip.write_videofile("corrected.mp4") #OSError
The write_videofile method again threw an exception:
File "\Python37-32\lib\site-packages\moviepy\audio\io\readers.py", line 169, in get_frame
"with clip duration=%d seconds, "%self.duration)
OSError: Error in file oldVideoName.flv,
Accessing time t=0.00-0.04 seconds, with clip duration=0 seconds,
Even if this method were to work, I would need to find a way to automate the process of discovering the video end time.
Using OpenCV CAP_PROP_FRAME_COUNT
One possible solution to finding the end time (video length) is to use cv2, per this post.
import cv2 #no error
vid=cv2.VideoCapture("oldVideoName.flv") #no error
vid.get(cv2.CAP_PROP_FRAME_COUNT) #returns -5.534023222112865e+17
I was not expecting to receive a negative float for this value. Further tests reveal to me that this float does not correspond at all with the length of the video, as all unfinalized videos return the same float for this request. (Normal videos do return their length for this method call) This is useful to iterate over a directory identifying unfinalized videos.
Is using MoviePy to correct a large quantity of unfinalized videos a viable or even possible solution? Is it better to use cv2 (Python OpenCV) for solving this problem?

I was able to fix the video files using yamdi, an open source metadata injector for FLV files. After downloading and installing yamdi, I can use the following command to repair an .flv file named oldVideoName.flv:
yamdi -i oldVideoName.flv -o corrected.flv
The command leaves oldVideoName.flv untouched, and saves a repaired file as corrected.flv.

Getting video properties with Python without calling external software

[Update:] Yes, it is possible, now some 20 months later. See Update3 below! [/update]
Is that really impossible? All I could find were variants of calling FFmpeg (or other software). My current solution is shown below, but what I really would like to get for portability is a Python-only solution that doesn't require users to install additional software.
After all, I can easily play videos using PyQt's Phonon, yet I can't get simply things like dimension or duration of the video?
My solution uses ffmpy (http://ffmpy.readthedocs.io/en/latest/ffmpy.html ) which is a wrapper for FFmpeg and FFprobe (http://trac.ffmpeg.org/wiki/FFprobeTips). Smoother than other offerings, yet it still requires an additional FFmpeg installation.
import ffmpy, subprocess, json
ffprobe = ffmpy.FFprobe(global_options="-loglevel quiet -sexagesimal -of json -show_entries stream=width,height,duration -show_entries format=duration -select_streams v:0", inputs={"myvideo.mp4": None})
print("ffprobe.cmd:", ffprobe.cmd) # printout the resulting ffprobe shell command
stdout, stderr = ffprobe.run(stderr=subprocess.PIPE, stdout=subprocess.PIPE)
# std* is byte sequence, but json in Python 3.5.2 requires str
ff0string = str(stdout,'utf-8')
ffinfo = json.loads(ff0string)
print(json.dumps(ffinfo, indent=4)) # pretty print
print("Video Dimensions: {}x{}".format(ffinfo["streams"][0]["width"], ffinfo["streams"][0]["height"]))
print("Streams Duration:", ffinfo["streams"][0]["duration"])
print("Format Duration: ", ffinfo["format"]["duration"])
Results in output:
ffprobe.cmd: ffprobe -loglevel quiet -sexagesimal -of json -show_entries stream=width,height,duration -show_entries format=duration -select_streams v:0 -i myvideo.mp4
{
"streams": [
{
"duration": "0:00:32.033333",
"width": 1920,
"height": 1080
}
],
"programs": [],
"format": {
"duration": "0:00:32.064000"
}
}
Video Dimensions: 1920x1080
Streams Duration: 0:00:32.033333
Format Duration: 0:00:32.064000
UPDATE after several days of experimentation: The hachoire solution as proposed by Nick below does work, but will give you a lot of headaches, as the hachoire responses are too unpredictable. Not my choice.
With opencv coding couldn't be any easier:
import cv2
vid = cv2.VideoCapture( picfilename)
height = vid.get(cv2.CAP_PROP_FRAME_HEIGHT) # always 0 in Linux python3
width = vid.get(cv2.CAP_PROP_FRAME_WIDTH) # always 0 in Linux python3
print ("opencv: height:{} width:{}".format( height, width))
The problem is that it works well on Python2 but not on Py3. Quote: "IMPORTANT NOTE: MacOS and Linux packages do not support video related functionality (not compiled with FFmpeg)" (https://pypi.python.org/pypi/opencv-python).
On top of this it seems that opencv needs the presence of the binary packages of FFmeg at runtime (https://docs.opencv.org/3.3.1/d0/da7/videoio_overview.html).
Well, if I need an installation of FFmpeg anyway, I can stick to my original ffmpy example shown above :-/
Thanks for the help.
UPDATE2: master_q (see below) proposed MediaInfo. While this failed to work on my Linux system (see my comments), the alternative of using pymediainfo, a py wrapper to MediaInfo, did work. It is simple to use, but it takes 4 times longer than my initial ffprobe approach to obtain duration, width and height, and still needs external software, i.e. MediaInfo:
from pymediainfo import MediaInfo
media_info = MediaInfo.parse("myvideofile")
for track in media_info.tracks:
if track.track_type == 'Video':
print("duration (millisec):", track.duration)
print("width, height:", track.width, track.height)
UPDATE3: OpenCV is finally available for Python3, and is claimed to run on Linux, Win, and Mac! It makes it really easy, and I verfied that external software - in particular ffmpeg - is NOT needed!
First install OpenCV via Pip:
pip install opencv-python
Run in Python:
import cv2
cv2video = cv2.VideoCapture( videofilename)
height = cv2video.get(cv2.CAP_PROP_FRAME_HEIGHT)
width = cv2video.get(cv2.CAP_PROP_FRAME_WIDTH)
print ("Video Dimension: height:{} width:{}".format( height, width))
framecount = cv2video.get(cv2.CAP_PROP_FRAME_COUNT )
frames_per_sec = cv2video.get(cv2.CAP_PROP_FPS)
print("Video duration (sec):", framecount / frames_per_sec)
# equally easy to get this info from images
cv2image = cv2.imread(imagefilename, flags=cv2.IMREAD_COLOR )
height, width, channel = cv2image.shape
print ("Image Dimension: height:{} width:{}".format( height, width))
I also needed the first frame of a video as an image, and used ffmpeg for this to save the image in the file system. This also is easier with OpenCV:
hasFrames, cv2image = cv2video.read() # reads 1st frame
cv2.imwrite("myfilename.png", cv2image) # extension defines image type
But even better, as I need the image only in memory for use in the PyQt5 toolkit, I can directly read the cv2-image into an Qt-image:
bytesPerLine = 3 * width
# my_qt_image = QImage(cv2image, width, height, bytesPerLine, QImage.Format_RGB888) # may give false colors!
my_qt_image = QImage(cv2image.data, width, height, bytesPerLine, QImage.Format_RGB888).rgbSwapped() # correct colors on my systems
As OpenCV is a huge program, I was concerned about timing. Turned out, OpenCV was never behind the alternatives. I takes some 100ms to read a slide, all the rest combined takes never more than 10ms.
I tested this successfully on Ubuntu Mate 16.04, 18.04, and 19.04, and on two different installations of Windows 10 Pro. (Did not have Mac avalable). I am really delighted about OpenCV!
You can see it in action in my SlideSorter program, which allows to sort images and videos, preserve sort order, and present as slideshow. Available here: https://sourceforge.net/projects/slidesorter/

OK, after investigating this myself because I needed it too, it looks like it can be done with hachoir. Here's a code snippet that can give you all the metadata hachoir can read:
import re
from hachoir.parser import createParser
from hachoir.metadata import extractMetadata
def get_video_metadata(path):
"""
Given a path, returns a dictionary of the video's metadata, as parsed by hachoir.
Keys vary by exact filetype, but for an MP4 file on my machine,
I get the following keys (inside of "Common" subdict):
"Duration", "Image width", "Image height", "Creation date",
"Last modification", "MIME type", "Endianness"
Dict is nested - common keys are inside of a subdict "Common",
which will always exist, but some keys *may* be inside of
video/audio specific stream subdicts, named "Video Stream #1"
or "Audio Stream #1", etc. Not all formats result in this
separation.
:param path: str path to video file
:return: dict of video metadata
"""
if not os.path.exists(path):
raise ValueError("Provided path to video ({}) does not exist".format(path))
parser = createParser(path)
if not parser:
raise RuntimeError("Unable to get metadata from video file")
with parser:
metadata = extractMetadata(parser)
if not metadata:
raise RuntimeError("Unable to get metadata from video file")
metadata_dict = {}
line_matcher = re.compile("-\s(?P<key>.+):\s(?P<value>.+)")
group_key = None # group_key stores which group we're currently in for nesting subkeys
for line in metadata.exportPlaintext(): # this is what hachoir offers for dumping readable information
parts = line_matcher.match(line) #
if not parts: # not all lines have metadata - at least one is a header
if line == "Metadata:": # if it's the generic header, set it to "Common: to match items with multiple streams, so there's always a Common key
group_key = "Common"
else:
group_key = line[:-1] # strip off the trailing colon of the group header and set it to be the current group we add other keys into
metadata_dict[group_key] = {} # initialize the group
continue
if group_key: # if we're inside of a group, then nest this key inside it
metadata_dict[group_key][parts.group("key")] = parts.group("value")
else: # otherwise, put it in the root of the dict
metadata_dict[parts.group("key")] = parts.group("value")
return metadata_dict
This seems to return good results for me right now and requires no extra installs. The keys seem to vary a decent amount by video and type of video, so you'll need to do some checking and not just assume any particular key is there. This code is written for Python 3 and is using hachoir3 and adapted from hachoir3 documentation - I haven't investigated if it works for hachoir for Python 2.
In case it's useful, I also have the following for turning the text-based duration values into seconds:
def length(duration_value):
time_split = re.match("(?P<hours>\d+\shrs)?\s*(?P<minutes>\d+\smin)?\s*(?P<seconds>\d+\ssec)?\s*(?P<ms>\d+\sms)", duration_value) # get the individual time components
fields_and_multipliers = { # multipliers to convert each value to seconds
"hours": 3600,
"minutes": 60,
"seconds": 1,
"ms": 1
}
total_time = 0
for group in fields_and_multipliers: # iterate through each portion of time, multiply until it's in seconds and add to total
if time_split.group(group) is not None: # not all groups will be defined for all videos (eg: "hrs" may be missing)
total_time += float(time_split.group(group).split(" ")[0]) * fields_and_multipliers[group] # get the number from the match and multiply it to make seconds
return total_time

Mediainfo is another choice. cross platform together with MediaInfoDLL.py and Mediainfo.DLL library
Download Mediainfo.dll from their site, CLI package to get DLL or both files including python script from https://github.com/MediaArea/MediaInfoLib/releases
working in python 3.6:
you create dict of parameters you want, keys have to be exact but values will be defined later, it is just to be clear what the value might be
from MediaInfoDLL import *
# could be in __init__ of some class
self.video = {'Format': 'AVC', 'Width': '1920', 'Height':'1080', 'ScanType':'Progressive', 'ScanOrder': 'None', 'FrameRate': '29.970',
'FrameRate_Num': '','FrameRate_Den': '','FrameRate_Mode': '', 'FrameRate_Minimum': '', 'FrameRate_Maximum': '',
'DisplayAspectRatio/String': '16:9', 'ColorSpace': 'YUV','ChromaSubsampling': '4:2:0', 'BitDepth': '8',
'Duration': '', 'Duration/String3': ''}
self.audio = {'Format': 'AAC', 'BitRate': '320000', 'BitRate_Mode': 'CBR', 'Channel(s)': '2', 'SamplingRate': '48000', 'BitDepth': '16'}
#a method within a class:
def mediainfo(self, file):
MI = MediaInfo()
MI.Open(file)
for key in self.video:
value = MI.Get(Stream.Video, 0, key)
self.video[key] = value
for key in self.audio:
# 0 means track 0
value = MI.Get(Stream.Audio, 0, key)
self.audio[key] = value
MI.Close()
.
.
#calling it from another method:
self.mediainfo(self.file)
.
# you'll get a dict with correct values, if none then value is ''
# for example to get frame rate out of that dictionary:
fps = self.video['FrameRate']

Piped FFMPEG won't write frames correctly

I am using Python's Image module to load JPEGs and modify them. After I have a modified image, I want to load that image in to a video, using more modified images as frames in my video.
I have 3 programs written to do this:
ImEdit (My image editing module that I wrote)
VideoWriter (writes to an mp4 file using FFMPEG) and
VideoMaker (The file I'm using to do everything)
My VideoWriter looks like this...
import subprocess as sp
import os
import Image
FFMPEG_BIN = "ffmpeg"
class VideoWriter():
def __init__(self,xsize=480,ysize=360,FPS=29,
outDir=None,outFile=None):
if outDir is None:
print("No specified output directory. Using default.")
outDir = "./VideoOut"
if outFile is None:
print("No specified output file. Setting temporary.")
outFile = "temp.mp4"
if (outDir and outFile) is True:
if os.path.exists(outDir+outFile):
print("File path",outDir+outFile, "already exists:",
"change output filename or",
"overwriting will occur.")
self.outDir = outDir
self.outFile = outFile
self.xsize,self.ysize,self.FPS = xsize,ysize,FPS
self.buildWriter()
def setOutFile(self,fileName):
self.outFile = filename
def setOutDir(self,dirName):
self.outDir = dirName
def buildWriter(self):
commandWriter = [FFMPEG_BIN,
'-y',
'-f', 'rawvideo',
'-vcodec','mjpeg',
'-s', '480x360',#.format(480,
'-i', '-',
'-an', #No audio
'-r', str(29),
'./{}//{}'.format(self.outDir,self.outFile)]
self.pW = sp.Popen(commandWriter,
stdin = sp.PIPE)
def writeFrame(self,ImEditObj):
stringData = ImEditObj.getIm().tostring()
im = Image.fromstring("RGB",(309,424),stringData)
im.save(self.pW.stdin, "JPEG")
self.pW.stdin.flush()
def finish(self):
self.pW.communicate()
self.pW.stdin.close()
ImEditObj.getIm() returns an instance of a Python Image object
This code works to the extent that I can load one frame in to the video and no matter how many more calls to writeFrame that I do, the video only every ends up being one frame long. I have other code that works as far as making a video out of single frames and that code is nearly identical to this code. I don't know what difference there is though that makes this code not work as intended where the other code does work.
My question is...
How can I modify my VideoWriter class so that I can pass in an instance of an Python's Image object and write that frame to an output file? I also would like to be able to write more than one frame to the video.
I've spent 5 hours or more trying to debug this, having not found anything helpful on the internet, so if I missed any StackOverflow questions that would point me in the right direction, those would be appreciated...
EDIT:
After a bit more debugging, the issue may have been that I was trying to write to a file that already existed, however, this doesn't make much sense with the -y flag in my commandWriter. the -y flag should overwrite any file that already exists. Any thoughts on that?

I suggest that you follow the OpenCV tutorial in writing videos. This is a very common way of writing video files from Python, so you should find many answers on the internet, if you can't get certain things to work.
Note that the VideoWriter will discard (and won't write) any frames that are not in the exact same pixel size that you give it on initialization.

having cv2.imread reading images from file objects or memory-stream-like data (here non-extracted tar)

I have a .tar file containing several hundreds of pictures (.png). I need to process them via opencv.
I am wondering whether - for efficiency reasons - it is possible to process them without passing by the disc. In other, words I want to read the pictures from the memory stream related to the tar file.
Consider for instance
import tarfile
import cv2
tar0 = tarfile.open('mytar.tar')
im = cv2.imread( tar0.extractfile('fname.png').read() )
The last line doesn't work as imread expects a file name rather than a stream.
Consider that this way of reading directly from the tar stream can be achieved e.g. for text (see e.g. this SO question).
Any suggestion to open the stream with the correct png encoding?
Untarring to ramdisk is of course an option, although I was looking for something more cachable.

Thanks to the suggestion of #abarry and this SO answer I managed to find the answer.
Consider the following
def get_np_array_from_tar_object(tar_extractfl):
'''converts a buffer from a tar file in np.array'''
return np.asarray(
bytearray(tar_extractfl.read())
, dtype=np.uint8)
tar0 = tarfile.open('mytar.tar')
im0 = cv2.imdecode(
get_np_array_from_tar_object(tar0.extractfile('fname.png'))
, 0 )

Perhaps use imdecode with a buffer coming out of the tar file? I haven't tried it but seems promising.