Develop Reference

I cannot convert a set of images taken from a FTP to a video using FFMPEG

I am writting a python code where I am trying to convert a set of images that I take from a FTP into a video using FFMPEG but I cannot. I have tried, instead of reading the folder where the images are, to read a txt file with the name of the images that I want to use, with the format needed in order that FFMPEG could read it propertly, but I get always the same error: Protocol 'ftp' not on whitelist 'tcp'
In the same code, I also try to change the format of one video and change the resolution and size, and this part of code works well.
However, writting as input the same reference of the FTP, the images' code fail and the video's code works.
Besides, I have tried in my terminal as local the same command I write in the code for the images, and in local it works propertly, but not in the code.
Here there is a part of my code:
Video's code (it works):
command = """ffmpeg -i {i} -an -crf {r} {o}""".format(i=src_path,o=path,r=resolution)
An example of this command when I run this is the next (I dont want to write the exact ip and port):
ffmpeg -i ftp://user:user#ip:port/landing_ffmpeg/pruebas/pruebahd.mp4 -an -crf 45 tmp/pruebasalida456.mp4
And next the images' code (it doesnt work):
command = """ffmpeg -loop 1 -framerate {ips} -i {i} -t 10 -pix_fmt yuv420p {o}""".format(i=src_path,o=path,ips=img_per_sec)
An example of this command is the next:
ffmpeg -loop 1 -framerate 2 -i ftp://user:user#ip:port/landing_ffmpeg/pruebas/prueba_imagenes/prueba06.jpg -t 10 -pix_fmt yuv420p tmp/videoimagen.mp4
And the error I get with this code is the next:
[ftp # 0x560eb3e11800] Protocol 'ftp' not on whitelist 'tcp'!
[image2 # 0x560eb3e09380] Could not open file : ftp://user:user#ip:port/landing_ffmpeg/pruebas/prueba_imagenes/prueba06.jpg
I dont get this error when I try to run the command of the video, only for the images. And both commands run propertly when I write in my terminal in local, with local paths.
I would appreciate if someone can help me to solve the problem and fix my code.
Thanks!

The error is saying it all. Try to whitelist the ftp:
ffmpeg -protocol_whitelist ftp -loop 1 -framerate 2 \
-i ftp://user:user#ip:port/landing_ffmpeg/pruebas/prueba_imagenes/prueba06.jpg \
-t 10 -pix_fmt yuv420p tmp/videoimagen.mp4

run bash script for each files in folder

i found this script but it need input filename and output filename to work
i'm windows user so i dont know how to run this script for each files in folder
what i want is :
sourcefile=$1 -> this should be input directory
destfile=$2 -> output directory or just originalfilename_preview
so when i'm try to excute script, it will run through files in input directory and excute two ffmpeg script inside
the first ffmpeg script will split video into multiple files in temp folder
the second ffmpeg merge those files in temp folder and complete the whole process with output folder or originalfilename_preview
-> loop for next files until completed
sourcefile=$1
destfile=$2
# Overly simple validation
if [ ! -e "$sourcefile" ]; then
echo 'Please provide an existing input file.'
exit
fi
if [ "$destfile" == "" ]; then
echo 'Please provide an output preview file name.'
exit
fi
# Get video length in seconds
length=$(ffprobe $sourcefile -show_format 2>&1 | sed -n 's/duration=//p' | awk '{print int($0)}')
# Start 20 seconds into the video to avoid opening credits (arbitrary)
starttimeseconds=20
# Mini-snippets will be 2 seconds in length
snippetlengthinseconds=2
# We'll aim for 5 snippets spread throughout the video
desiredsnippets=5
# Ensure the video is long enough to even bother previewing
minlength=$(($snippetlengthinseconds*$desiredsnippets))
# Video dimensions (these could probably be command line arguments)
dimensions=640:-1
# Temporary directory and text file where we'll store snippets
# These will be cleaned up and removed when the preview image is generated
tempdir=snippets
listfile=list.txt
# Display and check video length
echo 'Video length: ' $length
if [ "$length" -lt "$minlength" ]
then
echo 'Video is too short. Exiting.'
exit
fi
# Loop and generate video snippets
mkdir $tempdir
interval=$(($length/$desiredsnippets-$starttimeseconds))
for i in $(seq 1 $desiredsnippets)
do
# Format the second marks into hh:mm:ss format
start=$(($(($i*$interval))+$starttimeseconds))
formattedstart=$(printf "%02d:%02d:%02d\n" $(($start/3600)) $(($start%3600/60)) $(($start%60)))
echo 'Generating preview part ' $i $formattedstart
# Generate the snippet at calculated time
ffmpeg -i $sourcefile -vf scale=$dimensions -preset fast -qmin 1 -qmax 1 -ss $formattedstart -t $snippetlengthinseconds -threads $(nproc) $tempdir/$i.mp4
done
# Concat videos
echo 'Generating final preview file'
# Generate a text file with one snippet video location per line
# (https://trac.ffmpeg.org/wiki/Concatenate)
for f in $tempdir/*; do echo "file '$f'" >> $listfile; done
# Concatenate the files based on the generated list
ffmpeg -f concat -safe 0 -i $listfile -threads $(nproc) -an -tune zerolatency -x264opts bitrate=2000:vbv-maxrate=2000:vbv-bufsize=166 -vcodec libx264 -f mpegts -muxrate 2000K -y $destfile.mp4
echo 'Done! Check ' $destfile '.mp4!'
# Cleanup
rm -rf $tempdir $listfile
source: https://davidwalsh.name/video-preview
#Christopher Hoffman
wsl already installed, of course, i'm already run this script without problem but i need to manual input/output filename
./preview.sh input.mp4 out
#Renaud Pacalet
yes, all files in input directory or drag&drop files (but all files
in directory seem like easier)
i think modify script
output file have suffix in name "_preview"
if it have suffix, same input folder is ok,
video files (mkv,mp4,avi,..)
some files name have unicode character so i think input file will
inside " "

The easiest is probably to keep the script as it is and to use a bash loop to process all files in the input directory. Let's assume:
the input directory is /my/video/files,
you want to store all outputs in directory /some/where,
the script you show is in /else/where/myscript.sh,
you want to process all files in the input directory.
Just open a terminal where bash is the interactive shell and type:
shopt -s nullglob
chmod +x /else/where/myscript.sh
mkdir -p /some/where
cd /my/video/files
for f in *; do
/else/where/myscript.sh "$f" "/some/where/$f"
done
shopt -u nullglob
Explanations:
shopt -s nullglob enables the nullglob option. Without this, if there are no files at all in the input directory, there would still be one iteration of the loop with f=*. shopt -u nullglob disables it when we are done.
chmod +x /else/where/myscript.sh makes your script executable, just in case it was not already.
mkdir -p /some/where creates the output directory, just in case it did not exist yet.
cd /my/video/files changes the current directory to the input directory in which you have your video files.
for f in *; do loops over all files in the current directory (this is what the * stands for). In each iteration variable f is assigned the current file name.
/else/where/myscript.sh "$f" "/some/where/$f" executes your script with two parameters: the name of the input file and the name of the output file, both quoted with double quotes to prevent word splitting.
Note: if all files are not video files you can be more specific:
for f in *.mkv *.mp4 *.avi; do
...
Of course, for easier reuse, you can also create a new shell script file with all this.

Video File Size Optimization

I'm trying the 2-pass technique of the FFmpeg in python but couldn't find any python tutorials do this task.
is there is no way instead of using Subprocess? if there's any illustrative example please provide me.
Note:
I have tried the 2-pass in the script like that:
input_fit = {self.video_in:None}
output = {None:"-c:v h264 -b:v 260k -pass 1 -an -f mp4 NUL && ^",
self.video_out:("ffmpeg -i \"%s\" -c:v h264 -b:v 260k -pass 2 " %self.video_in)}
## video_out IS The Name of The output File ##
model = FFmpeg(inputs = input_fit, outputs= output)
print(model.cmd)
It Raises an error of
:: FFRuntimeError: exited with status 1,
but when i take the generated command and run it on the ffmpeg cmd it runs without errors and generates the video perfectly.
so anyone just could tell me what is the problem please?

Using wildcards in subprocess.Popen

I have found a couple answers that solve the problem of passing wildcards through Popen, but I can't seem to get those solutions to work for my particular case.
I have a project that will merge an audio and video file when a button is pressed, and I need to use subprocess.Popen to execute the command I want:
mergeFile = "ffmpeg -i /home/pi/Video/* -i /home/pi/Audio/test.wav -acodec copy -vcodec copymap 0:v -map 1:a /home/pi/Test/output.mkv"
proc= subprocess.Popen(shlex.split(mergeFiles), shell=True)
I basically want to take whatever file is in my Video folder (my project downloads videos from a camera and the name of the file changes every time a new video is saved) and merge the audio from a separate "Audio" folder and then save that file to yet another folder.
I have tried setting the shell to true, and nothing works.
I also don't know how the glob module would help me in this scenario.

The problem here is the shlex.split(). When used in conjunction with shell=True, it means that only the string ffmpeg is treated as a script, and that the other components of your command line are passed as arguments to that script (which it never looks at / reads).
mergeFile = "ffmpeg -i /home/pi/Video/* -i /home/pi/Audio/test.wav -acodec copy -vcodec copymap 0:v -map 1:a /home/pi/Test/output.mkv"
proc = subprocess.Popen(mergeFile, shell=True)
A better-practice alternative that still uses shell=True (if you're actually parameterizing the directory names and filenames) might be:
mergeFile=[
'ffmpeg -i "$1"/* -i "$2" -acodec copy -vcodec copymap 0:v -map 1:a "$3"',
'_', # placeholder for $0
"/home/pi/Video", # directory for $1 -- can use a variable here
"/home/pi/Audio/test.wav",
"/home/pi/Test/output.mkv",
]
subprocess.Popen(mergeFile, shell=True)
...in which case the script itself is constant (and can't have its meaning changed by values injected via filenames or other parameters), but out-of-band data can be provided.
Even better than that is to stop using shell=True altogether. Consider:
import subprocess, glob
mergeFile=[
'ffmpeg', '-i',
] + (glob.glob('/home/pi/Video/*') or ['/home/pi/Video/*']) + [
'-i', '/home/pi/Audio/test.wav',
'-acodec', 'copy',
'-vcodec', 'copymap', '0:v',
'-map', '1:a1',
'/home/pi/Test/output.mkv'
]
subprocess.Popen(mergefile)
The or ['/home/pi/Video/*'] exists to cause the same error message you'd get with a shell if no files matching the glob exist. Obviously, you could just abort in that case as well.

Is Python 'sys.argv' limited in the maximum number of arguments?

I have a Python script that needs to process a large number of files. To get around Linux's relatively small limit on the number of arguments that can be passed to a command, I am using find -print0 with xargs -0.
I know another option would be to use Python's glob module, but that won't help when I have a more advanced find command, looking for modification times, etc.
When running my script on a large number of files, Python only accepts a subset of the arguments, a limitation I first thought was in argparse, but appears to be in sys.argv. I can't find any documentation on this. Is it a bug?
Here's a sample Python script illustrating the point:
import argparse
import sys
import os
parser = argparse.ArgumentParser()
parser.add_argument('input_files', nargs='+')
args = parser.parse_args(sys.argv[1:])
print 'pid:', os.getpid(), 'argv files', len(sys.argv[1:]), 'argparse files:', len(args.input_files)
I have a lot of files to run this on:
$ find ~/ -name "*" -print0 | xargs -0 ls > filelist
748709 filelist
But it appears xargs or Python is chunking my big list of files and processing it with several different Python runs:
$ find ~/ -name "*" -print0 | xargs -0 python test.py
pid: 4216 argv files 1819 number of files: 1819
pid: 4217 argv files 1845 number of files: 1845
pid: 4218 argv files 1845 number of files: 1845
pid: 4219 argv files 1845 number of files: 1845
pid: 4220 argv files 1845 number of files: 1845
pid: 4221 argv files 1845 number of files: 1845
...
Why are multiple processes being created to process the list? Why is it being chunked at all? I don't think there are newlines in the file names and shouldn't -print0 and -0 take care of that issue? If there were newlines, I'd expect sed -n '1810,1830p' filelist to show some weirdness for the above example. What gives?
I almost forgot:
$ python -V
Python 2.7.2+

xargs will chunk your arguments by default. Have a look at the --max-args and --max-chars options of xargs. Its man page also explains the limits (under --max-chars).

Python does not seem to place a limit on the number of arguments but the operating system does.
Have a look here for a more comprehensive discussion.

Everything that you want from find is available from os.walk.
Don't use find and the shell for any of this.
Use os.walk and write all your rules and filters in Python.
"looking for modification times" means that you'll be using os.stat or some similar library function.

xargs will pass as much as it can, but there's still a limit. For instance,
find ~/ -name "*" -print0 | xargs -0 wc -l | grep total
will give you multiple lines of output.
You probably want to have your script either take a file containing a list of filenames, or accept filenames on its stdin.

The problem is that xargs is limited by the number of chars of the calling arguments (maximum 2091281).
A quick test showed this ranges from 5000 files - 55000 files, depending on the length of the path.
The solution to get more is to accept piping in the file path through standard-in instead.
find ... -print0 | script.py
#!/usr/bin/env python3
import sys
files = sys.stdin.read().split('\0')
...