Automated Concatenate with FFMPEG Using Re-Encode - python

I would like to concaternate a fairly large list of videos using FFMPEG.
The following answer from another question is highly relevant:
https://stackoverflow.com/a/11175851/5065462.
I explain below why it isn't sufficient for me.
I'm using MP4 files.
I am using Windows 10. I'd like to write a bat file with arguments. I'll then call this from either Command Prompt or PowerShell.
I want to automate Option 1 https://stackoverflow.com/a/11175851/5065462 to take as input a txt (or similar) file containing the filepaths for the videos to be concatenated.
I'm happy with all the default [#:v] [#:a] options.
An alternative option is just to write a small program, either in Command Prompt or python3 is fine, which outputs a text string that I just copy+paste into cmd/PS.
Unfortunately, I'm not sure how to use python to get filenames.
Option 2 in https://stackoverflow.com/a/11175851/5065462 looks great. Unfortunatley, the stream-encoding has issues with my mp4 files. I found they are fixed by using Option 1 in the linked answer. However, I don't want to type every filename each time.

The following python script will generate the command described in option 1.
To run, use python3 script_file.py video_directory output.mkv.
video_directory should contain only video files.
import argparse
import os
parser = argparse.ArgumentParser()
parser.add_argument("dir", help="the directory to the videos")
parser.add_argument("output", help="the name of the output file")
args = parser.parse_args()
files = os.listdir(args.dir)
cmd = "ffmpeg "
for file in files:
cmd += f"-i {file} "
cmd += '-filter_complex "'
for i in range(len(files)):
cmd += f"[{i}:v] [{i}:a] "
cmd += f'concat=n={len(files)}:v=1:a=1 [v] [a]" '
cmd += f'-map "[v]" -map "[a]" {args.output}'
print(cmd)
NB: replacing the final line by os.system(cmd) will run the command directly from python.

Related

Subprocess.call ffmpeg making empty file

I am trying to convert video from one format to other.
extension = '.avi'
extension_less_url = '../uploads/video'
import subprocess
subprocess.call(['ffmpeg', '-i', extension_less_url + extension, extension_less_url + '.mp4'])
But above produces an empty file named 'video.mp4'.
How to correct?
This is the error I am getting:
The encoder 'aac' is experimental but experimental codecs are not enabled, add '-strict -2' if you want to use it.
This problem is for '.mp4'. try something else like '.mkv' for the target format of your video.
I've done this and the problem is solved
import subprocess
subprocess.call('ffmpeg -f vfwcap -t 10 -r 25 -i 0 c:/test/sample11.avi')
python have a subprocess module which helps for running exec command.
In case if you want to run multiple cmd or shell commands one after another you can use the following :
subprocess.check_call("cd /home/pi/images; ls", shell=True)
here : "cd /home/pi/images" is one command which is separated using ";" to run another command "ls"

How to escape a spacebar in a path name with subprocess?

I'm trying to convert a file from .m4a to .mp3 using ffmpeg and I need to access to the music folder.
The path name of this folder is : C:\\Users\A B\Desktop\Music
I can't access it with subprocess.call() because only C:\\Users\A gets recognized. The white space is not processed.
Here's my python script :
import constants
import os
import subprocess
path = 'C:\\Users\A B\Desktop\Music'
def main():
files = sorted(os.listdir(path), key=lambda x: os.path.getctime(os.path.join(path, x)))
if "Thumbs.db" in files: files.remove("Thumbs.db")
for f in files:
if f.lower()[-3:] == "m4a":
process(f)
def process(f):
inFile = f
outFile = f[:-3] + "mp3"
subprocess.call('ffmpeg -i {} {} {}'.format('C:\\Users\A B\Desktop\Music', inFile, outFile))
main()
When I run it I get an error that states :
C:\Users\A: No such file or directory
I wonder if someones knows how to put my full path name (C:\Users\A B\Desktop\Music) in subprocess.call() ?
Beforehand edit: spaces or not, the following command line -i <directory> <infilename> <outfilename> is not correct for ffmpeg since it expects the -i option, then input file and output file, not a directory first. So you have more than one problem here (which explains the "permission denied" message you had, because ffmpeg was trying to open a directory as a file!)
I suppose that you want to:
read all files from directory
convert them all to a file located in the same directory
In that case, you could add quotes to your both input & output absolute files like this:
subprocess.call('ffmpeg -i "{0}\{1}" "{0}\{2}"'.format('C:\\Users\A B\Desktop\Music', inFile, outFile))
That would work, but that's not the best thing to do: not very performant, using format when you already have all the arguments already, you may not have knowledge of other characters to escape, etc... don't reinvent the wheel.
The best way to do it is to pass the arguments in a list so subprocess module handles the quoting/escaping when necessary:
path = r'C:\Users\A B\Desktop\Music' # use raw prefix to avoid backslash escaping
subprocess.call(['ffmpeg','-i',os.path.join(path,inFile), os.path.join(path,outFile)])
Aside: if you're the user in question, it's even better to do:
path = os.getenv("USERPROFILE"),'Desktop','Music'
and you could even run the process in the path directory with cwd option:
subprocess.call(['ffmpeg','-i',inFile, outFile],cwd=path)
and if you're not, be sure to run the script with elevated privileges or you won't get access to another user directory (read-protected)

Python script doesn't delete file from archive -- printed command via terminal works fine

I'm creating an archive in Python using this code:
#Creates archive using string like [proxy_16-08-15_08.57.07.tar]
proxyArchiveLabel = 'proxy_%s' % EXECUTION_START_TIME + '.tar'
log.info('Packaging %s ...' % proxyArchiveLabel)
#Removes .tar from label during creation
shutil.make_archive(proxyArchiveLabel.rsplit('.',1)[0], 'tar', verbose=True)
So this creates an archive fine in the local directory. The problem is, there's a specific directory in my archive I want to remove, due to it's size and lack of necessity for this task.
ExecWithLogging('tar -vf %s --delete ./roles/jobs/*' % proxyArchiveLabel)
# ------------
def ExecWithLogging(cmd):
print cmd
p = subprocess.Popen(cmd.split(' '), env=os.environ, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while(True):
log.info(p.stdout.readline().strip())
if(p.poll() is not None):
break
However, this seems to do basically nothing. The size remains the same. If I print cmd inside of the ExecWithLogging, and copy/past that command to a terminal in the working directory of the script, it works fine. Just to be sure, I also tried hard-coding the full path to where the archive is created as part of the tar -vf %s --delete command, but still nothing seemed to happen.
I do get this output in my INFO log: tar: Pattern matching characters used in file names, so I'm kind of thinking Popen is interpreting my command incorrectly somehow... (or rather, I'm more likely passing in something incorrectly).
Am I doing something wrong? What else can I try?
You may have to use the --wildcards option in the tar command, which enables pattern matching. This may well be what you are seeing in your log, be it somewhat cryptically.
Edit:
In answer to your question Why? I suspect that the shell is performing the wildcard expansion whilst the command proffered through Popen is not. The --wildcard option for tar, forces tar to perform the wildcard expansion.
For a more detailed explanation see here:
Tar and wildcards

Python - glob.glob doesn't find *.txt in specified filepath within Unix OS

I am converting some Python scripts I wrote in a Windows environment to run in Unix (Red Hat 5.4), and I'm having trouble converting the lines that deal with filepaths. In Windows, I usually read in all .txt files within a directory using something like:
pathtotxt = "C:\\Text Data\\EJC\\Philosophical Transactions 1665-1678\\*\\*.txt"
for file in glob.glob(pathtotxt):
It seems one can use the glob.glob() method in Unix as well, so I'm trying to implement this method to find all text files within a directory entitled "source" using the following code:
#!/usr/bin/env python
import commands
import sys
import glob
import os
testout = open('testoutput.txt', 'w')
numbers = [1,2,3]
for number in numbers:
testout.write(str(number + 1) + "\r\n")
testout.close
sourceout = open('sourceoutput.txt', 'w')
pathtosource = "/afs/crc.nd.edu/user/d/dduhaime/data/hill/source/*.txt"
for file in glob.glob(pathtosource):
with open(file, 'r') as openfile:
readfile = openfile.read()
souceout.write (str(readfile))
sourceout.close
When I run this code, the testout.txt file comes out as expected, but the sourceout.txt file is empty. I thought the problem might be solved if I change the line
pathtosource = "/afs/crc.nd.edu/user/d/dduhaime/data/hill/source/*.txt"
to
pathtosource = "/source/*.txt"
and then run the code from the /hill directory, but that didn't resolve my problem. Do others know how I might be able to read in the text files in the source directory? I would be grateful for any insights others can offer.
EDIT: In case it is relevant, the /afs/ tree of directories referenced above is located on a remote server that I'm ssh-ing into via Putty. I'm also using a test.job file to qsub the Python script above. (This is all to prepare myself to submit jobs on the SGE cluster system.) The test.job script looks like:
#!/bin/csh
#$ -M dduhaime#nd.edu
#$ -m abe
#$ -r y
#$ -o tmp.out
#$ -e tmp.err
module load python/2.7.3
echo "Start - `date`"
python tmp.py
echo "Finish - `date`"
Got it! I had misspelled the output command. I wrote
souceout.write (str(readfile))
instead of
sourceout.write (str(readfile))
What a dunce. I also added a newline bit to the line:
sourceout.write (str(readfile) + "\r\n")
and it works fine. I think it's time for a new IDE!
You haven't really closed the file. The function testout.close() isn't called, because you have forgotten the parentheses. The same is for sourceout.close()
testout.close
...
sourceout.close
Has to be:
testout.close()
...
sourceout.close()
If the program finishes all files are automatically closed so it is only important if you reopen the file.
Even better (the pythonic version) would be to use the with statement. Instead of this:
testout = open('testoutput.txt', 'w')
numbers = [1,2,3]
for number in numbers:
testout.write(str(number + 1) + "\r\n")
testout.close()
you would write this:
with open('testoutput.txt', 'w') as testout:
numbers = [1,2,3]
for number in numbers:
testout.write(str(number + 1) + "\r\n")
In this case the file will be automatically closed even when an error occurs.

os.system commands

I am developing a program which needs to use os.system because of the old Python limitations. Currently I'm stuck in one small spot.
os.system("C:\\FIOCheck\\xutil.exe -i get phy" +HBEA + ">C:\\FIOCheck\\HBEAResult.txt")
This is the piece of code I am trying to work through. It will access an external program, which has some parameters. HBEA is the variable I am trying to pass (which is received earlier in the program). The program then takes whatever the .exe created and pipes it to an external file. At this point, the variable HBEA is not being passed to the command line, so the .exe never runs, which causes the .txt to be blank. Since the file is blank, I cannot grab data from it and therefore cannot complete the program.
Any ideas?
EDIT:
So I attempted the following code per some suggestions:
cmd = "C:\\FIOCheck\\xutil.exe -i get phy " +HBEA + ">C:\\FIOCheck\\HBEAResult.txt"
print cmd
os.system(cmd)
The following output was generated:
50012BE00004BDFF #HBEA variable
C:\FIOCheck\xutil.exe -i get phy 50012BE00004BDFF>C:\FIOCheck\HBEAResult.txt #the cmd var
However this still isn't passing the value through. Is the HBEA variable too long?
SOLVED
This program worked with some editing from the best answer. The commands were passing correctly, however the way I formatted it was not correct. The new formatting looks like:
cmd = "C:\\FIOCheck\\xutil.exe -i " + HBEA + " get ver" + ">C:\\FIOCheck\\HBEAResult.txt"
os.system(cmd)
Thanks for the help!
os.system("C:\\FIOCheck\\xutil.exe -i get phy" +HBEA + ">C:\\FIOCheck\\HBEAResult.txt")
should that be
os.system("C:\\FIOCheck\\xutil.exe -i get phy " +HBEA + ">C:\\FIOCheck\\HBEAResult.txt")
and you can always build the string first
cmd = "C:\\FIOCheck\\xutil.exe -i get phy " +HBEA + ">C:\\FIOCheck\\HBEAResult.txt"
print cmd
os.system(cmd)

Categories

Resources