Develop Reference

How to pass variable to a subprocess.run in a Python script?

I have following the link:
to incorporate a command line command into my python script and it is working fine.
but i want o run the command over all the files present in a folder. How to send the file name to the command line? I think there should be some for loop but I cant hit the chord. Also I wnat to save the result in a .csv file.
import os
import subprocess
list_files = subprocess.run(["file","my_audio.wav"])
How to ?
for file_name in folder
output=subprocess.run(["file","file_name"])`
save output in .csv

Got the answer !
import os
import subprocess
import glob
with open("bit_rate.csv", "w") as fp:
for file_name in glob.glob('./*.wav'):
print(file_name)
subprocess.run(["file",file_name ], stdout=fp)

Moving all contents of a directory to another in Python

I've been trying to figure this out for hours with no luck. I have a list of directories that have subdirectories and other files of their own. I'm trying to traverse through all of them and move all of their content to a specific location. I tried shutil and glob but I couldn't get it to work. I even tried to run shell commands using subprocess.call and that also did not work either. I understand that it didn't work because I couldn't apply it properly but I couldn't find any solution that moves all contents of a directory to another.
files = glob.glob('Food101-AB/*/')
dest = 'Food-101/'
if not os.path.exists(dest):
os.makedirs(dest)
subprocess.call("mv Food101-AB/* Food-101/", shell=True)
# for child in files:
# shutil.move(child, dest)
I'm trying to move everything in Food101-AB to Food-101

shutil module of the standart library is the way to go:
>>> import shutil
>>> shutil.move("Food101-AB", "Food-101")
If you don't want to move Food101-AB folder itself, try using this:
import shutil
import os
for i in os.listdir("Food101-AB"):
shutil.move(os.path.join("Food101-AB", i), "Food-101")
For more information about move function:
https://docs.python.org/3/library/shutil.html#shutil.move

Try to change call function to run in order to retrieve the stdout, stderr and return code for your shell command:
from subprocess import run, CalledProcessError
source_dir = "full/path/to/src/folder"
dest_dir = "full/path/to/dest/folder"
try:
res = run(["mv", source_dir, dest_dir], check=True, capture_output=True)
except CalledProcessError as ex:
print(ex.stdout, ex.stderr, ex.returncode)

Passing a variable in shell/python

I'm using a python script where I'm using a shell command to copy from local to hdfs.
import os
import logging
import subprocess
filePath = "/tmp"
keyword = "BC10^Dummy-Segment"
for root, dirs, files in os.walk(filePath):
for file in files:
if keyword in file:
subprocess.call(["hadoop fs -copyFromLocal /tmp/BC10%5EDummy-Segment* /user/app"], shell=True)
subprocess.call(["hadoop fs -rm /tmp/BC10%5EDummy-Segment*"], shell=True)
I'm seeing this error:
copyFromLocal: `/tmp/BC10^Dummy-Segment*': No such file or directory
rm: `/tmp/BC10^Dummy-Segment_2019': No such file or directory
Updated code:
import glob
import subprocess
import os
from urllib import urlencode, quote_plus
filePath = "/tmp"
keyword = "BC10^Dummy-Segment"
wildcard = os.path.join(filePath, '{0}*'.format(keyword))
print(wildcard)
files = [urlencode(x, quote_via=quote_plus) for x in glob.glob(wildcard)]
subprocess.check_call(["hadoop", "fs", "-copyFromLocal"] + files + ["/user/app"])
#subprocess.check_call(["hadoop", "fs", "-rm"] + files)
Seeing error when I run:
Traceback (most recent call last):
File "ming.py", line 11, in <module>
files = [urlencode(x, quote_via=quote_plus) for x in glob.glob(wildcard)]
TypeError: urlencode() got an unexpected keyword argument 'quote_via'

I'm guessing you are URL-encoding the path to pass it properly to Hadoop, but in doing so you basically hide it from the shell. There really are no files matching the wildcard /tmp/BC10%5EDummy-Segment* where % etc are literal characters.
Try handling the glob from Python instead. With that, you can also get rid of that pesky shell=True; and with that change, it is finally actually correct and useful to pass the commands as a list of strings (never a list of a singe space-separated string, and with shell=True, don't pass a list at all). Notice also the switch to check_call so we trap errors and don't delete the source files if copying them failed. (See also https://stackoverflow.com/a/51950538/874188 for additional rationale.)
import glob
import subprocess
import os
from urllib import quote_plus
filePath = "/tmp"
keyword = "BC10^Dummy-Segment"
wildcard = os.path.join(filePath, '{0}*'.format(keyword))
files = [quote_plus(x) for x in glob.glob(wildcard)]
subprocess.check_call(["hadoop", "fs", "-copyFromLocal"] + files + ["/user/app"])
subprocess.check_call(["hadoop", "fs", "-rm"] + files)
This will not traverse subdirectories; but neither would your attempt with os.walk() do anything actually useful if it found files in subdirectories. If you actually want that to happen, please explain in more detail what the script should do.

How to get .avi files length

I am trying to loop over a directory of sub folders where every folder contains one .avi file that i want to retrieve its length in seconds.
I've found PyMedia http://pymedia.org/ and i understand it could possibly help me achieve this but i cannot find anything about avi duration / length in the documentation.
How would i be able to do that? also, if there is a different library of some sort i'd like to know aswel.
Edit: Added my final solution that works thanks to J.F. Sebastian
import sys
import glob
import os
from hachoir_core.cmd_line import unicodeFilename
from hachoir_core.i18n import getTerminalCharset
from hachoir_metadata import extractMetadata
from hachoir_parser import createParser
path = "z:\*"
for fpath in glob.glob(os.path.join(path, '*avi')):
filename = fpath
filename, real_filename = unicodeFilename(filename), filename
parser = createParser(filename, real_filename=real_filename)
metadata = extractMetadata(parser)
print fpath
print("Duration (hh:mm:ss.f): %s" % metadata.get('duration'))
print '\n'

You could use hachoir-metadata to extract avi duration from a file:
#!/usr/bin/env python
import sys
# $ pip install hachoir-{core,parser,metadata}
from hachoir_core.cmd_line import unicodeFilename
from hachoir_core.i18n import getTerminalCharset
from hachoir_metadata import extractMetadata
from hachoir_parser import createParser
filename = sys.argv[1]
charset = getTerminalCharset()
filename, real_filename = unicodeFilename(filename, charset), filename
parser = createParser(filename, real_filename=real_filename)
metadata = extractMetadata(parser)
print("Duration (hh:mm:ss.f): %s" % metadata.get('duration'))
It uses pure Python RIFF parser to extract info from avi file.
Example:
$ get-avi-duration.py test.avi
Duration (hh:mm:ss.f): 0:47:03.360000
Here's ffmpeg's output for comparison:
$ ffmpeg -i test.avi |& grep -i duration
Duration: 00:47:03.36, start: 0.000000, bitrate: 1038 kb/s
To print info about all avi files in a directory tree:
#!/usr/bin/env python
import os
import sys
from hachoir_metadata import extractMetadata
from hachoir_parser import createParser
def getinfo(rootdir, extensions=(".avi", ".mp4")):
if not isinstance(rootdir, unicode):
rootdir = rootdir.decode(sys.getfilesystemencoding())
for dirpath, dirs, files in os.walk(rootdir):
dirs.sort() # traverse directories in sorted order
files.sort()
for filename in files:
if filename.endswith(extensions):
path = os.path.join(dirpath, filename)
yield path, extractMetadata(createParser(path))
for path, metadata in getinfo(u"z:\\"):
if metadata.has('duration'):
print(path)
print(" Duration (hh:mm:ss.f): %s" % metadata.get('duration'))

If your server running any UNIX operation system you can use ffmpeg to do this. Usually just default command like ffmpeg myvideo.avi will give you full video details.
There's also a python wrapper for ffmpeg which probably will return video details in dictionary or list.
EDIT:
I've also found nice ffmpeg tool called ffprobe which can output length of video without additional fuss.
fprobe -loglevel error -show_streams inputFile.avi | grep duration | cut -f2 -d=

Not sure if there is a platform independent way to do this, but if you only need this to work on windows then it looks like MediaInfo (below) has a command line interface which you can use to output details about video files, which could then be parsed to get the information. Not the prettiest solution but looks like it should work.
http://mediainfo.sourceforge.net/en

Python MP3 ID3 Tag Editor

Re-organizing a large MP3 library for my friend's MP3 Player, I have the need to name the Title ID3 tag the same as the file name, and doing this via Windows Properties takes forever, so I was wondering if anyone has an idea of how to make a Python script that does this to all MP3's in a directory in rapid succession. Or at least a link to a library installable on Windows.

Look at this:
ID3 Tagging in Python
id3reader
Also Dive Into Python uses MP3 ID3 tags as an example.
Don't forget about PyPI - the Python Package Index.

Here is a python script I wrote to do this https://gitlab.com/tomleo/id3_folder_rename
#! /usr/bin/python
import os
import re
import glob
import subprocess
from mutagen.easyid3 import EasyID3
path = os.getcwd()
fpath = u"%s/*.mp3" % path
files = glob.glob(fpath)
for fname in files:
_track = EasyID3(fname)
track_num = _track.get('tracknumber')[0]
track_title = re.sub(r'/', '_', _track.get('title')[0])
if '/' in track_num:
track_num = track_num.split('/')[0]
if len(track_num) == 1:
track_num = "0%s" % track_num
_valid_fname = u"%s/%s %s.mp3" % (path, track_num, track_title)
if fname != _valid_fname:
subprocess.call(["/bin/mv", fname, _valid_fname])
It uses the mutagen python library for parsing the ID3 info. You'll have to tweak the subprocess call it to make it work with windows, but this should give you an idea for how to do it. Hope this helps.