I need to run a shell command inside subprocess.Popen in Python.
The command is:
$ virsh dumpxml server1 | grep 'source file' | awk -F\' '{print $2}'
The output is:
/vms/onion.qcow2
I'm having two challenges with the above command:
1) The command is inside a loop, and where you see 'server1', it is a variable that will have a server name.
2) Python is complaining about KeyError: 'print $2'
Here is what I have so far:
proc = subprocess.Popen(["virsh dumpxml {0} | grep 'source file' | awk -F\' '{print $2}'".format(vm)], stdout=subprocess.PIPE, shell=True)
stdout = proc.communicate()[0]
Thanks in advance.
While it's possible use libvirt directly from python, your problem is that { is the format string, and surrounds print $2 in your awk script as well, so you have to escape those braces like
proc = subprocess.Popen(["virsh dumpxml {0} | grep 'source file' | awk -F\\' '{{print $2}}'".format(vm)], stdout=subprocess.PIPE, shell=True)
stdout = proc.communicate()[0]
Related
I want to run the following lines of linux bash commands inside a python program.
tail /var/log/omxlog | stdbuf -o0 grep player_new | while read i
do
Values=$(omxd S | awk -F/ '{print $NF}')
x1="${Values}"
x7="${x1##*_}"
x8="${x7%.*}"
echo ${x8}
done
I know that for a single-line command, we can use the following syntax:
subprocess.call(['my','command'])
But, how can I use subprocess.call if there are several commands in multiple lines !?
quote https://mail.python.org/pipermail/tutor/2013-January/093474.html:
use subprocess.check_output(shell_command, shell=True)
import subprocess
cmd = '''
tail /var/log/omxlog | stdbuf -o0 grep player_new | while read i
do
Values=$(omxd S | awk -F/ '{print $NF}')
x1="${Values}"
x7="${x1##*_}"
x8="${x7%.*}"
echo ${x8}
done
'''
subprocess.check_output(cmd, shell=True)
I have try some other examples and it works.
Here is a pure python solution that I think does the same as your bash:
logname = '/var/log/omxlog'
with open(logname, 'rb') as f:
# not sure why you only want the last 10 lines, but here you go
lines = f.readlines()[-10:]
for line in lines:
if 'player_new' in line:
omxd = os.popen('omxd S').read()
after_ = omxd[line.rfind('_')+1:]
before_dot = after_[:after_.rfind('.')]
print(before_dot)
This question already has answers here:
How do I use subprocess.Popen to connect multiple processes by pipes?
(9 answers)
Closed 7 years ago.
Hi I am trying to run this command in python's subprocess with shlex split, however, I haven't found anything helpful for this particular case :
ifconfig | grep "inet " | grep -v 127.0.0.1 | grep -v 192.* | awk '{print $2}'
I get an with ifconfig error because the split with the single and double quotes and even the white space before the $ sign are not correct.
Please Help.
You can use shell=True (shell will interpret |) and triple quote string literal (otherwise you need to escape ", ' inside the string literal):
import subprocess
cmd = r"""ifconfig | grep "inet " | grep -v 127\.0\.0\.1 | grep -v 192\. | awk '{print $2}'"""
subprocess.call(cmd, shell=True)
or you can do it in harder way (Replacing shell pipeline from subprocess module documentation):
from subprocess import Popen, PIPE, call
p1 = Popen(['ifconfig'], stdout=PIPE)
p2 = Popen(['grep', 'inet '], stdin=p1.stdout, stdout=PIPE)
p3 = Popen(['grep', '-v', r'127\.0\.0\.1'], stdin=p2.stdout, stdout=PIPE)
p4 = Popen(['grep', '-v', r'192\.'], stdin=p3.stdout, stdout=PIPE)
call(['awk', '{print $2}'], stdin=p4.stdout)
I know there are posts already on how to use subprocess in python to run linux commands but I just cant get the syntax correct for this one. please help. This is the command I need to run...
/sbin/ifconfig eth1 | grep "inet addr" | awk -F: '{print $2}' | awk '{print $1}'
Ok this is what I have at the moment that gives a syntax error...
import subprocess
self.ip = subprocess.Popen([/sbin/ifconfig eth1 | grep "inet addr" | awk -F: '{print $2}' | awk '{print $1}'])
Any help greatly appreciated.
This has been gone over many, many times before; but here is a simple pure Python replacement for the inefficient postprocessing.
from subprocess import Popen, PIPE
eth1 = subprocess.Popen(['/sbin/ifconfig', 'eth1'], stdout=PIPE)
out, err = eth1.communicate()
for line in out.split('\n'):
line = line.lstrip()
if line.startswith('inet addr:'):
ip = line.split()[1][5:]
Here's how to construct the pipe in Python (rather than reverting to Shell=True, which is more difficult to secure).
from subprocess import PIPE, Popen
# Do `which` to get correct paths
GREP_PATH = '/usr/bin/grep'
IFCONFIG_PATH = '/usr/bin/ifconfig'
AWK_PATH = '/usr/bin/awk'
awk2 = Popen([AWK_PATH, '{print $1}'], stdin=PIPE)
awk1 = Popen([AWK_PATH, '-F:', '{print $2}'], stdin=PIPE, stdout=awk2.stdin)
grep = Popen([GREP_PATH, 'inet addr'], stdin=PIPE, stdout=awk1.stdin)
ifconfig = Popen([IFCONFIG_PATH, 'eth1'], stdout=grep.stdin)
procs = [ifconfig, grep, awk1, awk2]
for proc in procs:
print(proc)
proc.wait()
It'd be better to do the string processing in Python using re. Do this to get the stdout of ifconfig.
from subprocess import check_output
stdout = check_output(['/usr/bin/ifconfig', 'eth1'])
print(stdout)
I would like to store the result of a command in a variable.
By instance running : a = sudo("ls -l my_filename | awk '{print $11}'")
How could I achieve this ?
a = sudo("ls -l my_filename | awk '{print $11}'", capture=True )
I am having an issue with newlines in my command which involves the use of sed.
The scenario is as follows. When I execute the following command from Bash, I get:
cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed -e s/ph/\\nPH/g | grep -v ^$
PHysical id : 0core id : 0
PHysical id : 0core id : 1
As you can see, the sed command replaced ph with \nPH, such that I get a new line for each 'physical id...'
Now, I am calling this bash command from Python. Here is a small snippet of my code containing all relevant library imports.
import subprocess
cmd = 'cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed -e s/ph/\\nPH/g | grep -v ^$ '
subprocess.call(cmd, shell=True, universal_newlines=True, stdin=subprocess.PIPE)
The problem is that I get:
nPHysical id : 0core id : 0nPHysical id : 0core id : 1
on one line. It appears that the '\n' is not processed as the letter n is printed before PH.
I need to get the output nicely printed so that I can later add | sort | uniq | wc -l to my command to count the lines.
I would appreciate some help from the Bash-and-Python gurus out there.
Thank you.
Try:
cmd = r'cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed -e s/ph/\\nPH/g | grep -v ^$ '
The r means is a raw string. You might not be escaping some characters correctly.
Simply enclose the substitution command for sed with commas ("") to ensure it is passed correctly into python subprocess:
import subprocess
cmd = 'cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed -e "s/ph/\\nPH/g" | grep -v ^$ '
subprocess.call(cmd, shell=True, universal_newlines=True, stdin=subprocess.PIPE)
Output:
PHysical id : 0core id : 0
PHysical id : 0core id : 0