I'm trying to get the output of the following shell command in my python script,
hadoop fs -ls /projectpath/ | grep ^d | grep -v done | head -1 | awk {'print $8'}
I can successfully get the output through os.popen as follows:
import os
cmd = "hadoop fs -ls /projectpath/ | grep ^d | grep -v done | head -1 | awk {'print $8'}"
p = os.popen(cmd,"r")
while 1:
line = p.readline()
if not line: break
print line
But os.popen() is deprecated since python 2.6 so I wanted to replace the above snippet with the subprocess.Popen() function.
But the code snippet for subprocess.Popen() below gives a different result than the code snippet above.
import subprocess as sub
import shlex
cmd = "hadoop fs -ls /projectpath/ | grep ^d | grep -v done | head -1 | awk {'print $8'}"
args = shlex.split(cmd)
p = sub.Popen(args,stdout=sub.PIPE,stderr=sub.PIPE)
output, errors = p.communicate()
print output
The above command just gives output for 'hadoop fs -ls /projectpath/' part of the command.
I have tried consulting several references (http://docs.python.org/2/library/subprocess.html#popen-objects, Python, os.system for command-line call (linux) not returning what it should?) for subpocess.Popen() but cannot get it to execute the command in the string cmd. Can anyone point out what I'm doing wrong?
try this:
cmd = "hadoop fs -ls /projectpath/ | grep ^d | grep -v done | head -1 | awk {'print $8'}"
p = sub.Popen(cmd,stdout=sub.PIPE,stderr=sub.PIPE, shell=True)
Related
Would you please say me how can I convert this code:
path to app/adb devices | awk 'NR>1{print $1}' | xargs -n1 -I% adb -s % install app.apk
to python command?
You can always utilize the os.system() method:
import os
command = "path to app/adb devices | awk 'NR>1{print $1}' | xargs -n1 -I% adb -s % install app.apk"
os.system(command)
In my macOS, I using a wget downloading the www.test.com index page.
then I have below python code:
#-*- coding:utf-8 -*-
import subprocess
url = "https://www.test.com"
subprocess.call("ps -ef | grep wget | grep {0} | awk '{print $2}'".format(url), shell=True)
when I run it, I get issue:
subprocess.call("ps -ef | grep wget | grep {0} | awk '{print $2}'".format(url), shell=True)
KeyError: 'print $2'
and I switch the subprocess.call() to os.system(), still get this issue.
Your error is not caused by subprocess, rather the string format.
"ps -ef | grep wget | grep {0} | awk '{print $2}'".format(url)
you can use %s to forming string.
"ps -ef | grep wget | grep %s | awk '{print $2}'"%(url)
You need to use subprocess.run()
From Docs
Code needing to capture stdout or stderr should use run() instead
Note : Do not use stdout=PIPE or stderr=PIPE with this function. The child process will block if it generates enough output to a pipe to fill up the OS pipe buffer as the pipes are not being read from.
I have a process running in the background, a python one, with ps -ef I can see filename from running command : UID PID PPID ... python ./filename.py
How can I know where the file is located
pwdx < PID > gives full directory the process is running from.
So, the full script would be
ps -ef | grep 'your process' | awk '{print $2}' | xargs pwdx
Though, you can simplify this into
pgrep 'your process' | awk '{print $1}' | xargs pwdx
I have problem execute this code here
subprocess.check_output(['ps -ef | grep ftp | wc -l'],env=environ,shell=True)
When I execute from terminal
ps -ef | grep ftp | wc -l
I get "1" as output which is fine.
Now, I execute same code from my python files as subprocess.check_output and it gives me 2. That is strange. Any Ideas why is it happening. Here is the complete code:
def countFunction():
environ = dict(os.environ)
return subprocess.check_output(['ps -ef | grep ftp | wc -l'],env=environ,shell=True)
count = countFunction()
print count
EDIT:
Just to update , I do not have any ftp connections on.So command line is printing 1 on command which is fine.
Thanks
Arvind
The grep command will find itself:
$ ps -ef | grep ftp
wallyk 12546 12326 0 16:25 pts/3 00:00:00 grep ftp
If you don't want that, exclude the grep command:
$ ps -ef | grep ftp | grep -v ftp
$
It would be better to drop the -f switch to ps so that the command line arguments are not searched. That way, it won't find the grep ftp running:
$ ps -e | grep ftp | wc -l
The other questions aren't quite the same.
What I'm looking at achieving is a Python function which returns a list of all the IP addresses on a system, emulating the behaviour of:
ifconfig | grep 'inet addr:' | grep -v 127.0.0.1 | cut -d: -f2 | awk '{ print $1}'
You can use the subprocess python module to achieve this.
import subprocess
cmd = "ifconfig | grep 'inet addr:' | grep -v 127.0.0.1 | cut -d: -f2 | awk '{ print $1}'"
co = subprocess.Popen([cmd], shell = True, stdout = subprocess.PIPE)
ips = co.stdout.read().strip().split("\n")
That should give you a list of IP addresses.
PS : It'll be better to use the following command instead
ifconfig | grep inet | grep -v inet6 | grep -v 127.0.0.1 | awk '{print $2}' | cut -d\: -f2 | cut -d\ -f1
This will exclude IPV6 addresses if any.
Pure Python Way
If you are looking to do this entirely in Python, then checkout netifaces python module.