How can I hide the return code after calling a command using subprocess?
from netaddr import IPNetwork
import subprocess
for ip in IPNetwork('1.1.1.1/19'):
print subprocess.call(["host", str(ip)]
If I then pipe this to a file I get the host + ip but with a return code of 0 after each line.
subprocess.call returns the return code, and you are seeing it because you are printing it. The other output is being printed to stdout from the host program, since it isn't being redirected from the subprocess.call method.
If you want to capture the output of the process you are calling you should take a look at Popen's communicate method
Here is an example on how to capture stdout and stderr.
>>> proc = subprocess.Popen(['ls', '-alh', '/tmp/foo'],
... stdout=subprocess.PIPE,
... stderr=subprocess.PIPE,
... shell=False)
>>>
>>> stdout, stderr = proc.communicate()
>>> proc.returncode
0
Probably super dead by now, but figured I'd add my two cents since I was just looking into this.
I thought the same way that I'd have to encapsulate in print. I was getting error standard out plus the 0 exit code (standard error) after each line as well.
When I removed it from the print statement, it removed the exit code (standard error) but still printed the standard output (the output of the command).
Try this:
from netaddr import IPNetwork
import subprocess
for ip in IPNetwork('1.1.1.1/19'):
subprocess.call(["host", str(ip)])
I don't have the netaddr module installed so I can't tell you if it works or not.
Related
I'm trying to modify a python script to run a shell program, I do this with subprocess.call() like this:
def on_leftclick(self):
import subprocess
subprocess.call("mpc toggle", shell=True)
And I get this error when I left-click:
Could not parse JSON (lexical error: invalid char in json text.)
This is done in i3pystatus, a program that connects with i3bar (a part of the i3 window manager), I have already modified another script and it worked, an example:
def on_upscroll(self):
import subprocess
subprocess.call("pamixer --increase 1 --allow-boost", shell=True)
Also, I tried doing this:
import subprocess
subprocess.call("mpc toggle", shell=True)
in the python shell and it worked, so I don't get what the problem is.
Any help would be much appreciated.
This is caused by doing this in the i3pystatus module:
subprocess.call("mpc toggle") # parameter shell=True or False, doesn't matter
Because the subprocess.call() function by default passes the stdout and stderr file handles of the parent process (i.e. i3pystatus) to the new child (mpc), all the output from your mpc call get dumped into i3pystatus's own output. This output is supposed to be valid JSON, but now has a big unexpected chunk of random console spew in it. This results in the output of i3pystatus failing to parse as JSON.*
You have two options:
Use subprocess to capture/dispose of the sub-process's output
Dispose of it:
def on_leftclick(self):
subprocess.call("mpc toggle", shell=True,
stdout=subprocess.DEVNULL)
or, if you want the output, subprocess.Popen():
def on_leftclick(self):
p = subprocess.Popen(['mpc', 'toggle'],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
out, err = p.communicate() # .decode() these to strings if appropriate
retcode = p.returncode
Use the run_through_shell helper in i3pystatus
i3pystatus provides a helper to wrap subprocess for you: i3pystatus.core.command.run_through_shell:
from i3pystatus.core.command import run_through_shell
def on_leftclick(self):
run_through_shell(["mpc", "toggle"])
or, to get your shell to do the parsing of the command, give it the string:
def on_leftclick(self):
run_through_shell("mpc toggle", enable_shell=True)
You can also get the return code and stdout/stderr outputs easily, for you own nefarious purposes:
def on_leftclick(self):
retcode, out, err = run_through_shell(["mpc", "toggle"])
# out and err are already decoded to UTF-8 - you can't use this
# method if that's not true, e.g. if it's binary data
To be fair, run_through_shell didn't exist until the year after the question was asked (it was added in December 2014). i3pystatus uses both methods in the codebase.
TL;DR: Make sure your subprocess isn't spewing output into
i3pystatus's own output, because that needs to be valid JSON.
* Probably fails in this call to json.loads(), but I haven't proved that.
Is there a way that I can execute a shell program from Python, which prints its output to the screen, and read its output to a variable without displaying anything on the screen?
This sounds a little bit confusing, so maybe I can explain it better by an example.
Let's say I have a program that prints something to the screen when executed
bash> ./my_prog
bash> "Hello World"
When I want to read the output into a variable in Python, I read that a good approach is to use the subprocess module like so:
my_var = subprocess.check_output("./my_prog", shell=True)
With this construct, I can get the program's output into my_var (here "Hello World"), however it is also printed to the screen when I run the Python script. Is there any way to suppress this? I couldn't find anything in the subprocess documentation, so maybe there is another module I could use for this purpose?
EDIT:
I just found out that commands.getoutput() lets me do this. But is there also a way to achieve similar effects in subprocess? Because I was planning to make a Python3 version at some point.
EDIT2: Particular Example
Excerpt from the python script:
oechem_utils_path = "/soft/linux64/openeye/examples/oechem-utilities/"\
"openeye/toolkits/1.7.2.4/redhat-RHEL5-g++4.3-x64/examples/"\
"oechem-utilities/"
rmsd_path = oechem_utils_path + "rmsd"
for file in lMol2:
sReturn = subprocess.check_output("{rmsd_exe} {rmsd_pars}"\
" -in {sIn} -ref {sRef}".format(rmsd_exe=sRmsdExe,\
rmsd_pars=sRmsdPars, sIn=file, sRef=sReference), shell=True)
dRmsds[file] = sReturn
Screen Output (Note that not "everything" is printed to the screen, only a part of
the output, and if I use commands.getoutput everything works just fine:
/soft/linux64/openeye/examples/oechem-utilities/openeye/toolkits/1.7.2.4/redhat-RHEL5-g++4.3-x64/examples/oechem-utilities/rmsd: mols in: 1 out: 0
/soft/linux64/openeye/examples/oechem-utilities/openeye/toolkits/1.7.2.4/redhat-RHEL5-g++4.3-x64/examples/oechem-utilities/rmsd: confs in: 1 out: 0
/soft/linux64/openeye/examples/oechem-utilities/openeye/toolkits/1.7.2.4/redhat-RHEL5-g++4.3-x64/examples/oechem-utilities/rmsd - RMSD utility [OEChem 1.7.2]
/soft/linux64/openeye/examples/oechem-utilities/openeye/toolkits/1.7.2.4/redhat-RHEL5-g++4.3-x64/examples/oechem-utilities/rmsd: mols in: 1 out: 0
/soft/linux64/openeye/examples/oechem-utilities/openeye/toolkits/1.7.2.4/redhat-RHEL5-g++4.3-x64/examples/oechem-utilities/rmsd: confs in: 1 out: 0
To add to Ryan Haining's answer, you can also handle stderr to make sure nothing is printed to the screen:
p = subprocess.Popen(command, shell=True, stdin=subprocess.PIPE, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, close_fds=True)
out,err = p.communicate()
If subprocess.check_ouput is not working for you, use a Popen object and a PIPE to capture the program's output in Python.
prog = subprocess.Popen('./myprog', shell=True, stdout=subprocess.PIPE)
output = prog.communicate()[0]
the .communicate() method will wait for a program to finish execution and then return a tuple of (stdout, stderr) which is why you'll want to take the [0] of that.
If you also want to capture stderr then add stderr=subprocess.PIPE to the creation of the Popen object.
If you wish to capture the output of prog while it is running instead of waiting for it to finish, you can call line = prog.stdout.readline() to read one line at a time. Note that this will hang if there are no lines available until there is one.
I always used Subprocess.Popen, which gives you no output normally
(Using python 3.2 currently)
I need to be able to:
Run a command using subprocess
Both stdout/stderr of that command need be printed to the terminal in real-time (it doesn't matter if they both come out on stdout or stderr or whatever
At the same time, I need a way to know if the command printed anything to stderr (and preferably what it printed).
I've played around with subprocess pipes as well as doing strange pipe redirects in bash, as well as using tee, but as of yet haven't found anything that would work. Is this something that's possible?
My solution:
import subprocess
process = subprocess.Popen("my command", shell=True,
stdout=None, # print to terminal
stderr=subprocess.PIPE)
duplicator = subprocess.Popen("tee /dev/stderr", shell=True, # duplicate input stream
stdin=process.stderr,
stdout=subprocess.PIPE, # catch error stream of first process
stderr=None) # print to terminal
error_stream = duplicator.stdout
print('error_stream.read() = ' + error_stream.read())
Try something like this:
import os
cmd = 'for i in 1 2 3 4 5; do sleep 5; echo $i; done'
p = os.popen(cmd)
while True:
output = p.readline()
print(output)
if not output: break
In python2 you can catch stderr easily as well by using popen3 like this:
i, o, err = os.popen3(cmd)
but there seem to be no such function in python3. If you don find the way around this, try using subprocess.Popen directly, as described here: http://www.saltycrane.com/blog/2009/10/how-capture-stdout-in-real-time-python/
>>> import subprocess
>>> subprocess.Popen("pwd")
<subprocess.Popen object at 0xa7692cc>
>>> subprocess.call(["ls", "-l"])
0
I have tried the above command in Python interactive shell and expect to see the output
inside the shell environment. However, it ends up with just some return values.
What should I do in order to let the return results printed inside the shell?
It'd help to read the documentation first.
p = subprocess.Popen('pwd', stdout = subprocess.PIPE)
p.communicate() # returns (stdout, None)
Cat Plus Plus is right, however, if you prefer the call function, you can also do this like so:
import subprocess
import sys
subprocess.call('pwd', stdout=sys.stdout) # outputs results of `pwd` to stdout
I have a shell script that gets whois info for domains, and outputs taken or available to the shell depending on the domain.
I'd like to execute the script, and be able to read this value inside my Python script.
I've been playing around with subprocess.call but can't figure out how to get the output.
e.g.,
subprocess.call('myscript www.google.com', shell=True)
will output taken to the shell.
subprocess.call() does not give you the output, only the return code. For the output you should use subprocess.check_output() instead. These are friendly wrappers around the popen family of functions, which you could also use directly.
For more details, see: http://docs.python.org/library/subprocess.html
Manually using stdin and stdout with Popen was such a common pattern that it has been abstracted into a very useful method in the subprocess module: communicate
Example:
p = subprocess.Popen(['myscript', 'www.google.com'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
(stdoutdata, stderrdata) = p.communicate(input="myinputstring")
# all done!
import subprocess as sp
p = sp.Popen(["/usr/bin/svn", "update"], stdin=sp.PIPE, stdout=sp.PIPE, close_fds=True)
(stdout, stdin) = (p.stdout, p.stdin)
data = stdout.readline()
while data:
# Do stuff with data, linewise.
data = stdout.readline()
stdout.close()
stdin.close()
Is the idiom I use, obviously in this case I was updating an svn repository.
try subprocess.check_output.