Develop Reference

return output from terminal

I would like to ask if is there a way to return to python a terminated value from terminal? For example I'm using os.system to write this line to ubuntu terminal
cat /sys/devices/virtual/thermal/thermal_zone*/temp
to return a couple of values, and I would like to be able to access this values again in python code, is it possible?

os.system is being deprecated, or at least replaced by subprocess
You can capture output fairly easily with subprocess.run
result = subprocess.run(['cat', '/sys/devices/virtual/thermal/thermal_zone*/temp'], capture_output=True)
print(result.stdout)

Because you have * in command so you have to use shell=True and shell will put full name in place of *
import subprocess
process = subprocess.run('cat /sys/devices/virtual/thermal/thermal_zone*/temp', shell=True, capture_output=True)
print(process.stdout.decode())
Result on my Linux Mint:
66000
47000

I think your looking for something like this: Retrieving the output of subprocess.call()
example:
p = Popen(['ls', '-l'], stdin=PIPE, stdout=PIPE, stderr=PIPE)
output, err = p.communicate(b"input data that is passed to subprocess' stdin")
rc = p.returncode

If you're using an older version of Python(<3.5)
from subprocess import check_output
result = check_output(["cat", "/sys/devices/virtual/thermal/thermal_zone*/temp"])
print(result)

How to implement a complex process pipe in Python 2.6?

I like to have the Python (2.6, sorry!) equivalent of this shell pipe:
$ longrunningprocess | sometextfilter | gzip -c
That is, I have to call a binary longrunningprocess, filter its output through sometextfilter and need to get gzip output.
I know how to use subprocess pipes, but I need the output of the pipe chunkwise (probably using yield) and not all at once. E.g. this
https://security.openstack.org/guidelines/dg_avoid-shell-true.html
works only for getting all output at once.
Note, that both longrunningprocess and sometextfilter are external programs, that cannot be replaced with Python functions.
Thanks in advance for any hint or example!

Again, I thought it were difficult, while Python is (supposed to be) easy. Just concatenating the subprocesses just works, it seems:
def get_lines():
lrp = subprocess.Popen(["longrunningprocess"],
stdout=subprocess.PIPE,
close_fds=True)
stf = subprocess.Popen(["sometextfilter"],
stdin=lrp.stdout,
stdout=subprocess.PIPE,
bufsize=1,
close_fds=True)
for l in iter(stf.stdout.readline, ''):
yield l
lrp.stdout.close()
stf.stdout.close()
stf.stdin.close()
stf.wait()
lrp.wait()
[Changes by J.F. Sebastian applied. Thanks!]
Then I can use Pythons gzip for compression.

The shell syntax is optimized for one-liners, use it:
#!/usr/bin/env python2
import sys
from subprocess import Popen, PIPE
LINE_BUFFERED = 1
ON_POSIX = 'posix' in sys.builtin_module_names
p = Popen('longrunningprocess | sometextfilter', shell=True,
stdout=PIPE, bufsize=LINE_BUFFERED, close_fds=ON_POSIX)
with p.stdout:
for line in iter(p.stdout.readline, ''):
print line, # do something with the line
p.wait()
How do I use subprocess.Popen to connect multiple processes by pipes?
Python: read streaming input from subprocess.communicate()
If you want to emulate the pipeline manually:
#!/usr/bin/env python2
import sys
from subprocess import Popen, PIPE
LINE_BUFFERED = 1
ON_POSIX = 'posix' in sys.builtin_module_names
sometextfilter = Popen('sometextfilter', stdin=PIPE, stdout=PIPE,
bufsize=LINE_BUFFERED, close_fds=ON_POSIX)
longrunningprocess = Popen('longrunningprocess', stdout=sometextfilter.stdin,
close_fds=ON_POSIX)
with sometextfilter.stdin, sometextfilter.stdout as pipe:
for line in iter(pipe.readline, ''):
print line, # do something with the line
sometextfilter.wait()
longrunningprocess.wait()

Call a system command inside Python and get its output result, not the exit status

In Python, I want to count the number of lines in a file xh-2.txt.
import subprocess
subprocess.call("wc -l xh-2.txt",shell=True)
But this is giving me exit status, not the result of the command.
I know the command print os.popen("wc -l xh-2.txt|cut -d' ' -f1").read() will do the job,
but popen is depreciated and why use read()?
What is the best way to call a system command inside Python and get its output result, not exit status?

Use subprocess.check_output().
Run command with arguments and return its output as a byte string.
>>> import subprocess
>>> import shlex
>>> cmd = 'wc -l test.txt'
>>> cm = shlex.split(cmd)
>>> subprocess.check_output(cm,shell=True)
' 1 test.txt\n'
>>>

You could use subprocess recipe
from subprocess import Popen, PIPE
Popen("wc -l xh-2.txt", shell=True, stdout=PIPE).communicate()[0]

How to run " ps cax | grep something " in Python?

How do I run a command with a pipe | in it?
The subprocess module seems complex...
Is there something like
output,error = `ps cax | grep something`
as in shell script?

See Replacing shell pipeline:
import subprocess
proc1 = subprocess.Popen(['ps', 'cax'], stdout=subprocess.PIPE)
proc2 = subprocess.Popen(['grep', 'python'], stdin=proc1.stdout,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
proc1.stdout.close() # Allow proc1 to receive a SIGPIPE if proc2 exits.
out, err = proc2.communicate()
print('out: {0}'.format(out))
print('err: {0}'.format(err))
PS. Using shell=True can be dangerous. See for example the warning in the docs.
There is also the sh module which can make subprocess scripting in Python a lot more pleasant:
import sh
print(sh.grep(sh.ps("cax"), 'something'))

You've already accepted an answer, but:
Do you really need to use grep? I'd write something like:
import subprocess
ps = subprocess.Popen(('ps', 'cax'), stdout=subprocess.PIPE)
output = ps.communicate()[0]
for line in output.split('\n'):
if 'something' in line:
...
This has the advantages of not involving shell=True and its riskiness, doesn't fork off a separate grep process, and looks an awful lot like the kind of Python you'd write to process data file-like objects.

import subprocess
process = subprocess.Popen("ps cax | grep something",
shell=True,
stdout=subprocess.PIPE,
)
stdout_list = process.communicate()[0].split('\n')

Drop that 'ps' subprocess and back away slowly! :)
Use the psutil module instead.

import os
os.system('ps -cax|grep something')
If you wanna replace grep argument with some variable:
os.system('ps -cax|grep '+your_var)

Parsing a stdout in Python

In Python I need to get the version of an external binary I need to call in my script.
Let's say that I want to use Wget in Python and I want to know its version.
I will call
os.system( "wget --version | grep Wget" )
and then I will parse the outputted string.
How to redirect the stdout of the os.command in a string in Python?

One "old" way is:
fin,fout=os.popen4("wget --version | grep Wget")
print fout.read()
The other modern way is to use a subprocess module:
import subprocess
cmd = subprocess.Popen('wget --version', shell=True, stdout=subprocess.PIPE)
for line in cmd.stdout:
if "Wget" in line:
print line

Use the subprocess module:
from subprocess import Popen, PIPE
p1 = Popen(["wget", "--version"], stdout=PIPE)
p2 = Popen(["grep", "Wget"], stdin=p1.stdout, stdout=PIPE)
output = p2.communicate()[0]

Use subprocess instead.

If you are on *nix, I would recommend you to use commands module.
import commands
status, res = commands.getstatusoutput("wget --version | grep Wget")
print status # Should be zero in case of of success, otherwise would have an error code
print res # Contains stdout