Develop Reference

Store lines of text that result from shell command

Here is the code related to this section of the code:
command1 = '/usr/local/GMT5SAR/bin/ALOS_baseline ' + str(master_file) + ' ' + str(master_file)
print command1
p1 = Popen(command1, shell=True, stdout=PIPE)
out,err = p1.communicate()
print out
My command is working properly. Here's a screenshot of my console.
I need to store the lines that say lon_tie_point ..... and lat_tie_point ...... The issue i'm running into is that those lines aren't including in out, which is what i'm printing out. How can I go about doing this?

It seems that the lines containing the information you need are being printed on stderr instead of stdout. From the subprocess documentation:
subprocess.STDOUT
Special value that can be used as the stderr argument to Popen and indicates that standard error should go into the same handle as standard output.
Based on this, I think the following might work:
command1 = '/usr/local/GMT5SAR/bin/ALOS_baseline ' + str(master_file) + ' ' + str(master_file)
print command1
p1 = Popen(command1, shell=True, stdout=PIPE, stderr=STDOUT)
out,err = p1.communicate()
print out

access to file-output of external program

I'm trying to call a program from within Python that creates output and I want to work with this output when the external program has finished.
The programstring is
"sudo fing -r 1 > fingoutput.txt".
I managed to call the program with
from subprocess import call
cmd = ['sudo', 'fing', '-r 1']
call(cmd)
but I can't direct it's output to a file.
cmd = ['sudo', 'fing', '-r 1 > fingoutput.txt']
or
cmd = ['sudo', 'fing', '-r 1', '> fingoutput.txt']
produce
Error: multiple occurrences
I want to write the output to a file because it might be thousands of lines.
Thank you for your help,
Herbert.

You can use the stdout argument to redirect the output of your command to a file:
from subprocess import call
cmd = ['sudo', 'fing', '-r 1']
file_ = open('your_file.txt', 'w')
call(cmd, stdout=file_)

If you want to redirect to file from the shell-script itself you can always go for this
cmd = 'sudo fing -r 1 > fingoutput.txt'
call(cmd, shell=True)
Or
cmd = 'sudo fing -r 1 > fingoutput.txt'
p = Popen(cmd, shell=True, stdout=PIPE, stdin=PIPE)
Keeping shell=True may lead to security issues.

subprocess ssh command fails for some commands but not others (command works in terminal)

As part of a python script, I am hoping to capture the output of a shell command executed via ssh, namely
ssh User#999 screen -list
If I execute the above command directly in terminal, I get the results I need. However, when executing through subprocess.check_output as below, I get a non-zero exit status 1 error.
I am able to execute other commands via ssh and capture the output without problem.
Is there something specific about screen -list that does not like being called in this fashion?
import subprocess
srvr = 'User#999.99.999.9'
print("CMD 1: ===============")
cmd1 = "ssh " + srvr + " ls -l"
print ("COMMAND IS ..... " + cmd1 + "\n")
out1 = subprocess.check_output(cmd1, shell=True)
print(out1 + "\n")
print("CMD 2: ===============")
cmd2 = "ssh " + srvr + " screen -list"
print ("COMMAND IS ..... " + cmd2 + "\n")
out2 = subprocess.check_output(cmd2, shell=True)
print(out2 + "\n")
Error:
subprocess.CalledProcessError: Command '['ssh User#999.99.999.9 screen', '-list']' returned non-zero exit status 1

subprocess.check_output check the exit code of the subprocess; and it raises exception if the exit code is not zero.
If you don't care about exit code, use subprocess.Popen.communicate:
out1, err1 = subprocess.Popen(cmd1,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE).communicate()

That's how subprocess.check_output() is supposed to work. See: http://docs.python.org/2/library/subprocess.html
The command on your server is returning a non zero return code and thus is raising the appropriate Exception CalledProcessError.

Not able to give inputs to subprocess(process which runs adb shell command) after 100 iterations

I want to run a stress test for adb(android debug bridge) shell. ( adb shell in this respect just a command line tool provided by Android phones).
I create a sub-process from python and in this subprocess i execute 'adb shell' command. there are some commands which has to be given to this subprocess which I am providing via stdin proper of the sub process.
Everything seems to be fine but when I am running a stress test. after around 100 iterations the command which I give to stdin does not reach to subprocess. If I run commands in separate terminal it is running fine. but the problem is with this stdin.
Can anyone tell me what I am doing wrong. Below is the code sample
class ADB():
def __init__(self):
self.proc = subprocess.Popen('adb shell', stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True,bufsize=0)
def provideAMcommand(self, testParam):
try:
cmd1 = "am startservice -n com.test.myapp/.ADBSupport -e \"" + "command" + "\" \"" + "test" + "\""
cmd2 = " -e \"" + "param" + "\"" + " " + testParam
print cmd1+cmd2
sys.stdout.flush()
self.proc.stdin.write(cmd1 + cmd2 + "\n")
except:
raise Exception("Phone is not Connected to Desktop or ADB is not available \n")

If it works for the first few commands but blocks later then you might forgot to read from self.proc.stdout that might lead to (as the docs warn) to OS pipe buffer filling up and blocking the child process.
To discard the output, redirect it to os.devnull:
import os
from subprocess import Popen, PIPE, STDOUT
DEVNULL = open(os.devnull, 'wb')
# ...
self.proc = Popen(['adb', 'shell'], stdin=PIPE, stdout=DEVNULL, stderr=STDOUT)
# ...
self.proc.stdin.write(cmd1 + cmd2 + "\n")
self.proc.stdin.flush()
There is pexpect module that might be a better tool for a dialog-based interaction (if you want both read/write intermitently).

IN provideAMcommand you are writing to and flushing the stdout of your main process. That will not send anything to the stdin of the child process you have created with Popen. The following code creates a new bash child process, a bit like the code in your __init__:
import subprocess as sp
cproc = sp.Popen("bash", stdin=sp.PIPE, stdout=sp.PIPE, stderr=sp.PIPE, shell=True)
Now, the easiest way to communicate with that child process is the following:
#Send command 'ls' to bash.
out, err = cproc.communicate("ls")
This will send the text "ls" and EOF to bash (equal to running a bash script with only the text "ls" in it). Bash will execute the ls command and then quit. Anything that bash or ls write to stdout and stderr will end up in the variables out and err respectively. I have not used the adb shell, but I guess it behaves like bash in this regard.
If you just want your child process to print to the terminal, don't specify the stdout and stderr arguments to Popen.
You can check the exit code of the child, and raise an exception if it is non-zero (indicating an error):
if (cproc.returncode != 0):
raise Exception("Child process returned non-zero exit code")

Failing to capture stdout from application

I have the following script:
import subprocess
arguments = ["d:\\simulator","2332.txt","2332.log", "-c"]
output=subprocess.Popen(arguments, stdout=subprocess.PIPE).communicate()[0]
print(output)
which gives me b'' as output.
I also tried this script:
import subprocess
arguments = ["d:\\simulator","2332.txt","atp2332.log", "-c"]
process = subprocess.Popen(arguments,stdout=subprocess.PIPE)
process.wait()
print(process.stdout.read())
print("ERROR:" + str(process.stderr))
which gives me the output: b'', ERROR:None
However when I run this at the cmd prompt I get a 5 lines of text.
d:\simulator atp2332.txt atp2332.log -c
I have added to simulator a message box which pops up when it launches. This is presented for all three cases. So I know that I sucessfully launch the simulator. However the python scripts are not caturing the stdout.
What am I doing wrong?
Barry.

If possible (not endless stream of data) you should use communicate() as noted on the page.
Try this:
import subprocess
arguments = ["d:\\simulator","2332.txt","atp2332.log", "-c"]
process = subprocess.Popen(arguments, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
sout, serr = process.communicate()
print(sout)
print(serr)

The following code gives me text output on stdout.
Perhaps you could try it, and then substitute your command for help
import subprocess
arguments = ["help","2332.txt","atp2332.log", "-c"]
process = subprocess.Popen(arguments,stdout=subprocess.PIPE, stderr=subprocess.PIPE)
process.wait()
print 'Return code', process.returncode
print('stdout:', process.stdout.read())
print("stderr:" + process.stderr.read())