I'm trying to run a command from a python script, but I want to store the output the command produces and then check it for a substring, however it seems it is not being stored in my variable because it still prints to the screen.
So far I have this...
myfile = 'filename.txt'
result = subprocess.Popen(['myprogram.exe', '-f' + myfile], stdout=subprocess.PIPE).communicate()[0]
if result.find("error executing") != -1:
print "error!"
else:
print "success!"
I'm rather new to Python. Can anyone shed some light on WHY when I run this script, the myprogram.exe DOES execute, but it's output is still sent to the screen. If I print the result variable, it DOES have additional output from myprogram.exe, but I need the lines that show the error too.
You're only redirecting stdout. Looks like your program outputs errors to stderr (as expected), add stderr=subprocess.PIPE to the Popen call.
Related
I was trying to execute a windows console exe program using Popen on python 2.
import subprocess
#
# Other codes
#
p = subprocess.Popen(path_of_exe, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
output = p.stdout.read()
print output
p.stdin.write('mystring')
The program prints Command> when the program executed.
However, the output variable has nothing about the string.
It is always an empty string.
If I change stdout=subprocess.PIPE to stdout=sys.stdout, it printed outputs.
However, I need to get the output to parse the result from my input.
So, I want to grep the output was program.
I'm not sure why the subprocess.PIPE doesn't work.
Also, when I changed to stdout=sys.stdout, its output printed Command> infinitely.
The program should print the Command> only one time before I put string.
I also tried to get the output using p.communicate()[0], but it doesn't work.
I'm no idea with this behavior. When I worked in a Linux environment, there is no problem like that. I might think it is a problem of stdin or stdout buffer even though flush not worked.
I have tackled this problem before. Try and use this syntax when using Popn on Python2.
p=subprocess.Popen([python(where your python is located), file name, args],stdout=PIPE).stdout.read()
Let me know if this worked out for ya!
Is there a way in Powershell to redirect the print statements from python one place and the return statements to another place?
For instance i'm trying
$ python myPythonScript.py args >> logFile
I get print statement output in my log file (though it looks awful, cleaning will be the next job)
However, i do not get the return statement values from the python in the output. Nor do i catch them if I use *>>
Any ideas what's happening?
Python Example:
def main(args)
print "This goes to file"
#parse flags...
#do something here...
MyVar = "Please " + "Work"
return MyVar #doesn't go anywhere
if(__name__ == '__main__':
main(sys.argv)
The return value of a program (or exit code) can only be an integer, not a string.
First, ensure you return an int, or do exit(n) where n is an int.
You might want to also fix:
if(__name__ == '__main__'):
return main(sys.argv)
You can then access the return value of your program (script) in Powershell with echo %errorlevel%
If you really want a string to be used by powershell, you should:
print your logs on stderr instead of stdout
print the filename on stdout at end of execution
redirects stderr to your logfile and stdout to a powershell pipe | if you want a redirection - or you can execute the python script within parentheses $() so the result can be used on command line
I have a shell script TestNode.sh. This script has content like this:
port_up=$(python TestPorts.py)
python TestRPMs.py
Now, I want to capture the value returned by these scripts.
TestPorts.py
def CheckPorts():
if PortWorking(8080):
print "8080 working"
return "8080"
elif PortWorking(9090):
print "9090 working"
return "9090"
But as I checked the answers available, they are not working for me. The print is pushing the value in variable port_up, but I wanted that print should print on the console and the variable port_up should get the value from return statement. Is there a way to achieve this?
Note: I don't wish to use sys.exit(). Is it possible to achieve the same without this?
but I wanted that print should print on the console and the variable port_up should get the value from return statement.
Then don't capture the output. Instead do:
python TestPorts.py
port_up=$? # return value of the last statement
python TestRPMs.py
You could do:
def CheckPorts():
if PortWorking(8080):
sys.stderr.write("8080 working")
print 8080
But then I am not very happy to print "output" to stderr either.
Alternatively, you could skip printing that "8080 working" message in python script and print it from the shell script.
def CheckPorts():
if PortWorking(8080):
return "8080"
and:
port_up=$(python TestPorts.py)
echo "$port_up working"
python TestRPMs.py
To return an exit code from a Python script you can use sys.exit(); exit() may also work. In the Bash (and similar) shell, the exit code of the previous command can be found in $?.
However, the Linux shell exit codes are 8 bit unsigned integers, i.e. in the range 0-255, as mentioned in this answer. So your strategy isn't going to work.
Perhaps you can print "8080 working" to stderr or a logfile and print "8080" to stdout so you can capture it with $().
Is there a way that I can execute a shell program from Python, which prints its output to the screen, and read its output to a variable without displaying anything on the screen?
This sounds a little bit confusing, so maybe I can explain it better by an example.
Let's say I have a program that prints something to the screen when executed
bash> ./my_prog
bash> "Hello World"
When I want to read the output into a variable in Python, I read that a good approach is to use the subprocess module like so:
my_var = subprocess.check_output("./my_prog", shell=True)
With this construct, I can get the program's output into my_var (here "Hello World"), however it is also printed to the screen when I run the Python script. Is there any way to suppress this? I couldn't find anything in the subprocess documentation, so maybe there is another module I could use for this purpose?
EDIT:
I just found out that commands.getoutput() lets me do this. But is there also a way to achieve similar effects in subprocess? Because I was planning to make a Python3 version at some point.
EDIT2: Particular Example
Excerpt from the python script:
oechem_utils_path = "/soft/linux64/openeye/examples/oechem-utilities/"\
"openeye/toolkits/1.7.2.4/redhat-RHEL5-g++4.3-x64/examples/"\
"oechem-utilities/"
rmsd_path = oechem_utils_path + "rmsd"
for file in lMol2:
sReturn = subprocess.check_output("{rmsd_exe} {rmsd_pars}"\
" -in {sIn} -ref {sRef}".format(rmsd_exe=sRmsdExe,\
rmsd_pars=sRmsdPars, sIn=file, sRef=sReference), shell=True)
dRmsds[file] = sReturn
Screen Output (Note that not "everything" is printed to the screen, only a part of
the output, and if I use commands.getoutput everything works just fine:
/soft/linux64/openeye/examples/oechem-utilities/openeye/toolkits/1.7.2.4/redhat-RHEL5-g++4.3-x64/examples/oechem-utilities/rmsd: mols in: 1 out: 0
/soft/linux64/openeye/examples/oechem-utilities/openeye/toolkits/1.7.2.4/redhat-RHEL5-g++4.3-x64/examples/oechem-utilities/rmsd: confs in: 1 out: 0
/soft/linux64/openeye/examples/oechem-utilities/openeye/toolkits/1.7.2.4/redhat-RHEL5-g++4.3-x64/examples/oechem-utilities/rmsd - RMSD utility [OEChem 1.7.2]
/soft/linux64/openeye/examples/oechem-utilities/openeye/toolkits/1.7.2.4/redhat-RHEL5-g++4.3-x64/examples/oechem-utilities/rmsd: mols in: 1 out: 0
/soft/linux64/openeye/examples/oechem-utilities/openeye/toolkits/1.7.2.4/redhat-RHEL5-g++4.3-x64/examples/oechem-utilities/rmsd: confs in: 1 out: 0
To add to Ryan Haining's answer, you can also handle stderr to make sure nothing is printed to the screen:
p = subprocess.Popen(command, shell=True, stdin=subprocess.PIPE, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, close_fds=True)
out,err = p.communicate()
If subprocess.check_ouput is not working for you, use a Popen object and a PIPE to capture the program's output in Python.
prog = subprocess.Popen('./myprog', shell=True, stdout=subprocess.PIPE)
output = prog.communicate()[0]
the .communicate() method will wait for a program to finish execution and then return a tuple of (stdout, stderr) which is why you'll want to take the [0] of that.
If you also want to capture stderr then add stderr=subprocess.PIPE to the creation of the Popen object.
If you wish to capture the output of prog while it is running instead of waiting for it to finish, you can call line = prog.stdout.readline() to read one line at a time. Note that this will hang if there are no lines available until there is one.
I always used Subprocess.Popen, which gives you no output normally
I was looking to implement a python script that called another script and captured its stdout. The called script will contain some input and output messages eg
print ("Line 1 of Text")
variable = raw_input("Input 1 :")
print "Line 2 of Text Input: ", vairable
The section of the code I'm running is
import subprocess
cmd='testfile.py'
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
so, se = p.communicate()
print(so)
The problem that is occurring is that the stdout is not printing until after the script has been executed. This leaves a blank prompt waiting for the user input. Is there a way to get stdout to print while the called script is still running?
Thanks,
There are two problems here.
Firstly, python is buffering output to stdout and you need to prevent this. You could insert a call to sys.stdout.flush() in testfile.py as Ilia Frenkel has suggested, or you could use python -u to execute testfile.py with unbuffered I/O. (See the other stack overflow question that Ilia linked to.)
You need a way of asynchronously reading data from the sub-process and then, when it is ready for input, printing the data you've read so that the prompt for the user appears. For this, it would be very helpful to have an asynchronous version of the subprocess module.
I downloaded the asynchronous subprocess and re-wrote your script to use it, along with using python -u to get unbuffered I/O:
import async_subprocess as subprocess
cmd = ['python', '-u', 'testfile.py']
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
so = p.asyncread()
print so,
(so, se) = p.communicate()
print so
When I run this script using python -u I get the following results:
$ python -u script.py
Line 1 of Text
Input 1:
and the script pauses, waiting for input. This is the desired result.
If I then type something (e.g. "Hullo") I get the following:
$ python -u script.py
Line 1 of Text
Input 1:Hullo
Line 2 of Text Input: Hullo
You don't need to capture it's stdout really, just have the child program print out its stuff and quit, instead of feeding the output into your parent program and printing it there. If you need variable output, just use a function instead.
But anyways, that's not what you asked.
I actually got this from another stackoverflow question:
import subprocess, sys
cmd='testfile.py'
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
while True:
out = p.stdout.read(20)
if out == '' and p.poll() != None:
break
if out != '':
sys.stdout.write(out)
sys.stdout.flush()
First, it opens up your process: then it continually reads the output from p and prints it onto the screen using sys.stdout.write. The part that makes this all work is sys.stdout.flush(), which will continually "flush out" the output of the program.