I was trying to execute a windows console exe program using Popen on python 2.
import subprocess
#
# Other codes
#
p = subprocess.Popen(path_of_exe, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
output = p.stdout.read()
print output
p.stdin.write('mystring')
The program prints Command> when the program executed.
However, the output variable has nothing about the string.
It is always an empty string.
If I change stdout=subprocess.PIPE to stdout=sys.stdout, it printed outputs.
However, I need to get the output to parse the result from my input.
So, I want to grep the output was program.
I'm not sure why the subprocess.PIPE doesn't work.
Also, when I changed to stdout=sys.stdout, its output printed Command> infinitely.
The program should print the Command> only one time before I put string.
I also tried to get the output using p.communicate()[0], but it doesn't work.
I'm no idea with this behavior. When I worked in a Linux environment, there is no problem like that. I might think it is a problem of stdin or stdout buffer even though flush not worked.
I have tackled this problem before. Try and use this syntax when using Popn on Python2.
p=subprocess.Popen([python(where your python is located), file name, args],stdout=PIPE).stdout.read()
Let me know if this worked out for ya!
Related
I am writing a python script, I want to call from crontab. It script calls the xrandr command and saves its output in a variable like so:
output = subprocess.run('xrandr', shell=True, stdout=subprocess.PIPE).stdout.decode('utf-8')
I want the output of xrandr to be saved in a string.
This works all fine if I execute it from terminal, but if I run it using cron, the variable output stays empty.
the rest of the code is executed normally, so cron isn't the problem.
So how can I make this command execute properly?
thank you for your suggestions.
You want to store output, you can use communicate() here to help out, so like this:
from subprocess import PIPE
output = subprocess.run('xrandr', shell=True, stdout=subprocess.PIPE).stdout.decode('utf-8')
text = output.communicate()[0]
print(text)
OR maybe this, in that case you can remove the .stdout.decode('utf-8') not too sure but it give a shot with and without it:
from subprocess import PIPE
output = subprocess.run('xrandr', shell=True, stdout=subprocess.PIPE).stdout.decode('utf-8')
print(output.stdout)
I guess in cron environment the PATH variable is not set, so you should provide absolute path to xrandr (you can find it by which xrandr).
E.g. if this path /usr/bin/xrandr try
from subprocess import PIPE
output = subprocess.run('/usr/bin/xrandr', shell=True, stdout=subprocess.PIPE).stdout.decode('utf-8')
text = output.communicate()[0]
print(text)
The better way for my opinion is to capture stderr as well and log error when it occures.
The code is the following (on a python 2.7 prompt)
import subprocess as sp
a = sp.Popen(['bash', '-c', 'ssh [REDACTED] cat text.txt'],
stdout=sp.PIPE, stderr=sp.PIPE)
print(a.communicate()[0])
I would expect this program to print the entire "text.txt" file, however, it is missing the last few lines.
The file text.txt is in a remote server, which I assume is the gist of it. If I retrieve the file and then do subprocess.Popen locally, it works as expected.
Why does it happen and how can I fix it?
note: the result is the same if I use check_output
According to the documentation for communicate, the command shouldn't be used if the data size is very large. Try something like this and see if that solves your problem.
I'm trying to write a python script that returns a value which I can then pass in to a bash script. Thing is that I want a singe value returned in bash, but I want a few things printed to the terminal along the way.
Here is an example script. Let's call it return5.py:
#! /usr/bin/env python
print "hi"
sys.stdout.write(str(5))
what I want is to have this perform this way when I run it from the command line:
~:five=`./return5.py`
hi
~:echo $five
5
but what I get is:
~:five=`./return5.py`
~:echo $five
hi 5
In other words I don't know how to have a python script print and clear the stdout, then assign it to the specific value I want.
Not sure why #yorodm suggests not to use stderr. That's the best option I can think of in this case.
Notice that print will add a newline automatically, but when you use sys.stderr.write, you need to include one yourself with a "\n".
#! /usr/bin/env python
import sys
sys.stderr.write("This is an important message,")
sys.stderr.write(" but I dont want it to be considered")
sys.stderr.write(" part of the output. \n")
sys.stderr.write("It will be printed to the screen.\n")
# The following will be output.
print 5
Using this script looks like this:
bash$ five=`./return5.py`
This is an important message, but I dont want it to be considered part of the output.
It will be printed to the screen.
bash$ echo $five
5
This works because the terminal is really showing you three streams of information : stdout, stdin and stderr. The `cmd` syntax says "capture the stdout from this process", but it doesn't affect what happens to stderr. This was designed exactly for the purpose you're using it for -- communicating information about errors, warnings or what's going on inside the process.
You may not have realized that stdin is also displayed in the terminal, because it's just what shows up when you type. But it wouldn't have to be that way. You could imagine typing into the terminal and having nothing show up. In fact, this is exactly what happens when you type in a password. You're still sending data to stdin, but the terminal is not displaying it.
from my comment..
#!/usr/bin/env python
#foo.py
import sys
print "hi"
sys.exit(5)
then the output
[~] ./foo.py
hi
[~] FIVE=$?
[~] echo $FIVE
5
You can use stdout to output your messages and stderr to capture the values in bash. Unfortunately this is some weird behaviour as stderr is intended for programs to communicate error messages so I strongly advice you against it.
OTOH you can always process your script output in bash
Hi I am trying to execute shell script from python using following command.
os.system("sh myscript.sh")
in my shell script I have written some SOP's, now how do I get the SOP's in my Python so that I can log them into some file?
I know using subprocess.Popen I can do it, for some reason I can not use it.
p=subprocess.Popen(
'DMEARAntRunner \"'+mount_path+'\"',
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT
)
while 1:
line=p.stdout.readline()[:-1]
if not line:
break
write_to_log('INFO',line)
p.communicate()
If I understand your question correctly, you want something like this:
import subprocess
find_txt_command = ['find', '-maxdepth', '2', '-name', '*.txt']
with open('mylog.log', 'w') as logfile:
subprocess.call(find_txt_command, stdout=logfile, shell=False)
You can use Popen instead of call if you need to, the syntax is very similar. Notice that command is a list with the process you want to run and the arguments. In general you want to use Popen/call with shell=False, it prevents unexpected behavior that can be hard to debug and it is more portable.
Kindly check this official documentation which uses the subprocess module in python. It is currently the recommended way over os.system calls to execute system functions and retrieve the results. The link above gives examples very close to what you need.
I personally would advise you to leave the shell argument at its default value of False. In that case, the first argument isn't a string as you'd type into a terminal, but a list of "words", the first being the program, the ones after that being arguments. This means that there is no need to quote arguments, making your program more resilient to whitespace arguments and injection attacks.
This should do the trick:
p = subsprocess.Popen(['DMEARAntRunner', mount_path],
stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
As always with executing shell commands the question remains whether it's the easiest/best way to solve a problem, but that's another discussion altogether.
I'm calling rtmpdump via subprocess and trying to redirect its output to a file. The problem is that I simply can't redirect it.
I tried first setting up the sys.stdout to the opened file. This works for, say, ls, but not for rtmpdump. I also tried setting the sys.stderr just to make sure and it also didn't work.
I tried then using a ">> file" with the command line argument but again it doesn't seem to work.
Also for the record, for some reason, Eclipse prints rtmpdump's output even if I use subprocess.call instead of subprocess.check_output, and without having to call the print method. This is black magic!
Any suggestions?
Edit: Here's some sample code.
# /!\ note: need to use os.chdir first to get to the folder with rtmpdump!
command = './rtmpdump -r rtmp://oxy.videolectures.net/video/ -y 2007/pascal/bootcamp07_vilanova/keller_mikaela/bootcamp07_keller_bss_01 -a video -s http://media.videolectures.net/jw-player/player.swf -w ffa4f0c469cfbe1f449ec42462e8c3ba16600f5a4b311980bb626893ca81f388 -x 53910 -o test.flv'
split_command = shlex.split(command)
subprocess.call(split_command)
sys.stdout is the python's idea of the parent's output stream.
In any case you want to change the child's output stream.
subprocess.call and subprocess.Popen take named parameters for the output streams.
So open the file you want to output to and then pass that as the appropriate argument to subprocess.
f = open("outputFile","wb")
subprocess.call(argsArray,stdout=f)
Your talk of using >> suggest you are using shell=True, or think you are passing your arguments to the shell. In any case it is better to use the array form of subprocess, which avoid an unnecessary process, and any weirdness from the shell.
EDIT:
So I downloaded RTMPDump and tried it out, it would appear the messages are appearing on stderr.
So with the following program, nothing appears on the programs output, and the rtmpdump logs when into the stderr.txt file:
#!/usr/bin/env python
import os
import subprocess
RTMPDUMP="./rtmpdump"
assert os.path.isfile(RTMPDUMP)
command = [RTMPDUMP,'-r','rtmp://oxy.videolectures.net/video/',
'-y','2007/pascal/bootcamp07_vilanova/keller_mikaela/bootcamp07_keller_bss_01',
'-a','video','-s',
'http://media.videolectures.net/jw-player/player.swf',
'-w','ffa4f0c469cfbe1f449ec42462e8c3ba16600f5a4b311980bb626893ca81f388'
,'-x','53910','-o','test.flv']
stdout = open("stdout.txt","wb")
stderr = open("stderr.txt","wb")
subprocess.call(command,stdout=stdout,stderr=stderr)
See the link on getting the output from subprocess on SO
Getting the entire output from subprocess.Popen
https://stackoverflow.com/questions/tagged/subprocess
I guess the way would be to collect the output and write it to a file directly or provide file descriptors to which you output can be written.
Something like this:
f = open('dump.txt', 'wb')
p = subprocess.Popen(args, stdout=f, stderr=subprocess.STDOUT, shell=True)