When i am running a program in the console, i get some text output.
When i am running the same program in Popen(..), with the same parameters, stdout and stderr are empty.
I tried everything i could imagine like shell=False and shell=True, set stdout=subprocess.PIPE, did a os.chdir() to change into the directory of this program, try p.wait() and p.communicate(), set the command as a list and as a string, but nothing works.
example:
p = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
out, err = p.communicate()
--> out and err are empty strings, but if i ran this command in console i get a real output. Command is with fullpath, so its regardless where the command will be started.
My question is, are there mechanisms for programms to detect they weren't run in a real console? If so, how can i cheat.
Or miss i something?
(Python 2.7.8. x32 in Win7 x64)
from subprocess import Popen, STDOUT, PIPE
p = Popen(command, shell=True, stdout=PIPE, stderr=STDOUT, stdin=PIPE)
while p.poll() is None:
print(p.stdout.read())
p.stdout.close()
p.stdin.close()
Try this and see if it makes any difference. Also make sure command is a string and not a list/touple, shell=True for whatever reason works better or only with strings.
Also note that shell=True will get you hanged because it's insecure etc.
Also skipping .communicate() you'll need to tap off stdout otherwise the buffer will get full and you might hang both yours and the child process.
If this doesn't work, please provide more information. Such as the command used and the expected output (at least first few lines)
Related
As per solutions here and python manual following code snippet must not print anything on screen, including stderr of the cmd.
p = subprocess.Popen(cmd, cwd=tmpdir, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
o, e = p.communicate()
But somehow I get the error messages of the cmd on terminal. Has the behaviour changed? I have tried following variant, and it has same results too:
p = subprocess.Popen(cmd, cwd=tmpdir, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
o, e = p.communicate()
I have Python 2.7.5 on CentOs 7.
No, it didn't change. Your assumptions are wrong. Make sure you're running the correct piece of code, or try to find out what other things could be writing to the terminal. Because running a subprocess with stderr=subprocess.STDOUT will redirect all stderr output to stdout.
I have a Python script to capture network traffic with tcpdump in a subprocess:
p = subprocess.Popen(['tcpdump', '-I', '-i', 'en1',
'-w', 'cap.pcap'], stdout=subprocess.PIPE)
time.sleep(10)
p.kill()
When this script completes its work, I'm trying to open output .pcap file in Wireshark and getting this error:
"The capture file appears to have been cut short in the middle of a packet."
What solution could be applied for "proper" closing of tcpdump's subprocess?
Instead of p.kill(), you can use p.send_signal(subprocess.signal.SIGTERM) to send a terminate signal rather than a kill (p.terminate() does the same).
The Popen docs describe the send_signal() command. The documentation on signals is a bit weak, but a dir(subprocess.signal) will list all the signals you may send to the process, but terminate should allow it some time to clean up.
Found working solution:
I've changed p.kill() to p.terminate().
After this change the subprocess is "properly" finished (output of tcpdump subprocess with statistics available in console) and output .pcap file not damaged.
I had the same problem about closing subprocesses. This thread really helped, so thanks, especially to https://stackoverflow.com/users/3583715/rkh. My solution:
Before:
output = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True)
message = output.stdout.read()
output.stdout.close()
After reading the Popen docs:
output = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True)
message = output.stdout.read()
output.TerminateProcess()
For some reason, calling output.kill(), and/or output.terminate() or sending output.send_signal(subprocess.signal.SIGTERM) didn't work, but output.TerminateProcess() did.
i have a daemon running which gets command and executes it by:
subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
I never do anything with it afterwards, no wait(), no communicate()
is that okay to do so?
or are joining of the process required?
is there a similar thing to threading.deamon=True for subprocess.Popen?
Since you set stdout=subprocess.PIPE, stderr=subprocess.PIPE, and if you want to get the stdout, you can do this:
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
for line in p.stdout:
# do something here
#...
As you say the cmd is executed as a daemon, I think daemon process should not be run in this way. Since once your python process exit with error, the daemon process would exited and your data will lost.
And if you run other process by subprocess.Popen(), such as mysql, linux comands etc, it is ok to do so. But you should know the process status(the return code), you should check the status of the process by call poll() or wait(). For more information, see docs of subproces.Popen.
I want to execute a python subprocess in a new console. Once started, I want the user to be able to answer questions asked by this new process on stdin.
I tried the following code:
p = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=cwd, creationflags=subprocess.CREATE_NEW_CONSOLE)
(o, e) = p.communicate()
As soon as the subprocess asks for input on stdin the following error message is displayed:
EOFError: EOF when reading a line
Is it the good way to achieve this ?
As i'm not really interested in the stdout/stderr redirection, i tried this way:
subprocess.Popen(cmd, cwd=cwd, creationflags=subprocess.CREATE_NEW_CONSOLE)
It works fine now. I guess that it's not compatible to redirect standard input/outputs and to create a new console.
I'm trying to run a shell command from within a python script which needs to do several things
1. The shell command is 'hspice tran.deck >! tran.lis'
2. The script should wait for the shell command to complete before proceeding
3. I need to check the return code from the command and
4. Capture STDOUT if it completed successfully else capture STDERR
I went through the subprocess module and tried out a couple of things but couldn't find a way to do all of the above.
- with subprocess.call() I could check the return code but not capture the output.
- with subprocess.check_output() I could capture the output but not the code.
- with subprocess.Popen() and Popen.communicate(), I could capture STDOUT and STDERR but not the return code.
I'm not sure how to use Popen.wait() or the returncode attribute. I also couldn't get Popen to accept '>!' or '|' as arguments.
Can someone please point me in the right direction? I'm using Python 2.7.1
EDIT: Got things working with the following code
process = subprocess.Popen('ls | tee out.txt', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = process.communicate()
if(process.returncode==0):
print out
else:
print err
Also, should I use a process.wait() after the process = line or does it wait by default?
Just use .returncode after .communicate(). Also, tell Popen that what you're trying to run is a shell command, rather than a raw command line:
p = subprocess.Popen('ls | tee out.txt', shell=True, ...)
p.communicate()
print p.returncode
From the docs:
Popen.returncode
The child return code, set by poll() and wait() (and indirectly by communicate()). A None value indicates that the process hasn’t terminated yet.
A negative value -N indicates that the child was terminated by signal N (Unix only).
Here is example how to interact with shell:
>>> process = subprocess.Popen(['/bin/bash'], shell=False, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
>>> process.stdin.write('echo it works!\n')
>>> process.stdout.readline()
'it works!\n'
>>> process.stdin.write('date\n')
>>> process.stdout.readline()
'wto, 13 mar 2012, 17:25:35 CET\n'
>>>