So this is the scenario. I'm trying to program to run like namefile.py (argument) using this code:
process=subprocess.Popen([sys.executable,filename],stdout=subprocess.PIPE)
res=process.communicate(argument)
z=res[0].strip("\n").strip("\r")
Much of my supriseness, the program passes the (argument) not by the command line argument, but by standard output. What is the correct way?
Note: I want to send it, not receive it.
Simply consult the documentation.
The command-line arguments are passed to the Popen call, and have nothing to do with stdin/stdout.
import subprocess
args = ['/path/to/executable', 'first_arg', 'second_arg']
p = subprocess.Popen(args, stdout=subprocess.PIPE)
all_of_stdout = p.stdout.read()
exit_code = p.wait()
Related
My python script (python 3.4.3) calls a bash script via subprocess.
OutPST = subprocess.check_output(cmd,shell=True)
It works, but the problem is, that I only get half of the data. The subprocess I call, calls a different subprocess and I have the guess, that if the "sub subprocess" sends the EOF, my programm thinks, that that´s it and ends the check_output.
Has someone an idea how to get all the data?
You should use subprocess.run() unless you really need that fine grained of control over talking to the processing via its stdin (or doing something else while the process is running instead of blocking for it to finish). It makes capturing output super easy:
from subprocess import run, PIPE
result = run(cmd, stdout=PIPE, stderr=PIPE)
print(result.stdout)
print(result.stderr)
If you want to merge stdout and stderr (like how you'd see it in your terminal if you didn't do any redirection), you can use the special destination STDOUT for stderr:
from subprocess import STDOUT
result = run(cmd, stdout=PIPE, stderr=STDOUT)
print(result.stdout)
I am trying to retrieve some information from a Perl script using Python and subprocess:
command = ["perl","script.perl","arg1.txt","<","arg2.txt"]
print " ".join(command)
p = subprocess.Popen(command,stdout=subprocess.PIPE,shell=True)
text = p.stdout.read()
The join-statement simply prints the command as I would enter it in the terminal for double-checking the quality of the command. That one always works... But within Python, it hangs at the subprocess.Popen() (at p= ... ).
I also tried several other methods such as call() but to no avail.
It only outputs one line of text, so I don't know how that could be the problem.
There's no need to involve the shell if you only want a simple input redirection. Open the file in Python, and pass the file handle to Popen via the stdin argument.
with open("arg2.txt") as infile:
command = ["perl", "script.perl", "arg1.txt"]
p = subprocess.Popen(command, stdout=subprocess.PIPE, stdin=infile)
text = p.stdout.read()
or
command = "perl script.perl arg1.txt < arg2.txt"
p = subprocess.Popen(command,stdout=subprocess.PIPE,shell=True)
text = p.stdout.read()
With a list and shell=True, it's not clear to me why it the call to perl blocks. When I try something like
subprocess.call("cat < .bashrc".split(), shell=True)
it blocks as if it is still trying to read from the inherited standard input. If I provide it with input using
subprocess.call("cat < .bashrc".split(), shell=True, stdin=open("/dev/null"))
the call returns immediately. In either case, it appears that cat is ignoring its further arguments.
Submitting a complex cmd string made of a full file path to an executable, the multiple flags, arguments, parameters, inputs and outputs seems to require me to set shell=True otherwise subprocess.Popen is not able understand anything more complex than just a simple path to executable (with no spaces in a filepath).
In my example I have quite a long cmd:
cmd = " '/Application/MyApp.app/Contents/MacOS/my_executable' '/Path/to/input/files' -some -flags -here -could -be -a -lot '/full/path/to/output/files' "
Submitting this cmd to subprocess.Popen " results to an error that complains on something about the path and not being able to find it.
So instead of using :
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
check_call seems workings quite well:
proc = subprocess.check_call(cmd, shell=True)
Interesting, only after shell is set to True
shell=True
the subprocess.check_call works with a supplied cmd.
The side effect is that the rest of the code seems proceeds running without waiting for subprocess.check_call(cmd, shell=True) to finish first.
The code is designed the way that the rest of the execution is dependent on a result of subprocess.check_call(cmd, shell=True).
I wonder if there is anyway to enforce the code execution to wait till subprocess.check_call(cmd, shell=True) is finished. Thanks in advance!
As #mikkas suggest just use it as a list here is a working example:
mainProcess = subprocess.Popen(['python', pyfile, param1, param2], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
# get the return value from the method
communicateRes = mainProcess.communicate()
stdOutValue, stdErrValue = communicateRes
You are calling python.exe pyfile param1 param2
By using communicate() you can get the stdout and stderr as a Tuple
You can use python method split() to split your string to a list for example:
cmd = "python.exe myfile.py arg1 arg2"
cmd.split(" ")
Output:
['python.exe', 'myfile.py', 'arg1', 'arg2']
I think the check_call function should wait for the command to finish.
See the docs here
http://docs.python.org/2/library/subprocess.html
Check call does not wait. You need to do a process.wait() and check the return code explicitly to get the functionaly you want.
Process = subprocess.Popen('%s' %command_string,stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
Process.wait()
if Process1.returncode!=0:
print Process1.returncode
sendMail()
return
else:
sendMail()
I have a shell script that gets whois info for domains, and outputs taken or available to the shell depending on the domain.
I'd like to execute the script, and be able to read this value inside my Python script.
I've been playing around with subprocess.call but can't figure out how to get the output.
e.g.,
subprocess.call('myscript www.google.com', shell=True)
will output taken to the shell.
subprocess.call() does not give you the output, only the return code. For the output you should use subprocess.check_output() instead. These are friendly wrappers around the popen family of functions, which you could also use directly.
For more details, see: http://docs.python.org/library/subprocess.html
Manually using stdin and stdout with Popen was such a common pattern that it has been abstracted into a very useful method in the subprocess module: communicate
Example:
p = subprocess.Popen(['myscript', 'www.google.com'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
(stdoutdata, stderrdata) = p.communicate(input="myinputstring")
# all done!
import subprocess as sp
p = sp.Popen(["/usr/bin/svn", "update"], stdin=sp.PIPE, stdout=sp.PIPE, close_fds=True)
(stdout, stdin) = (p.stdout, p.stdin)
data = stdout.readline()
while data:
# Do stuff with data, linewise.
data = stdout.readline()
stdout.close()
stdin.close()
Is the idiom I use, obviously in this case I was updating an svn repository.
try subprocess.check_output.
I am using the subprocess module to call an external program (plink.exe) to log-in to a server; but when I call communicate to read the output, it is blocking. The code is below:
import subprocess
process = subprocess.Popen('plink.exe hello#10.120.139.170 -pw 123456'.split(), shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print process.communicate() #block here
I know the block is because plink.exe it still running; but I need to read the output before the subprocess terminates. Is there anyway to do that?
The whole purpose of the communicate method is to wait for the process to finish and return all the output. If you don't want to wait, don't call communicate. Instead, read from the stdout or stderr attribute to read the output.
If the process outputs to both stdout and stderr (and you want to read it separately), you will have to be careful to actually read from both without blocking, or you can deadlock. This is fairly hard on Windows, and you may wish to use the pexpect module instead.
Maybe because "plink.exe" needs to take in input arguments, if you don't pass them, it will block until data are given, you could try adding arguments in method communicate(input)
I faced a similar situation where I had to execute a single command lmstat -a and then get the output of the terminal.
If you just need to run a single command and then read the output, you can use the following code:
import subprocess
Username = 'your_username'
Password = 'your_password'
IP = 'IP_of_system'
Connection_type = '-ssh' #can have values -ssh -telnet -rlogin -raw -serial
p = subprocess.Popen(['plink', Connection_type, '-l', Username, '-pw', Password, IP], \
shell = False, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
out, err = p.communicate('lmstat -a\nexit\n'.encode())
print(out.decode())