I want to input one argument to the subprocess.communicate, but it always be out of my expects.
the folder tree:
├── file1
├── file2
├── main.py
the content of main.py:
import subprocess
child = subprocess.Popen(["ls"], stdin=subprocess.PIPE, universal_newlines=True)
filepath = '/Users/haofly'
child.communicate(filepath)
whatever i change the filepath to, the result only lists current folder(file1,file2,main.py).
Is I misunderstanding the communicate? How I send data to the Popen?
And how about ssh command if i want to send password?
subprocess.Popen(['ssh', 'root#ip'], stdin=subprocess.PIPE, universal_newlines=True)
You cannot 'pipe' data into ls - it lists directories based on the provided CLI arguments - but you should be able to use xargs to achieve what you want (essentially passing your folder as an argument to ls) if you don't want to provide it with the command itself:
import subprocess
child = subprocess.Popen(["xargs", "ls"], stdin=subprocess.PIPE, universal_newlines=True)
filepath = '/Users/haofly'
child.communicate(filepath)
When you use ls manually, do you type ls alone, hit Enter, and then type in the filepath in response to a prompt? That's how you're trying to use it here - the parameter to .communicate() becomes the subprocess's standard input, which in fact ls ignores completely. It wants the directory to list as a command-line parameter, which you would specify as ["ls", filepath].
I think you are missing the a shell parameter in your Popen call:
import subprocess
process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)
process.wait()
print process.returncode
'ls' is probably not the best command to illustrate the concept, but if you want to pass in arguments to a command you would have to do something similar to this:
cmd = ['cmd', 'opt1', 'optN']
p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
stdin=subprocess.PIPE)
out, err = p.communicate('args')
print out
Related
I have an external application called gmx which works like so:
gmx trjconv -f test.trr -s test.tpr -o test.xtc -center -fit progressive -ur compact << eof
1
1
0
eof
This commands creates the file test.xtc in the current directory.
I would like to call this from a python subprocess. I tried the following:
p = subprocess.Popen(
['gmx', 'trjconv', '-f test.trr', '-s test.tpr', '-o test.xtc', '-center', '-ur compact', '-fit progressive'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
p.stdin.write(b'1\n1\n0\n')
p.stdin.close()
But it fails silently in the sense that no error occurs but nothing happens. I read some posts about passing eof but I could not adapt them to my needs. Would you have any idea about how to do this ?
The problem was in the way my Popen was written. Rewriting it as:
p = subprocess.Popen(
['gmx', 'trjconv', '-f', 'test.trr', '-s', 'test.tpr', '-o', 'test.xtc', '-center', '-ur', 'compact', '-fit', 'progressive'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
p.communicate(b'1\n1\n0')
did the job.
There were at least two mistakes:
the arguments list must contain the arguments one by one (e.g. not
'-f test.trr' but '-f', 'test.trr')
using communicate when using both stdin and stdout
When I execute below program, it list file correctly.
import subprocess
foo = subprocess.run("ls /home/my_home",
shell=True,
executable="/bin/bash",
stdout=subprocess.PIPE,
stdin=subprocess.PIPE,
stderr=subprocess.PIPE)
my_std_out = foo.stdout.decode("utf-8")
But when execute below program, there is nothing in stdout.
import subprocess
foo = subprocess.Popen(["ls /home/my_home"],
shell=True,
executable="/bin/bash",
stdout=subprocess.PIPE,
stdin=subprocess.PIPE,
stderr=subprocess.PIPE)
my_std_out = foo.stdout.read().decode("utf-8")
I wonder is there anything wrong with my second part program?
Thankyou in advance!
From python docs:
"communicate() returns a tuple (stdout_data, stderr_data). The data will be strings if streams were opened in text mode; otherwise, bytes."
Therefore, if you'd like to get output via Popen, you have to unpack the retruned tuple from communicate() like this:
out, err = foo.communicate()
In [150]: out
Out[150]: b''
In [151]: err
Out[151]: b"ls: cannot access '/home/my_home': No such file or directory\n"
I think the bash command and the path should be placed between quotes each when you use brackets like the following
import subprocess foo = subprocess.Popen(["ls", "/home/my_home"], shell=True, executable=/bin/bash, stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE) my_std_out = foo.stdout.read().decode("utf-8")
I'm using multiple commands to run:
e.g. cd foo/bar; ../../run_this -arg1 -arg2="yeah_ more arg1 arg2" arg3=/my/path finalarg
Running with:
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
(out, err) = p.communicate()
But this spits output on screen (Python 2.7.5)
And out is empty string.
You have shell=True, so you're basically reading the standard output of the shell spawned, not the standard output of the program you want to run.
I'm guessing you're using shell=True to accommodate the directory changing. Fortunately, subprocess can take care of that for you (by passing a directory via the cwd keyword argument):
import subprocess
import shlex
directory = 'foo/bar'
cmd = '../../run_this -arg1 -arg2="yeah_ more arg1 arg2" arg3=/my/path finalarg'
p = subprocess.Popen(shlex.split(cmd), cwd=directory, stdout=subprocess.PIPE)
(out, err) = p.communicate()
As per comment I added stderr too and that worked!:
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
So I'm trying to move away from os.popen to subprocess.popen as recommended by the user guide. The only trouble I'm having is I can't seem to find a way of making readlines() work.
So I used to be able to do
list = os.popen('ls -l').readlines()
But I can't do
list = subprocess.Popen(['ls','-l']).readlines()
ls = subprocess.Popen(['ls','-l'], stdout=subprocess.PIPE)
out = ls.stdout.readlines()
or, if you want to read line-by-line (maybe the other process is more intensive than ls):
for ln in ls.stdout:
# whatever
With subprocess.Popen, use communicate to read and write data:
out, err = subprocess.Popen(['ls','-l'], stdout=subprocess.PIPE).communicate()
Then you can always split the string from the processes' stdout with splitlines().
out = out.splitlines()
Making a system call that returns the stdout output as a string:
lines = subprocess.check_output(['ls', '-l']).splitlines()
list = subprocess.Popen(['ls', '-l'], stdout=subprocess.PIPE).communicate()[0].splitlines()
straight from the help(subprocess)
A more detailed way of using subprocess.
# Set the command
command = "ls -l"
# Setup the module object
proc = subprocess.Popen(command,
shell=True,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
# Communicate the command
stdout_value,stderr_value = proc.communicate()
# Once you have a valid response, split the return output
if stdout_value:
stdout_value = stdout_value.split()
I think something is getting subtly mangeled when I attempt to execute a subprocess from a python script
I attempt to execute vlc with some (a lot) of arguments.
the instance of vlc that arises complains:
Your input can't be opened:
VLC is unable to open the MRL ' -vvv rtsp://192.168.1.201:554/ch0_multicast_one --sout=#transcode{acodec=none}:duplicate{dst=rtp{sdp=rtsp://:5544/user_hash.sdp},dst=display} :no-sout-rtp-sap :no-sout-standard-sap :ttl=1 :sout-keep'. Check the log for details.
Here is the python code
pid = subprocess.Popen(["vlc "," -vvv rtsp://%s" % target_nvc.ip_address + ":554/ch0_multicast_one --sout=#transcode{acodec=none}:duplicate{dst=rtp{sdp=rtsp://:5544/user_hash.sdp},dst=display} :no-sout-rtp-sap :no-sout-standard-sap :ttl=1 :sout-keep" ], stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
I have examined the output of the subprocess function (using a shell), and if I copy paste that string into my cmd window, the vlc instance works fine... Is this a privilege thing?
Since you're passing a list to subprocess.Popen, each parameter must be in its own element. So you'd want something like:
pid = subprocess.Popen([
"vlc",
"-vvv",
"rtsp://%s:554/ch0_multicast_one" % target_nvc.ip_address,
# etc
], ...)
Each parameter (that the shell would normally parse apart for you) must be in a separate list element.
You can also pass a single command line string and let the shell pull it apart:
pid = subprocess.Popen("vlc -vvv rtsp://...", shell=True, ...)
Using the first form is better for commands that have lots of arguments.
You should use this...
pid = subprocess.Popen(["vlc", "-vvv",
"rtsp://%s" % target_nvc.ip_address + ":554/ch0_multicast_one",
"--sout=#transcode{acodec=none}:duplicate{dst=rtp{sdp=rtsp://:5544/user_hash.sdp},dst=display}",
":no-sout-rtp-sap", ":no-sout-standard-sap",
":ttl=1", ":sout-keep" ], stdout=subprocess.PIPE,
stderr=subprocess.PIPE, stdin=subprocess.PIPE)
movies_path = glob.glob("D:\\MOVIES\**\*\*\*.mp4", recursive=True) + \
glob.glob("D:\\MOVIES\**\*\*\*.mkv", recursive=True) + \
glob.glob("D:\\MOVIES\**\*\*\*.avi", recursive=True)
# probably the right movie
rightMoviePath = difflib.get_close_matches(which_movie, movies_path, len(movies_path), 0)
movie_name = rightMoviePath[0].split("\\")[-1]
hebrew_subtitle_path = glob.glob(rightMoviePath[0].replace(movie_name, "Hebrew.srt"))[0]
english_subtitle_path = glob.glob(rightMoviePath[0].replace(movie_name, "English.srt"))[0]
process, player = subprocess.Popen(["C:\\Users\\yonat\\Downloads\\VLC\\vlc.exe", "--sub-file", hebrew_subtitle_path, rightMoviePath[0]],
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)