Develop Reference

Python subprocess, execute shell command which prompts for password

I am using subprocess in python to execute a custom vpn command which excepts password to execute.
below is the command which asks password:
./vpn -u <user_id> -d "description" /var/tmp/1.txt
password: XXXX
below is vpn shell command which works perfectly for above command.
/usr/bin/expect -c 'spawn ./vpn -u <user_id> -d "description" /var/tmp/1.txt; expect "Password"; send "<Password here>\r"; interact'
In python I am trying to achieve the same with below subprocess module, where below script executes below is the output.
"(b"\x1b[31m\xe2\x9c\x97 Can't read 'user id' password from the console.\x1b[0m""
import subprocess
args = ["./vpn", "-u", "<user_id>", "-d", "description", "/var/tmp/1.txt"]
# args = ['sudo','cat', '/var/tmp/1.txt']
proc = subprocess.Popen(args,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
proc.stdin.write('<passowrd>'.encode())
stdout, stderr = proc.communicate()
print(stdout)
Note: if I use time.sleep(2) before writing password, it prompts me for password in console without any error and my goal is to give password in the script.
Is there any better way in python to pass the shell command as this is pretty lengthy OR provide password in subprocess module execute seamlessly.

You are using Expect for Shell/Bash CLI command. When you use Python, just use pexpect to get the same functionality in Python.
Explore pexpect at https://github.com/pexpect/pexpect
Example passing password at executed program is here: https://github.com/pexpect/pexpect/blob/master/examples/passmass.py
Example:
import pexpect
child = pexpect.spawn('ftp ftp.openbsd.org')
child.expect('Name .*: ')
child.sendline('anonymous')
child.expect('Password:')
child.sendline('noah#example.com')

I tried in shell script and it works well: Just for anyone if it will be help full:
/usr/bin/expect<<vpn
spawn ./vpn -u <user_id> -d "description" /var/tmp/1.txt
expect "Password"
send "<password here>\r"
interact
sleep 5
vpn
echo "It's done"

python subprocess and wget ask-password

I'm running the following python code:
import subprocess
host = "ftp://localhost:2121"
p = subprocess.Popen(
['wget', '-P', '/tmp/output', '-N', '-r', '-l', 'inf', '--ask-password', '--user', 'anonymous', host],
stdin=subprocess.PIPE)
p.communicate("password\n")
if p.returncode != 0:
raise RuntimeError('wget command failed with return code: %d' % p.returncode)
It appears that password is not sent to wget since the script will hang showing:
Password for user ‘anonymous’:
Pressing enter causes wget to exit with an error code of '1'
Pressing keys then pressing enter causes wget to begin downloading as expected.
The ftp server is local and has anonymous access enabled. Python version is 2.7.8

Try adding this before p.communicate:
p.stdin.write('password\n')
p.stdin.flush()
stdout, stderr = p.communicate()

It appears that wget reads the password from the tty rather than stdin. These are not the same!
The sudo command allows you to switch to stdin
also
Certain programs read directly from /dev/tty, not stdin. "passwd" for example. So it's difficult to script them. Expect is one way around that - it can trick the program by providing input to them:
One way to achieve reading the password from stdin is to construct a url containing the user & pass:
ftp://user:password#ftp.server.com/link.txt
And using the -l - switch to pass the link into wget via stdin. This is suggested here

Using process.communicate after popen fails

I'm trying to send command using ssh spawn to remote machine.
I'm sending the command using Popen() and I can see the command was done but after that I'm trying to use communicate() (to close the session and get a return code) and the program get stuck.
cmd = """expect -c 'spawn ssh -o LogLevel=quiet """ \
"""-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o PreferredAuthentications=publickey """ \
"""-o ConnectTimeout=10 -2 -o CheckHostIP=no -i {0} {1}#{2} ;\
"""expect "#";send "mkdir test9\n" ;interact'""".format(self.sshKey, SSHUser, self.hostname)
process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
p_stdoutdata, p_stderrdata = process.communicate()
When I'm reaching the communicate the program gets stuck.

Don't use Popen to call ssh. Instead, use a library that does it all in-process in Python, such as the excellent Paramiko. In your case it might look something like this:
import paramiko
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.WarningPolicy())
client.connect(self.hostname, 22, SSHUser, password)
sftp = paramiko.SFTPClient.from_transport(client.get_transport())
sftp.mkdir("test9")
sftp.close()
client.close()

python popen stdout strderr return from ssh

I have a funtion which builds a ssh sommand lines and passes this to popen so I can run commands on a remote box ( I know I could use Fabirc or paramiko but for interests sake I wanted to do this).
so the fuction call is a wrapper arround Poepn (partly)
def runcmd(cmd, host=None, stdout=None, stderr=None, both=None, shell=None, user=None)
I invoke it for test with
runcmd('cd ~/app-scripts; grep "below and commment" *',user="testuser")
and the command it build up is shown as
ssh -F /home/testuser/.ssh/config testuser#localhost 'cd ~/app-scripts; grep "below and commment" *'
This runs on the "remote" host fine and the stdout correctly show the results --
however the stderr also contains all the login messages from the remote host.
The internal fuction call to Popen is
p = Popen(sshcmd,stdout=PIPE,stderr=PIPE,shell=True)
stdout_data,stderr_data = p.communicate()
What am i missing?

subprocess timeout when exceeds a certain time limit

I'm using the subprocess module of python to run an ssh command over my servers for collecting their disk usage. The one thing on which i'm stuck is if the ssh is not configured in any server then subprocess prompt for the password input which makes my whole script stuck and then i have to voluntarily kill the script itself. I just want it to let go all the servers which asks for password prompt(where ssh is not configured) and continue processing the rest.
def MyFunction(server):
msg=""
ps = subprocess.Popen("ssh -l mygroup %s 'df -k /some/directory'" % server,stdout=subprocess.PIPE,shell=True)
out, err = ps.communicate()
if err != None:
msg += "\n"+err
else:
msg = out
return msg
server_list= ['server A','server B','server C','server D']
for server in server_list:
Final_msg+=MyFunction(server)
Any help would be appreciated! :)

If it is just the thing that you want to avoid ssh ask you for anything, then you can forbid it to do so.
You can use the SSH option
BatchMode
If set to “yes”, passphrase/password querying will be disabled.
This option is useful in scripts and other batch jobs where no user is present to supply the password.
The argument must be “yes” or “no”. The default is “no”.
So just add -o BatchMode=yes:
ps = subprocess.Popen("ssh -o BatchMode=yes -l mygroup %s 'df -k /some/directory'" % server, stdout=subprocess.PIPE, shell=True)
BTW, why do you need shell=True here? Better do
ps = subprocess.Popen(["ssh", "-o", "BatchMode=yes", "-l", "mygroup", server, "df -k /some/directory"], stdout=subprocess.PIPE)
as it is cleaner, safer and internally simpler.