I am developing a backup python program, one of the modules makes a rsync backup of a remote folder into my local device.
This is the part of the code where I have the problem:
try:
process = subprocess.Popen(
['sshpass',
'-p',
password,
'rsync',
'-avz',
'-e',
'ssh -o StrictHostKeyChecking=no',
'-p',
port,
'{}#{}:{}'.format(user, host, folder),
dest_folder],
stdout=subprocess.PIPE
)
output = process.communicate()[0]
if int(process.returncode) != 0:
print('Command failed. Return code : {}'.format(process.returncode))
exit(1)
return output
except Exception as e:
print(e)
exit(1)
The shown error is:
Unexpected remote arg: debian#12.345.67.89:/folder1/folder2/
rsync error: syntax or usage error (code 1) at main.c(1372) [sender=3.1.3]
Command failed. Return code : 1
I believe that the problem is with the array in Popen. If I run the single command in bash I rsync successfully.
What should I change from subprocess.Popen array?
This is caused by -p flag being consumed by rsync rather than ssh, and as such the destination effectively is set to port.
The -e argument of rsync takes exactly one parameter. That means that only 'ssh -o StrictHostKeyChecking=no' will be passed as an argument. Unluckily for you, -p is an actual flag of rsync, and as such it's processed without error. It means 'preserve permissions'. This means that rather than setting port to use with ssh, you're passing a flag to rsync, and the next parameter gets interpreted as a destination.
You can fix it by changing
'ssh -o StrictHostKeyChecking=no',
'-p',
port,
to simply
'ssh -o StrictHostKeyChecking=no -p {}'.format(port),
Related
I'm running the following python code:
import subprocess
host = "ftp://localhost:2121"
p = subprocess.Popen(
['wget', '-P', '/tmp/output', '-N', '-r', '-l', 'inf', '--ask-password', '--user', 'anonymous', host],
stdin=subprocess.PIPE)
p.communicate("password\n")
if p.returncode != 0:
raise RuntimeError('wget command failed with return code: %d' % p.returncode)
It appears that password is not sent to wget since the script will hang showing:
Password for user ‘anonymous’:
Pressing enter causes wget to exit with an error code of '1'
Pressing keys then pressing enter causes wget to begin downloading as expected.
The ftp server is local and has anonymous access enabled. Python version is 2.7.8
Try adding this before p.communicate:
p.stdin.write('password\n')
p.stdin.flush()
stdout, stderr = p.communicate()
It appears that wget reads the password from the tty rather than stdin. These are not the same!
The sudo command allows you to switch to stdin
also
Certain programs read directly from /dev/tty, not stdin. "passwd" for example. So it's difficult to script them. Expect is one way around that - it can trick the program by providing input to them:
One way to achieve reading the password from stdin is to construct a url containing the user & pass:
ftp://user:password#ftp.server.com/link.txt
And using the -l - switch to pass the link into wget via stdin. This is suggested here
I'm trying to send command using ssh spawn to remote machine.
I'm sending the command using Popen() and I can see the command was done but after that I'm trying to use communicate() (to close the session and get a return code) and the program get stuck.
cmd = """expect -c 'spawn ssh -o LogLevel=quiet """ \
"""-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o PreferredAuthentications=publickey """ \
"""-o ConnectTimeout=10 -2 -o CheckHostIP=no -i {0} {1}#{2} ;\
"""expect "#";send "mkdir test9\n" ;interact'""".format(self.sshKey, SSHUser, self.hostname)
process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
p_stdoutdata, p_stderrdata = process.communicate()
When I'm reaching the communicate the program gets stuck.
Don't use Popen to call ssh. Instead, use a library that does it all in-process in Python, such as the excellent Paramiko. In your case it might look something like this:
import paramiko
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.WarningPolicy())
client.connect(self.hostname, 22, SSHUser, password)
sftp = paramiko.SFTPClient.from_transport(client.get_transport())
sftp.mkdir("test9")
sftp.close()
client.close()
I have a funtion which builds a ssh sommand lines and passes this to popen so I can run commands on a remote box ( I know I could use Fabirc or paramiko but for interests sake I wanted to do this).
so the fuction call is a wrapper arround Poepn (partly)
def runcmd(cmd, host=None, stdout=None, stderr=None, both=None, shell=None, user=None)
I invoke it for test with
runcmd('cd ~/app-scripts; grep "below and commment" *',user="testuser")
and the command it build up is shown as
ssh -F /home/testuser/.ssh/config testuser#localhost 'cd ~/app-scripts; grep "below and commment" *'
This runs on the "remote" host fine and the stdout correctly show the results --
however the stderr also contains all the login messages from the remote host.
The internal fuction call to Popen is
p = Popen(sshcmd,stdout=PIPE,stderr=PIPE,shell=True)
stdout_data,stderr_data = p.communicate()
What am i missing?
I need to ssh into the server and execute few commands and process the response using subprocess. Here's my code
command = 'ssh -t -t buildMachine.X.lan; sudo su - buildbot ; build-set sets/set123'
print "submitting command"
result = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
print "got response"
response,err = result.communicate()
print response
This is getting stuck. I have seen other threads talking about passing a list instead of string to subprocess and removing shell=True.. I did that too but didn't work.
Ultimately I need the result of last command i.e. build-set in order to extract some information out of it.. help?
I figured the solution by using univerio's comment
The command needs to be
command = 'ssh -t -t buildMachine.X.lan \'sudo su - buildbot \'build-set sets/set123\'\''
Individual commands are like argument to previous command. This works.
I'm using the subprocess module of python to run an ssh command over my servers for collecting their disk usage. The one thing on which i'm stuck is if the ssh is not configured in any server then subprocess prompt for the password input which makes my whole script stuck and then i have to voluntarily kill the script itself. I just want it to let go all the servers which asks for password prompt(where ssh is not configured) and continue processing the rest.
def MyFunction(server):
msg=""
ps = subprocess.Popen("ssh -l mygroup %s 'df -k /some/directory'" % server,stdout=subprocess.PIPE,shell=True)
out, err = ps.communicate()
if err != None:
msg += "\n"+err
else:
msg = out
return msg
server_list= ['server A','server B','server C','server D']
for server in server_list:
Final_msg+=MyFunction(server)
Any help would be appreciated! :)
If it is just the thing that you want to avoid ssh ask you for anything, then you can forbid it to do so.
You can use the SSH option
BatchMode
If set to “yes”, passphrase/password querying will be disabled.
This option is useful in scripts and other batch jobs where no user is present to supply the password.
The argument must be “yes” or “no”. The default is “no”.
So just add -o BatchMode=yes:
ps = subprocess.Popen("ssh -o BatchMode=yes -l mygroup %s 'df -k /some/directory'" % server, stdout=subprocess.PIPE, shell=True)
BTW, why do you need shell=True here? Better do
ps = subprocess.Popen(["ssh", "-o", "BatchMode=yes", "-l", "mygroup", server, "df -k /some/directory"], stdout=subprocess.PIPE)
as it is cleaner, safer and internally simpler.