I have a simple function that calls the same subprocess.popen command on multiple items from a list. It executes the first 2 items in the list no problem, however the action on the third hangs.
It seems that the process_null.communicate() never executes or doesnt finish at least as I never get the output from the 3rd item in the list. I have tried changing the list, but get the same results. Any ideas to whats going on here?
def check_list(server_list):
null_list = []
for target in server_list:
command_null="rpcclient -N -U '' {}". format (str(target))
process_null = subprocess.Popen(command_null, shell=True, stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
output_null = process_null.communicate()[0].replace('\n','')
if "NT_STATUS_ACCESS_DENIED" in output_null or "NT_STATUS_CONNECTION_REFUSED" in output_null or "NT_STATUS_UNSUCCESSFUL" in output_null:
print '[-] ' + target + '\tDenied NULL Session'
else:
print '[+] ' + target + '\tAccepted NULL Session'
null_list.append(target)
return null_list
Output
[-] 192.168.1.3 Denied NULL Session
[-] 192.168.1.65 Denied NULL Session
When rpcclient successfully establishes a connection it starts a shell and waits for commands to be entered on stdin if none are given using the -c flag, which is what happens here. You don't see the prompt because you redirect all output (stdout + stderr) to a pipe, but you don't do the same for stdin, which means the input will be read from the same tty the python interpreter is running on.
Also, you shouldn't use shell=True with a string argument if not absolutely necessary, use a list of arguments instead:
command_null = ['rpcclient', '-N', '-U', '', str(target)]
To fix your problem you have two options:
supply a command to execute when successfully connected:
command_null = ['rpcclient', '-N', '-U', '', '-c', 'exit', str(target)]
use stdin=PIPE when opening the process, which causes rcpclient to exit when stdin is closed by communicate():
process_null = subprocess.Popen(command_null, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE)
Related
I'm running the following python code:
import subprocess
host = "ftp://localhost:2121"
p = subprocess.Popen(
['wget', '-P', '/tmp/output', '-N', '-r', '-l', 'inf', '--ask-password', '--user', 'anonymous', host],
stdin=subprocess.PIPE)
p.communicate("password\n")
if p.returncode != 0:
raise RuntimeError('wget command failed with return code: %d' % p.returncode)
It appears that password is not sent to wget since the script will hang showing:
Password for user ‘anonymous’:
Pressing enter causes wget to exit with an error code of '1'
Pressing keys then pressing enter causes wget to begin downloading as expected.
The ftp server is local and has anonymous access enabled. Python version is 2.7.8
Try adding this before p.communicate:
p.stdin.write('password\n')
p.stdin.flush()
stdout, stderr = p.communicate()
It appears that wget reads the password from the tty rather than stdin. These are not the same!
The sudo command allows you to switch to stdin
also
Certain programs read directly from /dev/tty, not stdin. "passwd" for example. So it's difficult to script them. Expect is one way around that - it can trick the program by providing input to them:
One way to achieve reading the password from stdin is to construct a url containing the user & pass:
ftp://user:password#ftp.server.com/link.txt
And using the -l - switch to pass the link into wget via stdin. This is suggested here
I'm trying to automate setting new passwords using the Unix pass program.
I understand that there is a Python library, pexpect, that might help, but I would like to avoid using third-party libraries.
When using a terminal, the flow looks like this:
$ pass insert --force gmail
>> Enter password for gmail: <type in password using masked prompt>
>> Retype password for gmail: <reenter password>
What I would like my function to do:
Run the command pass insert --force {entry_name}
Capture the output (and echo it for testing)
Check output for the presence of 'password for gmail', and if True
write '{password}\n' to stdin
write '{password}\n' to stdin again
Echo any errors or messages for testing
Issues:
I'm stuck on step 2. The subprocess either hangs indefinitely, times out with an error, or produces no output.
Attempts:
I've tried configurations of Popen(), using both stdin.write() and communicate().
I've set wait() calls at various points.
I've tried both the shell=True and shell=False options (prefer False for security reasons)
Code:
def set_pass_password(entry_name, password):
from subprocess import Popen, PIPE
command = ['pass', 'insert', '--force', entry_name]
sub = Popen(command, stdin=PIPE, stdout=PIPE, stderr=PIPE, universal_newlines=True)
# At this point I assume that the command has run, and that there is an "Enter password..." message
message = sub.stdout.read() # also tried readline() and readlines()
print(message) # never happens, because process hangs on stdout.read()
if 'password for {}'.format(entry_name) in message:
err, msg = sub.communicate(input='{p}\n{p}\n'.format(p=password))
print('errors: {}\nmessage: {}'.format(err, msg))
Edit: the original answer was about passwd, which is what's used to set passwords. I noticed late that you use pass, which is a keystore (doesn't actually change the Unix password). The pass program works differently and will not print a prompt if stdin is not a tty. Therefore the following very simple program works:
def set_pass_password(entry_name, password):
from subprocess import Popen, PIPE
command = ['pass', 'insert', '--force', entry_name]
sub = Popen(command, bufsize=0, stdin=PIPE, stdout=PIPE, stderr=PIPE)
err, msg = sub.communicate(input='{p}\n{p}\n'.format(p=password))
print('errors: {}\nmessage: {}'.format(err, msg))
if __name__ == "__main__":
set_pass_password("ttt", "ttt123asdqwe")
(you will see that both stderr and stdout are empty, if the command succeeded).
For the passwd command:
FYI: the passwd command outputs the prompt to stderr, not stdout.
NOTE: rather than sending the password twice in the same 'write', you might need to wait for the second prompt before sending the password again.
For this simple case, code similar to yours should work, but in general you should use select on all the pipes and send/receive data when the other side is ready, so you don't get deadlocks.
I'm using a python script to manage ssh fingerprint problems after a workstation(s) is reimaged.
I attempt to connect with ssh, and if I get a any warnings I deal with them.
However, if there are no errors, then I am asked for a password to connect. At this point I want to terminate the process. However, the script hangs on the password request.
Here's the method:
def ssh_fingerprint_changed(node):
"""
Checks if a node's ssh fingerprint has changed or an old key is found, which can occur when a node is reimaged.
It does this by attempting to connect via ssh and inspecting stdout for an error message.
:param node: the ip or hostname of the node
:return: True if the node's fingerprint doesn't match the client's records. Else False.
"""
changed = False
cmd = ["ssh", "-q", ADMIN_USER + "#" + node, "exit"]
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stdin=subprocess.PIPE, universal_newlines=True)
print("Checking for fingerprint changes")
for line in proc.stdout: # loop on lines
print("in for loop") # NEVER REACHES HERE IF NO ERRORS, WAITING FOR PASSWORD
if b"Offending key" in line:
print("Offending key found.")
proc.stdin.write(b"no\n") # don't connect
changed = True
elif b"REMOTE HOST IDENTIFICATION HAS CHANGED!" in line:
print("REMOTE HOST IDENTIFICATION HAS CHANGED!")
changed = True
print(changed) # NEVER REACHES HERE IF NO ERRORS, WAITING FOR PASSWORD
if not changed: # then everything's good, but it will be waiting for a password to connect
print("Good to go, terminating ssh test.")
rc = proc.terminate()
else:
rc = proc.wait()
return changed
If I run this from the terminal ./my_python_script.py, I have the problems. Oddly, if I run in PyCharm, it doesn't hang on the password request and terminates shh, continuing with the script as expected.
The easy answer is simply to tell ssh that you don't want to support password authentication at all; you'll still get the messages you want if the host key is changed, but you won't ever have the process hanging waiting for a password to be entered.
cmd = ['ssh',
'-o', 'PasswordAuthentication no', ### <-- THIS LINE HERE
'-o', 'StrictHostKeyChecking yes', ### also, never modify known_hosts
'-q',
'%s#%s' % (ADMIN_USER, + node),
'exit']
If you did not want to process other prompts, I would suggest setting stdin=subprocess.DEVNULL (in Python 3) or passing the -n argument to ssh to prevent stdin from being passed to the process at all.
I'm using the subprocess module of python to run an ssh command over my servers for collecting their disk usage. The one thing on which i'm stuck is if the ssh is not configured in any server then subprocess prompt for the password input which makes my whole script stuck and then i have to voluntarily kill the script itself. I just want it to let go all the servers which asks for password prompt(where ssh is not configured) and continue processing the rest.
def MyFunction(server):
msg=""
ps = subprocess.Popen("ssh -l mygroup %s 'df -k /some/directory'" % server,stdout=subprocess.PIPE,shell=True)
out, err = ps.communicate()
if err != None:
msg += "\n"+err
else:
msg = out
return msg
server_list= ['server A','server B','server C','server D']
for server in server_list:
Final_msg+=MyFunction(server)
Any help would be appreciated! :)
If it is just the thing that you want to avoid ssh ask you for anything, then you can forbid it to do so.
You can use the SSH option
BatchMode
If set to “yes”, passphrase/password querying will be disabled.
This option is useful in scripts and other batch jobs where no user is present to supply the password.
The argument must be “yes” or “no”. The default is “no”.
So just add -o BatchMode=yes:
ps = subprocess.Popen("ssh -o BatchMode=yes -l mygroup %s 'df -k /some/directory'" % server, stdout=subprocess.PIPE, shell=True)
BTW, why do you need shell=True here? Better do
ps = subprocess.Popen(["ssh", "-o", "BatchMode=yes", "-l", "mygroup", server, "df -k /some/directory"], stdout=subprocess.PIPE)
as it is cleaner, safer and internally simpler.
I'm attempting to use the python subprocess module to log in to a secure ftp site and then grab a file. However I keep getting hung up on just trying to send the password when it is requested. I so far have the following code:
from subprocess import Popen, PIPE
proc = Popen(['sftp','user#server', 'stop'], stdin=PIPE)
proc.communicate('password')
This still stops at the password prompt. If I enter the password manually it then goes to the ftp site and then enters the password on the command line. I've seen people suggest using pexpect but long story short I need a standard library solution. Is there anyway with subprocess and/or any other stdlib? What am I forgetting above?
Try
proc.stdin.write('yourPassword\n')
proc.stdin.flush()
That should work.
What you describe sounds like stdin=None where the child process inherits the stdin of the parent (your Python program).
Perhaps you should use an expect-like library instead?
For instance Pexpect (example). There are other, similar python libraries as well.
from subprocess import Popen, PIPE
proc = Popen(['sftp','user#server', 'stop'], stdin=PIPE)
proc.communicate(input='password')
Try with input=‘password’ in communicate, that worked for me.
Use Paramiko for SFTP. For anything else, this works:
import subprocess
args = ['command-that-requires-password', '-user', 'me']
proc = subprocess.Popen(args,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
proc.stdin.write('mypassword\n')
proc.stdin.flush()
stdout, stderr = proc.communicate()
print stdout
print stderr
For some reason, I couldn't get any of the standard library answers here to work for me - getting very strange problems with all of them. Someone else here: unable to provide password to a process with subprocess [python] had the same problem, and concluded that ultimately you just have to go with pexpect to be able to send a password.
I wanted to add my final code here to just save the time of anyone having a similar problem, since I wasted so much time on this (Python 3, 2020):
ssh_password = getpass("user's password: ")
ssh_password = (ssh_password + "\n").encode()
scp_command = 'scp xx.xx.xx.xx:/path/to/file.log /local/save/path/'
child = pexpect.spawn(scp_command)
# make output visible for debugging / progress watching
child.logfile = sys.stdout.buffer
i = child.expect([pexpect.TIMEOUT, "password:"])
if i == 0:
print("Got unexpected output: {} {}".format(child.before, child.after))
return
else:
child.sendline(ssh_password)
child.read()
The above code runs an SCP command to pull a file from the remote server onto your local computer - alter the server IP and paths as necessary.
Key things to remember:
Have to have a pexpect.TIMEOUT in the child.expect call
Have to encode to bytes whatever strings you pass in, and have to use the default encode
Write pexpect output to sys.stdout.buffer so that you can actually see what is going on
Have to have a child.read() at the end
I would recommend scrapping the subprocess approach and using the paramiko package for sftp access.
This same problem plagued me for a week. I had to submit a password from user input through subprocess securely because I was trying to avoid introducing a command injection vulnerability. Here is how I solved the problem with a little help from a colleague.
import subprocess
command = ['command', 'option1', '--password']
subprocess.Popen(command, stdin=subprocess.PIPE).wait(timeout=60)
The .wait(timeout=int) was the most important component because it allows the user to feed input to stdin. Otherwise, the timeout is defaulted to 0 and leaves the user no time to enter input, which consequently results in a None or null string. Took me FOREVER to figure this out.
For repeat use-cases where you know you'll have to do this multiple times, you can override the popen function and use it as a private method which I was told by the same programmer is best practice if you anticipate someone else will be interested in maintaining the code later on and you don't want them to mess with it.
def _popen(cmd):
proc_h = subprocess.Popen(cmd, stdin=subprocess.PIPE)
proc_h.wait(timeout=60)
return proc_h.poll() == os.EX_OK
It is important to remove stdout=subprocess.PIPE if the user is going to be prompted for input. Otherwise, the process appears to hang for 60 seconds, and the user doesn't get a prompt, nor do they realize they are expected to give a password. The stdout will naturally go to the shell window and allow the user to pass input to popen().
Also, just to explain why you return proc_h.poll() == os.EX_OK, is that it returns 0 if the command succeeded. This is just c-style best-practice for when you want to return system error codes in the event the function fails, while accounting for the fact that return 0 will be treated as "false" by the interpreter.
This is a pure Python solution using expect - not pexpect.
If on Ubuntu you first need to install expect with:
sudo apt install expect
Python 3.6 or later:
def sftp_rename(from_name, to_name):
sftp_password = 'abigsecret'
sftp_username = 'foo'
destination_hostname = 'some_hostname'
from_name = 'oldfilename.txt'
to_name = 'newfilename.txt'
commands = f"""
spawn sftp -o "StrictHostKeyChecking no"
{sftp_username}#{destination_hostname}
expect "password:"
send "{sftp_password}\r"
expect "sftp>"
send "rename {from_name} {to_name}\r"
expect "sftp>"
send "bye\r"
expect "#"
"""
sp = subprocess.Popen(['expect', '-c', commands], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
since what you want is just grab a file, I am trying to use "sub process" but it is not works for me. So now I am using paramiko, here is my code:
here is one tutorial I found online
Transfer a file from local server to remote server and vice versa using paramiko of python
"https://www.youtube.com/watch?v=dtvV2xKaVjw"
underneath is my code for transfering all the files in one folder from Linux to windows
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname='11.11.11.1111', username='root', password='********', port=22)
sftp_client = ssh.open_sftp()
source_folder = '/var/ftp/file_pass'
local_folder = 'C:/temp/file_pass'
inbound_files = sftp_client.listdir(source_folder)
print(inbound_files)
for ele in inbound_files:
try:
path_from = source_folder + '/' + ele
path_to = local_folder + '/'+ ele
sftp_client.get(path_from, path_to)
except:
print(ele)
sftp_client.close()
ssh.close()
Python have a built in library called ftplib, that can be used for ftp processes without any hassle. (Assuming the remote server have a ftp service running)
from ftplib import FTP
ftp = FTP('ftp.us.debian.org') # connect to host, default port
ftp.login() # user anonymous, passwd anonymous#
##'230 Login successful.'
ftp.cwd('debian') # change into "debian" directory
##'250 Directory successfully changed.'
ftp.retrlines('LIST')
Otherwise, You can use scp command, which is a command line tool. The problem with the password can be avoided creating password less user for remote host.
import os
os.system('scp remoteuser#remotehost:/remote/location/remotefile.txt /client/location/')
To create a passwordless user in linux systems,
Fallow below Steps. Fallow this SO answer.
> ssh-keyscan remotehost
> known_hosts ssh-keygen -t rsa # ENTER toevery field (One time)
> ssh-copy-id remoteuser#remotehost
The safest way to do this is to prompt for the password beforehand and then pipe it into the command. Prompting for the password will avoid having the password saved anywhere in your code. Here's an example:
from getpass import getpass
from subprocess import Popen, PIPE
password = getpass("Please enter your password: ")
proc = Popen("sftp user#server stop".split(), stdin=PIPE)
# Popen only accepts byte-arrays so you must encode the string
proc.communicate(password.encode())
import subprocess
args = ['command', 'arg1', 'arg2']
proc = subprocess.Popen(args, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
proc.stdin.write(b'password') ##The b prefix is necessary because it needs a byte type
proc.stdin.flush()
stdout, stderr = proc.communicate()
print(stdout)
print(stderr)
You just forgot the line return (aka user pressing Enter) in your password.
from subprocess import Popen, PIPE
proc = Popen(['sftp','user#server', 'stop'], stdin=PIPE)
proc.communicate('password\n'.encode())
Also .encode() because by default proc.communicate() accept bytes-like object.