Develop Reference

Read output of a command executed on SSH server with Paramiko as string, not as bytes

I wrote a script a few years ago in Python 2.x and am trying to port it to run in 3.x
Here are the basics of what I am doing:
NetworkDevices is a file that contains host and access credentials for a list of devices.
DebugFile is an output file I dump stuff too as I go (depending on the debug level), so I can see when a device sent something I wasn't expecting.
NetworkDevices = open(WorkingDir + DeviceFilename,"r")
DebugFile = open(WorkingDir + DebugFilename,"w",1)
Try
# Create instance of SSHClient object
ssh_pre = paramiko.SSHClient()
# Automatically add untrusted hosts (make sure okay for security policy in your environment)
ssh_pre.set_missing_host_key_policy(paramiko.AutoAddPolicy())
# initiate SSH connection
ssh_pre.connect(InputVars[0], username=InputVars[1], password=InputVars[2], timeout=10)
# Use invoke_shell to establish an 'interactive session'
Conn = ssh_pre.invoke_shell()
except
# SSH connection failed, lets try a telnet connection.
Conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
# Open Telnet Connection to device
Conn.connect((InputVars[0],23))
except
#Print some error message saying you couldn't connect, blah blah
else
output = str(Conn.recv(10000))
else
output = str(Conn.recv(10000))
# After I establish my connection via object Conn, I can always just pull more data from
# the same connection object regardless of whether it's ssh or telnet.
if DebugVar > 3:
DebugFile.write("Here is the initial response after connecting.\n")
DebugFile.write("<<<1>>>\n"+output+"\n<<<1>>>\n")
Then do a bunch of stuff depending on what is to be collected, device type, etc.
The output I get is this:
Here is the initial response after connecting:
<<<1>>>
b'\r\nUShaBTC-DCTNW-CORE01#'
<<<1>>>
I am trying to get this:
Here is the initial response after connecting.
<<<1>>>
UShaBTC-DCTNW-CORE01#
<<<1>>>
I've tried using formatted string literals, but that keeps leading me down the path for using string slicing, which I am trying to avoid. It's a really simple thing I'm trying to do. Receive a "chunk" (to use a generic term) of data, look in that data for a certain string, keyword, or structure, then send something to the connection, and get another chunk of data.
Yes, what I'm looking for is in what I receive either way. I can still search on it, but at some point I need to write what I received out to a file that is readable, and I'm not having much luck doing that.
Hey one more question....
Apparently Python 3 doesn't support unbufferred file output. As I have shown I dump data to a debug file (depending on the debug level my script runs at). In a crash, it's pretty important to get the last bit of data that I received to figure out why it crashed, so I always wrote data received immediately with no buffering to my debug file, so I could figure out later where it crashed and why. Is there a way to write to a file in Python 3 without buffering? I dropped back to a buffer of 1 (instead of 0) so my script will run in Python 3.x, but I still might not see the last (1 byte I guess, haven't actually been able to find out what the units are for that buffer size).
Appreciate any help.

Just call decode on the byte string:
output = Conn.recv(10000).decode('utf-8')
Though in general, you should not use SSHClient.invoke_shell. Use SSHClient.exec_command. See What is the difference between exec_command and send with invoke_shell() on Paramiko?

Ending infinite loop for a bot

I have created a chat bot for Twitch IRC, I can connect and create commands etc etc, however I cannot use keyboard-interrupt in the command prompt. I suspect it is because it's stuck in this infinite loop, and I don't know how to fix this? I am new to programming btw!
Here is the code I have in my Run.py, openSocket() is defined in another file, basically connection to the server. s = socket.socket.
First part in the while-loop basically just reads the server messages, I think it's pretty straight forward for you guys!
s = openSocket()
joinRoom(s)
readbuffer = ""
while True:
readbuffer = readbuffer + s.recv(1024).decode("utf-8")
temp = str.split(readbuffer, "\n")
readbuffer = temp.pop()
for line in temp:
if "PING" in line:
s.send("PONG :tmi.twitch.tv\r\n".encode("utf-8"))
print("---SENT PONG---")
printMessage(getUser, getMessage, line)
message = getMessage(line)
for key in commands:
command = key
if command in message:
sendMessage(s, commands[command])
((Edit: I also have this problem where the connection to the server seems to time out for whatever reason. I managed to get it keep connection with ping/pong for about 40-45min, but then it disconnected again.
EDIT:
Sorry the original post was super messy. I have created this pastebin with the least amount of code I could use to recreate the problem.
If the IRC chat is inactive it will disconnect, and I can't get it to send 2 pings in a row without any messages in between, not sure if that's because it disconnects before the 2nd ping or because of the 2nd ping.
On at least one occasion it has disconnected even before I got the first ping from the server.
Pastebin: pastebin.com/sXUW50sS

Part of code that you posted doesn't have much to do with problem you described.
This is a guess (although an educated one). In you socket connection you are probably using try: except: and using Pokemon approach (gotta catch 'em all)
Thing here would be to find a line where you are doing something like this:
except:
pass
and change it to:
except (KeyboardInterrupt, SystemExit):
raise
except:
pass
Obviously I'm not trying to say here that your porgram should catch all exceptions and just pass like if nothing happened. Main point is that you are probably already doing that (for i-have-no-idea-why reasons) and you should have special treatment for system errors.

Prompting with previous input from sys.stdin

I have a standard server-client TCP setup. The basic idea is a chat system. Looking at only the client's side of the conversation, the client prompts the user for input with:
sys.stdout.write('<%s> ' % username)
sys.stdout.flush()
using the following logic:
while True:
socket_list = [sys.stdin, s]
read_sockets, write_sockets, error_sockets = select.select(socket_list, [], [])
for sock in read_sockets:
if sock == s:
data = sock.recv(4096)
if data:
output('\a\r%s' % data) #output incoming message
sys.stdout.write('<%s> ' % username) #prompt for input
sys.stdout.flush()
else:
raise SystemExit
else:
msg = getASCII(sys.stdin.readline()) # returns only the ascii
if msg:
s.send(msg)
sys.stdout.write('<%s> ' % username)
sys.stdout.flush())
(Note: truncated snippet. Full code can be found here Linked code has been updated and so is no longer relevant.)
The problem is, when the user is typing and it gets an incoming message from the server, the client outputs the message and prompts for input again. The message that was being typed is still in the stdin buffer but has gone from the screen. If the user presses enter to send the message, the entire message will be sent including what was in the buffer, but on the user's screen, only the second part of the message, the part after the interruption, will be displayed.
I have a possible solution, which is that when I prompt for input, I check if there's anything in the buffer and output that along with the prompt, but I have no idea how to implement it. Any help is appreciated.

To implement your solution, you will have to read from stdin in an unbuffered way. readline() and read() block until an EOL or EOF. You need the data from stdin BEFORE the return key is pressed. To achieve that, this might prove helpful: http://code.activestate.com/recipes/134892-getch-like-unbuffered-character-reading-from-stdin/ When you are about to write data, you could then read from stdin, store it somewhere and output it again after outputting the message. As select won't be called for stdin, make a separate read-thread that reads stdin. Use locks for accessing stdin's data so far.

As an alternative to implementing your own editing line input function as discussed in the question's comments, consider this approach: Change the scrolling region to leave out the screen's bottom line (the user input line) and enter the scrolling region only temporarily to output incoming server messages. That answer contains an example.

The problem seems to be that you are letting the messages from the other user interrupt the typing. I recommend you either only listen to one thing at a time (when the user is typing you let him finish and press enter before listening for remote messages) or you listen for the user's input one key at a time and build up your own buffer (see Polling the keyboard (detect a keypress) in python). The downside to the later approach is that you need to implement key editing, etc. There may be a library that accomplishes this for you.
Note that in most chat programs, the area you type is in a separate window/screen region than where you are seeing the messages. All messages (yours as well as others) show up when complete in this message area. Perhaps you can use just display messages (independent of input) somewhere else on the screen.

Python telnet connection successful

I'm wondering if it is possible to know if my telnet connection is successful?
So, if I'm connected to my switch and if I could write commands
telnet = telnetlib.Telnet(IP)
telnet.read_until(b"User Name:")
telnet.write(b"LOGIN\n")
telnet.read_until(b"Password:")
telnet.write(b"PASSWORD\n")
# Here I want to know if I'm connected

Don't use read_all if you plan on writing something after authentication. It blocks the connection until EOF is reached / connection is closed.
First check the output telnet server is giving when an authentication is successful using putty or something else. read_untill the string to be matched after authentication.
telnet = telnetlib.Telnet(IP)
telnet.read_until(b"User Name:")
telnet.write(b"LOGIN\n")
telnet.read_until(b"Password:")
telnet.write(b"PASSWORD\n")
telnet.read_untill("string to be matched")

You could go this way:
def is_connected(telnet_obj ):
answer = telnet_obj.read_all()
if "connected" in answer: #this test condition is not real is an example
return True
else:
return False
If you observe what yout router/switch returns you can test for that condition. In this case testing for the presence of a string or lack in the answer variable.

Python client side in chat

I have a problem while trying to build the client side of a chat. I just in the begining, this is my code:
import socket
my_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
my_socket.connect(("10.10.10.69",1234))
while True:
message=raw_input("your message: ")
if(message=="quit"):
my_socket.close()
break
my_socket.send(message)
data=my_socket.recv(1024)
print "message from server:" , data
The problem is the raw_input. When a user sends a message the other users are stacked on the raw_input so only when they sends a message too they get the new messages.
How can I fix it (without using threads)?

As I commented, use select.select if your chat client is running in Unix.
For example:
import socket
import sys
import select
my_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
my_socket.connect(("10.10.10.69",1234))
sys.stdout.write('your message: ')
sys.stdout.flush()
while True:
r, w, x = select.select([sys.stdin, my_socket], [], [])
if not r:
continue
if r[0] is sys.stdin:
message = raw_input()
if message == "quit":
my_socket.close()
break
my_socket.send(message)
sys.stdout.write('your message: ')
sys.stdout.flush()
else:
data = my_socket.recv(1024)
print "message from server:" , data

raw_input holds up the thread it's in, so you can't retrieve messages in the same thread while waiting for it. Thus, I recommend using 2 threads, one which receives new messages (say every ten seconds) and one which sends messages (using code similar to the existing code).
If you're not committed to raw_input and are really against threads, you might want to look into the select module.

There are suggestions to use select() for stdin, but seems they aren't fixing the main problem. With select, imagine the local user is entering a line, and in a middle of this process your program outputs another user message, and, the editing of local input will be screwed. Either you don't allow to show new messages during entering the current one (why?) or screen is messed up. If you do this in a terminal, you should go using curses or some more powerful tool; with curses you can at least split input and output into different screen areas. Of course, a graphics library (like wxWidgets) is even more generally suitable for a user interface, if it's allowed in your case.