I am using the Python module telnetlib to create a telnet session (with a chess server), and I'm having an issue I really can't wrap my brain around. The following code works perfectly:
>>> f = login("my_server") #code for login(host) below.
>>> f.read_very_eager()
This spits out everything the server usually prints upon login. However, when I put it inside a function and then call it thus:
>>> def foo():
... f = login("my_server")
... return f.read_very_eager()
...
>>> foo()
I get nothing (the empty string). I can check that the login is performed properly, but for some reason I can't see the text. So where does it get swallowed?
Many thanks.
For completeness, here is login(host):
def login(host, handle="guest", password=""):
try:
f = telnetlib.Telnet(host) #connect to host
except:
raise Error("Could not connect to host")
f.read_until("login: ")
try:
f.write(handle + "\n\r")
except:
raise Error("Could not write username to host")
if handle == "guest":
f.read_until(":\n\r")
else:
f.read_until("password: ")
try:
f.write(password + "\n\r")
except:
raise Error("Could not write password to host")
return f
The reason why this works when you try it out manually but not when in a function is because when you try it out manually, the server has enough time to react upon the login and send data back. When it's all in one function, you send the password to the server and never wait long enough for the server to reply.
If you prefer a (probably more correct) technical answer:
In file telnetlib.py (c:\python26\Lib\telnetlib.py on my Windows computer), function read_very_eager(self) calls self.sock_avail() Now, function sock_avail(self) does the following:
def sock_avail(self):
"""Test whether data is available on the socket."""
return select.select([self], [], [], 0) == ([self], [], [])
What this does is really simple: if there is -anything- to read from our socket (the server has answered), it'll return True, otherwise it'll return False.
So, what read_very_eager(self) does is: check if there is anything available to read. If there is, then read from the socket, otherwise just return an empty string.
If you look at the code of read_some(self) you'll see that it doesn't check if there is any data available to read. It'll try reading till there is something available, which means that if the server takes for instance 100ms before answering you, it'll wait 100ms before returning the answer.
I'm having the same trouble as you, unfortunately the combination of select.select, which I have in a while loop until I am able to read, and then calling read_some() does not work for me, still only reading 1% of the actual output. If I put a time.sleep(10) on before I read and do a read_very_eager() it seems to work...this is a very crude way of doing things but it does work..I wish there was a better answer and I wish I had more reputation points so I could respond to user387821 and see if he has any additional tips.
Related
I'm trying to write a function that downloads and/or updates files via GitHub based on their age. I broke it into two functions:
update_data(), which handles logic tied to checking whether the local file exists and if so, determines the local file's "age" by grabbing the remote file's last-modified date and comparing it against the local file.
download_file(), which is called INSIDE of update_data(), it hosts the logic of downloading the actual file
BOTH functions use requests.get() to download something and it can raise a ConnectionError if there's something wrong with the Internet connection at the time of download.
Here's the issue: I tried including the try/except logic only in update_data(), thinking that since download_file() is inside it, then if a ConnectionError is raised inside download_file() then surely that will be taken care of. Apparently I'm wrong, and download_file() ignores the try/except error checking of whatever function it's in. Is there a way to make the download_file() function follow the try/except rules of the function it's nested in? And if not, how else should I handle what I'm trying to do in a clean and concise way?
Here's a snippet of update_data(). I'm not including what's in download_file() since all that matters is it lacks the try/except part that this function does have.
for file_name, remote_path in file_paths.items():
while True:
try:
local_file_date = datetime.fromtimestamp(os.path.getmtime(file_name))
api_url = f"https://api.github.com/repos/owid/covid-19-data/commits?path={remote_path}&per_page=1"
last_updated = requests.get(api_url).json()[0]['commit']['committer']['date']
remote_file_date = datetime.strptime(last_updated, "%Y-%m-%dT%H:%M:%SZ")
delta = (remote_file_date - local_file_date).days
if delta >= 0:
choice = input(f"{file_name} is OUTDATED. It's roughly {str(delta + 1)} days old. "
"Would you like to update it? (Y/N): ")
if "y" in choice.lower():
download_file(file_name, remote_path)
break
except ConnectionError:
print("CAN'T CONNECT TO THE INTERNET! Your files might be out of date."
"Try checking your internet connection or Github's status.")
return
I have created a chat bot for Twitch IRC, I can connect and create commands etc etc, however I cannot use keyboard-interrupt in the command prompt. I suspect it is because it's stuck in this infinite loop, and I don't know how to fix this? I am new to programming btw!
Here is the code I have in my Run.py, openSocket() is defined in another file, basically connection to the server. s = socket.socket.
First part in the while-loop basically just reads the server messages, I think it's pretty straight forward for you guys!
s = openSocket()
joinRoom(s)
readbuffer = ""
while True:
readbuffer = readbuffer + s.recv(1024).decode("utf-8")
temp = str.split(readbuffer, "\n")
readbuffer = temp.pop()
for line in temp:
if "PING" in line:
s.send("PONG :tmi.twitch.tv\r\n".encode("utf-8"))
print("---SENT PONG---")
printMessage(getUser, getMessage, line)
message = getMessage(line)
for key in commands:
command = key
if command in message:
sendMessage(s, commands[command])
((Edit: I also have this problem where the connection to the server seems to time out for whatever reason. I managed to get it keep connection with ping/pong for about 40-45min, but then it disconnected again.
EDIT:
Sorry the original post was super messy. I have created this pastebin with the least amount of code I could use to recreate the problem.
If the IRC chat is inactive it will disconnect, and I can't get it to send 2 pings in a row without any messages in between, not sure if that's because it disconnects before the 2nd ping or because of the 2nd ping.
On at least one occasion it has disconnected even before I got the first ping from the server.
Pastebin: pastebin.com/sXUW50sS
Part of code that you posted doesn't have much to do with problem you described.
This is a guess (although an educated one). In you socket connection you are probably using try: except: and using Pokemon approach (gotta catch 'em all)
Thing here would be to find a line where you are doing something like this:
except:
pass
and change it to:
except (KeyboardInterrupt, SystemExit):
raise
except:
pass
Obviously I'm not trying to say here that your porgram should catch all exceptions and just pass like if nothing happened. Main point is that you are probably already doing that (for i-have-no-idea-why reasons) and you should have special treatment for system errors.
I'm creating an instant messenger program for my school's common drive. I have everything working except for on small detail. In the code below it checks for a new message from a friend and prints the last message they sent. If there are no messages it says so. The problem is when it moves to the next step of the code it waits for the user to put in an input. Until you give an input it won't let you receive any more messages because the program stops reading and searching the while loop and gets caught on the input statement. I want to know if there is anyway to make an input statement optional. To say that it doesn't require an input but if there is an input it will send it and do it's thing. I just can't seem to figure out a way to make the input statement optional. Any ideas or working code would be greatly appreciated. If you need the entire code I don't have a problem with sending it to you or posting it. This is the only bit of code that should really matter for this problem though.
LastMessage = ""
while Message:
Path = "Message"+SendTo+"-"+UserName+".txt"
if path.isfile(Path):
LoadMessage = open(Path, "rb")
NewMessage = pickle.load(LoadMessage)
LoadMessage.close()
else:
NewMessage = "Sorry, No messages found"
if LastMessage != NewMessage:
LastMessage = NewMessage
print(NewMessage)
print("")
SendMessage = raw_input() #--- This is where it gets caught up! ---
Save = open("Message"+UserName+"-"+SendTo+".txt", "wb")
pickle.dump(SendMessage, Save)
Save.close()
You have two main options as I see it:
Simultaneous input and checking (various options, search for e.g. threading or multiprocessing from the standard library); or
Input with timeout and loop (see e.g. How to set time limit on raw_input).
So it sounds like you want to do two separate things at the same time - look for input from a user and poll for new messages from other users. Jonrsharpe gives threading as his first option to solve this and I agree its the most straightforward. What you need to do is something like this:
import threading
class InputMessageThread(threading.Thread):
def run(self):
SendMessage = raw_input() # This thread will hang here for input but thats
# OK as original thread will keep going
Save = open("Message"+UserName+"-"+SendTo+".txt", "wb")
pickle.dump(SendMessage, Save)
Save.close()
inputthread = InputMessageThread()
inputthread.start()
# rest of your code here
While you are at it though you might want to look at some other issues. For example if I understand what you are trying to do correctly you are going to have a file containing a message from a source user to a destination user. But if the source user sends a second message before this file gets processed then the first message will be overwritten. In practice you may never see this but some sort of handshaking to make sure the message has actually been sent before you allow the next to be written would be a more robust approach.
I am using Python telnetlib for taking the "running config" output from router.How to store the "show running-config" output in a variable.And print the variable.My requirement is the each and every output will display in the console when executing each and every line of the code.Is there any option to aviod these print statements.
import telnetlib
#import getpass
ipaddr = "10.1.1.1"
passwd = "abcd123"
tn = telnetlib.Telnet(ipaddr)
if Password:
try:
print (tn.write (password + "\n"))
print(tn.read_until("Router>"))
print(tn.write("enable\n"))
print(tn.read_until("Password:"))
print(tn.write(passwd + "\n"))
print(tn.read_until("Router#"))
print(tn.write("show clock\n"))
print(tn.read_until("#"))
print(tn.write("show running-config\n"))
print(tn.write("\040\n"))
print(tn.write("\040\n"))
print(tn.write("\040\n"))
print(tn.read_until("#"))
print(tn.write("logout\n"))
print(tn.read_until(">"))
print tn.close
If I understand you correctly you wish to print out to your local console the output of each command which you run on the remote console. I am not sure why it needs to be synchronous except you say that is a requirement. You might want to make sure you understand the requirements. In any case, since your requirement is that the output be printed, you don't need to print your input...
I highly recommend storing the output into a variable even if you need to print it immediately simply because I see no benefit of retrieving the data unless you are going to act on the data and if you merely print the data you cannot act on it. Store it in a variable and then it can be printed as well as acted upon. I doubt the human eye would be able to tell the difference in storing it and then writing it all at once rather than piecemeal.
Your try block, as written, will never happen because you have to read from the telnet session first before you can evaluate if 'Password:' is on the remote console.
Some further suggestions:
First, write terminal length 0, that will avoid having to handle paused output.
Second, since I am lazy, any variables I know I am only using to pass to the remote unit I save with a newline character.
Third, always give it a timeout or else it runs the risk of waiting forever for a match that might never come.
Fourth, have a look at Telnet.expect(list, [timeout]). I find it far more useful than a simple read_until; it allows you to look for multiple responses and act on each of them accordingly. It is quite useful for handling errors. It returns a three item tuple that represents the index of the matched item (-1 if no match) as well as the matched text (or everything in the buffer if no match).
Fifth, write a decorator for your telnet session to log in. You know it will be used at least once every time you interact with a remote unit, and more if you are loading new firmware. Develop a library of functions that you can reuse rather than writing it out each time. Lazy is good.
import telnetlib
import sys
ipaddr = "10.1.1.1"
passwd = "abcd123"
def login(tn):
global passwd
passwd=passwd+'\n'
def error_check(tmp):
if tmp[0]==-1:
print "Unexpected response"
print "tmp[2]
sys.exit()
tmp=tn.expect(["Password:",], 5)
error_check(tmp)
tn.write(passwd)
tmp=expect([">",],5)
error_check(tmp)
tn.write('en\n')
tmp=expect(["Password", "#'],5)
error_check(tmp)
if tmp(0)==0: #if someone left enable unlocked, don't send a password
tn.write(passwd)
tmp=expect(["#',],5)
error_check(tmp)
tn = telnetlib.Telnet(ipaddr)
login(tn)
tn.write('terminal length 0')
tmp=expect(["#',],5)
tn.write('sho clock')
now=tn.expect(["#",], 5)
print now
tn.write('sho run')
print run
cfg=tn.expect(["#",], 5)
tn.write("logout\n"))
bye=tn.expect([">",], 5)
print bye
tn.close()
I'm trying to develop a very simple proof-of-concept to retrieve and process data in a streaming manner. The server I'm requesting from will send data in chunks, which is good, but I'm having issues using httplib to iterate through the chunks.
Here's what I'm trying:
import httplib
def getData(src):
d = src.read(1024)
while d and len(d) > 0:
yield d
d = src.read(1024)
if __name__ == "__main__":
con = httplib.HTTPSConnection('example.com', port='8443', cert_file='...', key_file='...')
con.putrequest('GET', '/path/to/resource')
response = con.getresponse()
for s in getData(response):
print s
raw_input() # Just to give me a moment to examine each packet
Pretty simple. Just open an HTTPS connection to server, request a resource, and grab the result, 1024 bytes at a time. I'm definitely making the HTTPS connection successfully, so that's not a problem at all.
However, what I'm finding is that the call to src.read(1024) returns the same thing every time. It only ever returns the first 1024 bytes of the response, apparently never keeping track of a cursor within the file.
So how am I supposed to receive 1024 bytes at a time? The documentation on read() is pretty sparse. I've thought about using urllib or urllib2, but neither seems to be able to make an HTTPS connection.
HTTPS is required, and I am working in a rather restricted corporate environment where packages like Requests are a bit tough to get my hands on. If possible, I'd like to find a solution within Python's standard lib.
// Big Old Fat Edit
Turns out in my original code I had simply forgot to update the d variable. I initialized it with a read outside the yield loop and never changed it in the loop. Once I added it back in there it worked perfectly.
So, in short, I'm just a big idiot.
Is your con.putrequest() actually working? Doing a request with that method requires you to also call a bunch of other methods as you can see in the official httplib documentation:
http://docs.python.org/2/library/httplib.html
As an alternative to using the request() method described above, you
can also send your request step by step, by using the four functions
below.
putrequest()
putheader()
endheaders()
send()
Is there any reason why you're not using the default HTTPConnection.request() function?
Here's a working version for me, using request() instead:
import httlplib
def getData(src, chunk_size=1024):
d = src.read(chunk_size)
while d:
yield d
d = src.read(chunk_size)
if __name__ == "__main__":
con = httplib.HTTPSConnection('google.com')
con.request('GET', '/')
response = con.getresponse()
for s in getData(response, 8):
print s
raw_input() # Just to give me a moment to examine each packet
You can use the seek command to move the cursor along with your read.
This is my attempt at the problem. I apologize if I made it less pythonic in process.
if __name__ == "__main__":
con = httplib.HTTPSConnection('example.com', port='8443', cert_file='...', key_file='...')
con.putrequest('GET', '/path/to/resource')
response = con.getresponse()
c=0
while True:
response.seek(c*1024,0)
data =d.read(1024)
c+=1
if len(data)==0:
break
print data
raw_input()
I hope it is at least helpful.