Okay, so in some experimentation with sockets and networking, I've set up a very basic chat client/server that runs on Unix flawlessly. Now I'm hitting some errors in the translation to win32. In preface, I understand that select.select() on Windows won't accept socket objects and have (I think) compensated coherently by not passing it the socket object, but the socket number. However, the script still hangs on the select.select() function and I have no idea why. The script only hangs until the server receives a message, after which it allows the client to send messages, however the client can't receive messages from the server, no matter what. I'm trying to troubleshoot these two bugs as best as I can, but my research has turned up dry. Here's the problem code, thanks in advance.
while True:
socket_list.append(s)
read_sockets, write_sockets, error_sockets = select.select(socket_list, [], [], 20)
if not (read_sockets or write_sockets or error_sockets):
if afk == False:
s.send('[Status]: '+str(_user)+' has gone afk.\n')
sys.stdout.write('\n[+]: You have gone afk.\n')
afk = True
prompt()
for sock in read_sockets:
print ('Starting for sock in read_sockets') #DEBUG#
if sock == s:
print ('Getting here.') #DEBUG#
data = sock.recv(4096)
if not data:
sys.stdout.write('[!]: Disconnected from chat server by server.\n'+W)
choice = raw_input('[*]: Press Enter to continue.')
_logic()
else:
sys.stdout.write(data)
else:
# Rest of the Program (Runs correctly) #
It sounds like you forgot to set the sockets non-blocking. Like pretty much all status reporting functions, select does not make future guarantees. You also need to handle the case where read returns a "would block" indication. You can't rely on select to predict the results of future read operations.
Related
I'm a beginner in the field of sockets and lately trying ti create a terminal chat app with that.I still have trouble understanding setblocking and select functions
"This is the code i have taken from a website i'm reading from and in the code if there is nothing in data, how does it mean that the socket has been disconnected and please also do explain what affect the setblocking in the server or the client does.I have read somewhere that setblocking allows to move on if the data has been fully not recieved,i'm not quite satisfied with the explaination.Please explain in simple words "
import select
import socket
import sys
import Queue
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setblocking(0)
server_address = ('localhost', 10000)
server.bind(server_address)
server.listen(5)
inputs = [ server ]
outputs = [ ]
message_queues = {}
while inputs:
readable, writable, exceptional = select.select(inputs, outputs, inputs)
for s in readable:
if s is server:
connection, client_address = s.accept()
connection.setblocking(0)
inputs.append(connection)
message_queues[connection] = Queue.Queue()
else:
data = s.recv(1024)
if data:
message_queues[s].put(data)
if s not in outputs:
outputs.append(s)
else:
if s in outputs:
outputs.remove(s)
inputs.remove(s)
s.close()
if there is nothing in data, how does it mean that the socket has been disconnected
The POSIX specification of recv() says:
Upon successful completion, recv() shall return the length of the message in bytes. If no messages are available to be
received and the peer has performed an orderly shutdown, recv() shall return 0. …
In the Python interface, return value 0 corresponds to a returned buffer of length 0, i. e. nothing in data.
what affect the setblocking in the server or the client does.
The setblocking(0) sets the socket to non-blocking, i. e. if e. g. the accept() or recv() cannot be completed immediately, the operation fails rather than blocks until complete. In the given code, this can hardly happen, since the operations are not tried before they are possible (due to the use of select()). However, the example is bad in that it includes output in the select() arguments, resulting in a busy loop since output is writable most of the time.
Short description:
Client sends server data via TCP socket. Data varies in length and is strings broken up by the delimiter "~~~*~~~"
For the most part it works fine. For a while. After a few minutes data winds up all over the place. So I start tracking the problem and data is ending up in the wrong place because the full thing has not been passed.
Everything comes into the server script and is parsed by a different delimiter -NewData-* then placed into a Queue. This is the code:
Yes I know the buffer size is huge. No I don't send data that kind of size in one go but I was toying around with it.
class service(SocketServer.BaseRequestHandler):
def handle(self):
data = 'dummy'
#print "Client connected with ", self.client_address
while len(data):
data = self.request.recv(163840000)
#print data
BigSocketParse = []
BigSocketParse = data.split('*-New*Data-*')
print "Putting data in queue"
for eachmatch in BigSocketParse:
#print eachmatch
q.put(str(eachmatch))
#print data
#self.request.send(data)
#print "Client exited"
self.request.close()
class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
pass
t = ThreadedTCPServer(('',500), service)
t.serve_forever()
I then have a thread running on while not q.empty(): which parses the data by the other delimiter "~~~*~~~"
So this works for a while. An example of the kind of data I'm sending:
2016-02-23 18:01:24.140000~~~*~~~Snowboarding~~~*~~~Blue Hills~~~*~~~Powder 42
~~~*~~~Board Rental~~~*~~~15.0~~~*~~~1~~~*~~~http://bigshoes.com
~~~*~~~No Wax~~~*~~~50.00~~~*~~~No Ramps~~~*~~~2016-02-23 19:45:00.000000~~~*~~~-15
But things started to break. So I took some control data and sent it in a loop. Would work for a while then results started winding up in the wrong place. And this turned up in my queue:
2016-02-23 18:01:24.140000~~~*~~~Snowboarding~~~*~~~Blue Hills~~~*~~~Powder 42
~~~*~~~Board Rental~~~*~~~15.0~~~*~~~1~~~*~~~http://bigshoes.com
~~~*~~~No Wax~~~*~~~50.00~~~*~~~No Ramps~~~*~~~2016-02-23 19:45:00.000000~~~*~
Cutting out the last "~~-15".
So the exact same data works then later doesn't. That suggests some kind of overflow to me.
The client connects like this:
class Connect(object):
def connect(self):
host = socket.gethostname() # Get local machine name
#host = "127.0.0.1"
port = 500 # Reserve a port for your service.
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#print('connecting to host')
sock.connect((host, port))
return sock
def send(self, command):
sock = self.connect()
#recv_data = ""
#data = True
#print('sending: ' + command)
sock.sendall(command)
sock.close()
return
It doesn't wait for a response because I don't want it hanging around waiting for one. But it closes the socket and (as far as I understand) I don't need to flush the socket buffer or anything it should just be clearing itself when the connection closes.
Would really appreciate any help on this one. It's driving me a little spare at this point.
Updates:
I'm running this on both my local machine and a pretty beefy server and I'd be pushed to believe it's a hardware issue. The server/client both run locally and sockets are used as a way for them to communicate so I don't believe latency would be the cause.
I've been reading into the issues with TCP communication. An area where I feel I'll quickly be out of my depth but I'm starting to wonder if it's not an overflow but just some king of congestion.
If sendall on the client does not ensure everything is sent maybe some kind of timer/check on the server side to make sure nothing more is coming.
The basic issue is that your:
data = self.request.recv(163840000)
line is not guaranteed to receive all the data at once (regardless of how big you make the buffer).
In order to function properly, you have to handle the case where you don't get all the data at once (you need to track where you are, and append to it). See the relevant example in the Python docs on using a socket:
Now we come to the major stumbling block of sockets - send and recv operate on the network buffers. They do not necessarily handle all the bytes you hand them (or expect from them), because their major focus is handling the network buffers. In general, they return when the associated network buffers have been filled (send) or emptied (recv). They then tell you how many bytes they handled. It is your responsibility to call them again until your message has been completely dealt with.
As mentioned, you are not receiving the full message even though you have a large buffer size. You need to keep receiving until you get zero bytes. You can write your own generator that takes the request object and yields the parts. The nice side is that you can start processing messages while some are still coming in
def recvblocks(request):
buf = ''
while 1:
newdata = request.recv(10000)
if not newdata:
if buf:
yield buf
return
buf += newdata
parts = buf.split('*-New*Data-*')
buf = parts.pop()
for part in parts:
yield part
But you need a fix on your client also. You need to shutdown the socket before close to really close the TCP connection
sock.sendall(command)
sock.shutdown(request.SHUT_RDWR)
sock.close()
OK, so i have a pretty simple turn based application (game).
Each user sends a request on the server and then wait for response. The important is, that only ONE user makes the request on the server (send his actions) and all other users are just waiting, if the server sends them some data, so they must always check (in loop) if something is coming from server.
I´m using the built-in module of python "socket" and the way i manage the clients is this: For every user i create one thread, in which runs infinite loop, until the application ends, which checks for request (if it is the turn of the user) or checks if it got anything to sent to the other users. Now let´s move to Clients. Every client has again one thread with infinite loop, waiting for data from server.
The problem is that the GUI is made in PyQt4.4, where i cant get into the loop of the PyQt itself (although i have seen, that it is possible to do this with twisted, but then i would have to rewrite my code) so i have to use the thread, that means i can use classic python threading library or QThread, but QThread sadly doesn´t have any Events, which are pretty crucial because i want always wait after the message from the server for the response of the program, so i can send response to the server again. On the other hand, I am not sure, if i can use Thread from threading to emit signals. So which one is the right one to go?
Btw: is actually ok, to run the infinite client and server side loop? Because in every tutorial I have seen, the client close the connection right after he gets his answer.
EDIT:
here´s some of the code
Server side loop for connection with client:
while self.running:
if self.is_on_turn == p and self.reply is not None:
cmd = conn.recv(1024)
if cmd == '':
conn.close()
return
cmd = eval(cmd)
if self.is_on_turn != p: # User is not on turn
print "END"
conn.sendall("END")
else:
self.queue.put(cmd)
ev.wait() # Here works another program with the message and decide what to do next
ev.clear() #
conn.sendall(str(self.message))
if self.reply:
ev.wait() #
ev.clear() #
if self.reply:
r = conn.recv(1024)
if r == '':
conn.close()
return
self.queue.put(eval(r))
ev.wait() #
ev.clear() #
conn.sendall(str(self.message))
conn.close()
Client side loop:
def main_loop(self, pipe, conn, e, o): #e is event, o is bool (whether the client has to answer back to the server)
is_on_turn = conn.recv(4096)
pipe.send((is_on_turn))
while True:
if is_on_turn == h or o.value and o.value is not None:
conn.send(str(pipe.recv()))
pipe.send(eval(conn.recv(4096)))
e.wait()
e.clear()
The pipe is there, because I made it in multiprocessing at first, there should the emit signal for the PyQt instead, but as I said, I am not sure which approach to use
So the result is, that I have just used QTcpServer and QTcpSocket as sugessted by ekhumoro, which resulted in much cleaner code and easier management :)
I've been looking around all day, but I haven't been able to fix the problem I've got with my chat client here.
Here's the issue: I recently decided to change the client so that it would allow the user to input any message they wanted without having to wait for a reply first (blocking or something stops my program until a reply is in)
I decided to use the select.select module to do so, but after writing a couple different versions of my client today trying to get it to work, I keep getting stuck at this one point.
Whenever I enter a message, the loop gets stuck somewhere, (probably at .recv data)
how can I fix this? Nothing I try gets it to go by that.
Edit: To be more clear, when I run, I get to the point where I input the message, hit enter and then nothing happens at all. It just stays running like that.
from socket import *
import select
import sys #because why not?
print("New Chat Client Using Select Module")
HOST = input("Host: ")
PORT = int(input("Port: "))
s = socket(AF_INET,SOCK_STREAM)
print("Trying to connect....")
s.connect((HOST,PORT))
s.setblocking(0)
# Not including setblocking(0) because select handles that.
print("You just connected to",HOST,)
# Lets now try to handle the client a different way!
while True:
Incoming_data = [s]
Exportable_data = []
Exceptions = []
User_input = input("Your message: ")
rlist,wlist,xlist = select.select(Incoming_data,Exportable_data,Exceptions)
if User_input == True:
Exportable_data += [User_input]
for i in rlist:
data = i.recv(1024)
if data == "":
continue
for i in wlist:
if Exportable_data is True:
i.send(Exportable_data)
continue
Is there any way I can override the blocking (that I presume is the problem) when it's set to receive data? Won't s.setblocking(0) make it so it won't block(?) (With or without it still gets stuck)
Thanks for taking a look
I think you should have separate thread or process which will interact with your socket and another thread, which will accept user input and print chat messages. Interaction between the threads you can do using 2 queues: for incoming and outgoing messages.
Have a look at threading and queue modules.
I'm currently working on a simple Python script that connects to an IP/Port and lets you preform commands. Kinda like telnet. I want it to connect to lets say MySQL(Just an example) and execute MySQL commands like you would with Telnet or NetCat.
But lets say I connect to a service, I can't execute commands. I'm trying to connect to a custom port on localhost. Let me show the code:
try:
sock.connect((host, port))
recvdata = sock.recv(socksize)
print("Type your commands now")
while(1): #Setting a loop
print(recvdata)
userInput = raw_input(">")
sock.send(userInput + '\r\n\r\n')
print(recvdata)
except KeyboardInterrupt:
print("\nConnection closed")
sys.exit()
So this is what I get when I connect:
:: !Test service! ::
If I type anything, I just get the banner again. The same banner over and over no matter what.
Your recvdata = sock.recv(socksize) is outside of the recv loop. I think, a good introduction to socket programming in Python might be helpful to you. Your previous question and this one would have been easily spotted by you using that introduction tutorial. Have a look at an example socket echo server/client (at this site), which you can modify further and proceed with your task. Run two separate terminals, run the echo server at one end and use the echo client with address pointint to 127.0.0.1.
Well, no wonder. You never actually call sock.recv on the communicating socket after you send your input.
Prior to the loop, recvdata contains the "banner" from the initial connect. But once you're in the loop, this variable is never changed.
I believe what you want is:
sock.connect((host, port))
recvdata = sock.recv(socksize)
print(recvdata)
print("Type your commands now")
while(1): #Setting a loop
userInput = raw_input(">")
sock.send(userInput + '\r\n\r\n')
recvdata = sock.recv(socksize)
print(recvdata)