How to know if a non-blocking socket is closed? - python

I'm trying to write my own TCP Non-Blocking Server to handle multiple long lasting socket connections rather than opening many threads to handle them.
I've written my over-complicated, hard to use syntax but have the issue forms when I'm trying to detect a closed socket.
In a normal threaded TCP Socket Server I would use detected a b'' from the socket.read(size) function, However this is not possible with a nonblocking socket as it will always return a BlockingIOError
I have also tried catching theese following events
except BrokenPipeError:
conn.abort()
except ConnectionResetError:
conn.abort()
except ConnectionAbortedError:
conn.abort()
except socket.error:
conn.abort()
(conn is a class that houses the client socket and address from socket.accept())
I'm unsure what to do, but here is a deeply simplified extract from my code:
def loop_listen(self):
while self.running == True:
cr, addr = self.server.accept()
crs = SocketHandler(self, cr, addr)
self.client_handler(crs)
self.connections.append(crs)
crs.events["open"]()
crs.cr.setblocking(0)
def loop_recv(self):
while self.running == True:
time.sleep(self.poll_time)
for conn in self.connections:
try:
data = conn.cr.recv(self.poll_size)
print(data)
if (data == b''):
conn.abort()
except BlockingIOError:
data = None
except BrokenPipeError:
conn.abort()
except ConnectionResetError:
conn.abort()
except ConnectionAbortedError:
conn.abort()
except socket.error:
conn.abort()
if (data != None):
conn.events["msg"](data)
(Both loops are separate threads)
And incase you wanted it, here is the conn class
class SocketHandler:
def __init__(self, server, cr, addr):
self.server = server
self.cr = cr
self.addr = addr
self.events = {"msg": emptyCallback, "close": "emptyCallback","open":emptyCallback}
self.cache = b""
def message(self, func):
self.events["msg"] = func
def close(self, func):
self.events["close"] = func
def open(self, func):
self.events["open"] = func
def send(self, data):
self.cr.send(data)
def abort(self):
self.cr.close()
self.events["close"]()
self.server.connections.remove(conn)
This works fine on Windows but on Ubuntu it does not call the conn.abort().
Any help would be greatly appreciated.
Thanks, Sam.

The official way to detect a closed connection on a non-blocking socket is exactly the same as blocking sockets. They return empty data from recv().
Example:
# Server
import socket
import time
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('localhost', 12345))
s.listen(1)
while True:
conn, addr = s.accept()
conn.setblocking(0)
print("New connection from " + str(addr) + ".")
while True:
try:
data = conn.recv(1024)
if not data:
break
print("Received:", data)
except BlockingIOError:
time.sleep(0.001)
print("Closed.")
# Client
import socket
import time
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('localhost', 12345))
for i in range(5):
time.sleep(0.3)
s.send(str(i).encode('utf-8'))
s.close()
There is one special case where this won't work, as described in the official docs, section When Sockets Die. It happens, when sockets don't shut down gracefully. There basically is no way for recv() to detect when a socket is dead without a graceful shutdown. It might be that this is what you are seeing.
There are multiple ways to resolve that. For one, create some kind of timeout that closes and discards a socket if it didn't receive a message for a sensible amount of time. Secondly, you could actively send messages. Detecting a dead socket is much easier for send() than for recv().
Further, this works on Linux. I didn't test it on Windows. The internal implementation of the sockets class is very platform dependent, so it might be a Windows bug.

Related

Python socket: Errno 9 Bad file descriptor when recv

My problem is like this. I would need to create a client program for quick trial that can send out some commands and listen to data(after which I would need to parse it) from the same socket. So I have created two threads(one to issue command (not shown here), the other to listen to data) to handle this after I created the sockets and connect out to the server. Server is written in other langauges.
As the same socket is to be used, I thought that the socket should be set to be unblocking
the socket after creation is send to the thread as an arguement.
I tried to run the python program. And there is a problem of OSError: [Errno 9] Bad file descriptor. I have narrowed down the problem. It got to do with the recv function.
Most of the solution down in the forum seem to point to socket closure as the main problem but i really could not see how since the socket closure was placed out of while loop.
So need somebody help to point to the problem. And here is my code (as below)
import socket
import errno
import sys
import threading
HOST = "192.168.50.35"
PORT = 2356
def listener(sock, q):
print("status thread created")
while q != True:
try:
data = sock.recv(1024)
except socket.error as socketerr:
if socketerr == errno.EAGAIN or socketerr == errno.EWOULDBLOCK:
sleep(1)
print('Data is unavailable')
continue
else:
print(socketerr)
sys.exit(1)
else:
print(f" new {data!r} recieved")
### processed data
### some processing set but not shown here
continue
sock.close()
def connect():
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
try:
s.connect((HOST, PORT))
s.setblocking(0)
s.settimeout(1)
except socket.error as socketerr:
print("Error: ", socketerr)
return s
if __name__ == '__main__':
print_machine_info()
q = False
s = connect()
print("socket created")
t1 = threading.Thread(target=status, args=(s,q))
t1.daemon = False
t1.start()

Getting a ConnectionResetError when client script is stopped on UDP socket

I am making a script that will allow multiple clients to see live camera footage from the server script, this all works fine until one of the client scripts is closed, then a ConnectionResetError is raised, To avoid this I used a try and except block to catch the ConnectionResetError but the same error is raised every time after a connection is lost. Using just socket.recv stops the ConnectionResetError but socket.recv does not return the senders address which is required for the script to send the video stream back to the client.
Server:
host = "0.0.0.0"
port = 5000
buffer_size = 1024
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(("", port))
listeners = list() # the addresses of the clients that want the video stream
def handle_queue(sock):
while True:
try:
message, address = sock.recvfrom(buffer_size) # block the thread until a packet arrives
print(address)
message = str(message, "utf-8") # decode the message
if message == "join":
listeners.append(address) # add the list of listeners
else:
print("unknown queue msg: ", message)
except ConnectionResetError:
print("The connection was forcefully quit")
queue_handler_thread = Thread(target=handle_queue, args=(sock,), daemon=True)
queue_handler_thread.start() # start the queue
the script then uses sock.sendto() for each address in the listeners list
Client:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(bytes("join","utf-8"), (host, port))
while True:
data, address = sock.recvfrom(max_length) # block main thread until a packet is received
I believe what you are looking for is socket.getpeername().
This will return the remote address to which the socket is connected.
Your approach in handling the data in the wire whilst the connection is closed/lost is the right way. Handling via try/catch or using a recv() or recvfrom() method to wait for a response before closing the socket.
Since posting this i have discovered that it is infact TCP and this will not work for the project I am trying to do. Any more guidance and help would be appreciated, I cant seem to find any examples of Multiclient UDP Servers that run through one socket.
I partially resolved my system, after using the Example provided by AztCrw4282.
I was able to find a solution, I instead used the socket.accept() aproach, I am unsure as to whether this is UDP or TCP (I want UDP but with the connect system I think that is the handshake for TCP, but I am not 100% sure), but it works for now.
The client has to connect to the server then the server will accept or deny the connection, upon accepting a thread will be created for that clients connection to be managed. If any errors are thrown when interacting with that client their connection will be closed.
Server
try:
ServerSocket.bind((host, port))
except socket.error as e:
print(str(e))
print('Waiting for a Connection..')
ServerSocket.listen(5)
connections = list()
def threaded_client(connection):
connection.send(str.encode('Welcome to the Server\n'))
while True:
try:
data = str(connection.recv(2048),"utf-8") # this needs to be try catched
print("Packet Recv: ", data)
if data == "join":
print("Client Joined")
if data == "quit":
break
if not data:
break
except ConnectionResetError:
break
print("Closing a connection") # need to handle leaving the stream
connection.close()
def handle_stream():
for connection in connections:
try:
connection.send(bytes(json.dumps(frame_info) ,"utf-8"))
except:
print("Packet send failure, kicking client")
connections.remove(connection)
while True:
Client, address = ServerSocket.accept()
print('Connected to: ' + address[0] + ':' + str(address[1]))
connections.append(Client)
Thread(target=threaded_client, args=(Client, ), daemon=True).start()
ThreadCount += 1
print('Thread Number: ' + str(ThreadCount))
The only part that changes for the client is the part where it connects to the server
try:
ClientSocket.connect((host, port))
except socket.error as e:
print(str(e))

Multithreading sockets with a central relay-like server

I have previously managed to implement a client-server socket script which relays messages between a single client and the server and I'm now trying to implement a multiple-client system.
More specifically, I would like to use the server as some sort of medium between two clients which retrieves information from one client and relays it to the other. I had tried to attach and send the port number of the receiving client and then extract it from the message on the server side. After that, I would try and send it to whatever socket with that port number but I ran into some trouble (as port numbers are determined at the point of sending I believe?) so now I am simply just trying to relay the sent message back to all clients. However, the problem is that the message is only being sent to the server and not being relayed to the desired client.
I had previously tried to implement a peer-to-peer system but I ran into trouble so I decided to take a step back and do this instead.
Server.py:
import socket, _thread, threading
import tkinter as tk
SERVERPORT = 8600
HOST = 'localhost'
class Server():
def __init__(self):
self.Connected = True
self.ServerSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.ServerSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR,1)
self.ServerSocket.bind((HOST, SERVERPORT))
self.ServerSocket.listen(2)
self.Clients = []
def Listen(self):
print('Server is now running')
while self.Connected:
ClientSocket, Address = self.ServerSocket.accept()
self.Clients.append(Address)
print('\nNew user connected', Address)
t = threading.Thread(target=self.NewClient, args=(ClientSocket,
Address))
t.daemon = True
t.start()
self.Socket.close()
def NewClient(self, ClientSocket, Address):
while self.Connected:
if ClientSocket:
try:
ReceivedMsg = ClientSocket.recv(4096)
print('Message received from', Address, ':', ReceivedMsg)
self.Acknowledge(ClientSocket, Address)
if ReceivedMsg.decode('utf8').split()[-1] != 'message':
ReceiverPort = self.GetSendPort(ReceivedMsg)
self.SendToClient(ClientSocket,ReceivedMsg,ReceiverPort)
except:
print('Connection closed')
raise Exception
ClientSocket.close()
def Acknowledge(self, Socket, Address):
Socket.sendto(b'The server received your message', Address)
def GetSendPort(self, Msg):
MsgDigest = Msg.decode('utf8').split()
return int(MsgDigest[-1])
def SendToClient(self, Socket, Msg, Port):
Addr = (HOST, Msg)
for Client in self.Clients:
Socket.sendto(Msg, Client)
def NewThread(Func, *args):
if len(args) == 1:
t = threading.Thread(target=Func, args=(args,))
elif len(args) > 1:
t = threading.Thread(target=Func, args=args)
else:
t = threading.Thread(target=Func)
t.daemon = True
t.start()
t.join()
Host = Server()
NewThread(Host.Listen)
And the Client(.py):
import socket, threading
import tkinter as tk
Username = 'Ernest'
PORT = 8601
OtherPORT = 8602
SERVERPORT = 8600
HOST = '127.0.0.1'
class Client():
def __init__(self, Username):
self.Connected, self.Username = False, Username
self.Socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
def Connect(self):
print('Trying to connect')
try:
self.Socket.connect((HOST, SERVERPORT))
self.Connected = True
print(self.Username, 'connected to server')
Msg = MsgUI(self.Username)
Msg.Display()
except Exception:
print('Could not connect to server')
raise Exception
def SendMsg(self):
if self.Connected:
Msg = '{} sent you a message {}'.format(self.Username, OtherPORT)
self.Socket.sendall(bytes(Msg, encoding='utf8'))
self.GetResponse()
def GetResponse(self, *args):
AckMsg = '\n{} received the message'.format(self.Username)
NMsg = '\n{} did not receive the message'.format(self.Username)
if self.Connected:
Msg = self.Socket.recv(4096)
print(Msg)
if Msg:
self.Socket.sendall(bytes(AckMsg, encoding='utf8'))
else:
self.Socket.sendall(bytes(NMsg, encoding='utf8'))
class MsgUI():
def __init__(self, Username):
self.Username = Username
self.entry = tk.Entry(win)
self.sendbtn = tk.Button(win, text='send', command=Peer.SendMsg)
def Display(self):
self.entry.grid()
self.sendbtn.grid()
win.mainloop()
win = tk.Tk()
Peer = Client(Username)
Peer.Connect()
I want a message to be sent whenever the user presses the send button in the tkinter window, but at the same time, it is continually 'listening' to see if it received any messages.
I also previously tried to run the GetResponse method in the Client in another thread and instead of if self.Connected I used while self.Connected and it still didn't work.
UPDATE
After some helpful comments, I have edited the two files as such:
The server now holds the two sockets for each client which is run first. The server file is imported into the client file as a module. Each client file is then run and each client runs a function in the server file, requesting to use the socket. If the request is allowed (i.e. no error was thrown), the socket is connected, added to a set of clients stored in the server file and then returned to the client file. The client then uses this socket to send and receive messages.
Server.py
import socket, _thread, threading
import tkinter as tk
SERVERPORT = 8600
HOST = 'localhost'
class Server():
def __init__(self):
self.Connected = True
self.ServerSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.ServerSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR,1)
self.ServerSocket.bind((HOST, SERVERPORT))
self.ServerSocket.listen(2)
self.Clients = {}
def ConnectClient(self, Username, Port):
Socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.Clients[Username] = [Socket, Port, False]
try:
self.Clients[Username][0].connect((HOST, SERVERPORT))
self.Clients[Username][2] = True
print('Opened port for user', Username)
return Socket
except Exception:
print('Could not open port for user', Username)
raise Exception
def Listen(self):
print('Server is now running')
while self.Connected:
ClientSocket, Address = self.ServerSocket.accept()
print('\nNew user connected', Address)
t = threading.Thread(target=self.NewClient, args=(ClientSocket,
Address))
t.daemon = True
t.start()
self.Socket.close()
def NewClient(self, ClientSocket, Address):
while self.Connected:
if ClientSocket:
try:
ReceivedMsg = ClientSocket.recv(4096)
if b'attempting to connect to the server' in ReceivedMsg:
ClientSocket.send(b'You are now connected to the server')
else:
print('Message received from', Address, ':',ReceivedMsg)
#self.Acknowledge(ClientSocket, Address)
ReceiverPort = self.GetSendPort(ReceivedMsg)
if ReceiverPort != None:
self.SendToClient(ClientSocket,ReceivedMsg,
ReceiverPort)
except:
print('Connection closed')
raise Exception
ClientSocket.close()
def Acknowledge(self, Socket, Address):
Socket.sendto(b'The server received your message', Address)
def GetSendPort(self, Msg):
MsgDigest = Msg.decode('utf8').split()
try:
Port = int(MsgDigest[-1])
except ValueError:
Port = None
return Port
def SendToClient(self, Socket, Msg, Port):
Addr = (HOST, Port)
Receiver = None
for Client, Vars in self.Clients.items():
if Vars[1] == Port:
Receiver = Client
self.Clients[Receiver][0].sendto(Msg, Addr)
def NewThread(Func, *args):
if len(args) == 1:
t = threading.Thread(target=Func, args=(args,))
elif len(args) > 1:
t = threading.Thread(target=Func, args=args)
else:
t = threading.Thread(target=Func)
t.daemon = True
t.start()
t.join()
Host = Server()
if __name__ == '__main__':
NewThread(Host.Listen)
And Client.py
import socket, threading, Server
import tkinter as tk
Username = 'Ernest'
PORT = 8601
OtherPORT = 8602
SERVERPORT = 8600
HOST = '127.0.0.1'
class Client():
def __init__(self, Username):
self.Connected, self.Username = False, Username
def Connect(self):
print('Requesting to connect to server')
try:
self.Socket = Server.Host.ConnectClient(self.Username, PORT)
self.Connected = Server.Host.Clients[self.Username][2]
Msg = '{} is attempting to connect to the server'.format(self.Username)
self.Socket.sendall(bytes(Msg, encoding='utf8'))
ReceivedMsg = self.Socket.recv(4096)
print(ReceivedMsg)
Msg = MsgUI(self.Username)
Msg.Display()
except Exception:
print('Could not connect to server')
raise Exception
def SendMsg(self):
try:
if self.Connected:
Msg = '{} sent you a message {}'.format(self.Username,OtherPORT)
self.Socket.sendall(bytes(Msg, encoding='utf8'))
self.GetResponse()
except Exception:
print('Connection closed')
raise Exception
def GetResponse(self, *args):
AckMsg = '\n{} received the message'.format(self.Username)
NMsg = '\n{} did not receive the message'.format(self.Username)
if self.Connected:
Msg = self.Socket.recv(4096)
print(Msg)
if Msg:
self.Socket.sendall(bytes(AckMsg, encoding='utf8'))
else:
self.Socket.sendall(bytes(NMsg, encoding='utf8'))
class MsgUI():
def __init__(self, Username):
self.Username = Username
self.entry = tk.Entry(win)
self.sendbtn = tk.Button(win, text='send', command=Peer.SendMsg)
def Display(self):
self.entry.grid()
self.sendbtn.grid()
win.mainloop()
win = tk.Tk()
Peer = Client(Username)
Peer.Connect()
Now the problem is more of a python and scope problem. When trying to relay the message back to the client, I was getting a KeyError as the Clients dictionary was still empty. When making the function call to the server in the client file, it's clear that the update to the dictionary happens in the client file rather than the server file - which is in a different instance. I need a method of changing the contents of the Clients dictionary that is called to action by the client file but takes effect in the server file.
Are you committed to multithreading? Threads don't run concurrently in python ( due to the GIL), and while they are one way to handle concurrent operations, they aren't the only way and usually they're not the best way, unless they're the only way. Consider this code, which doesn't handle failure cases well, but seems to work as a starting point.
import socket, select, Queue
svrsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
svrsock.setblocking(0)
svrsock.bind(('', 17654))
svrsock.listen(16)
client_queues = {}
write_ready=[] # we'll update this for clients only that have things in the queue
while client_queues.keys() + [svrsock] :
readable, writable, exceptional = select.select(client_queues.keys() + [svrsock] , write_ready, [])
for rd in readable:
if rd is svrsock: # reading listening socket == accepting connection
conn, addr = svrsock.accept()
print("Connection from {}".format(addr))
conn.setblocking(0)
client_queues[conn] = Queue.Queue()
else:
data = rd.recv(1024)
if data:
# TODO: send to all queues
print("Message from {}".format(rd.getpeername()))
for sock, q in client_queues.iteritems():
q.put("From {}: {}".format( rd.getpeername(), data))
if sock not in write_ready:
write_ready.append(sock)
for rw in writable:
try:
data = client_queues[rw].get_nowait()
rw.send(data)
except Queue.Empty:
write_ready.remove(rw)
continue
The concept is pretty simple. The server accepts connections; each connection (socket) is associated with a queue of pending messages. Each socket that's ready for reading is read from, and its message is added to each client's queue. The recipient client is added into the write_ready list of clients with data pending, if it's not already in there. Then each socket that's ready for writing has its next queued message written to it. If there are no more messages, the recipient is removed from the write_ready list.
This is very easy to orchestrate if you don't use multithreading because all coordination is inherent in the order of the application. With threads it would be more difficult and a lot more code, but probably not more performance due to the gil.
The secret to handling multiple I/O streams concurrently without multithreading is select. In principle it's pretty easy; we pass select() a list of possible sockets for reading, another list of possible sockets for writing, and a final list that for this simplified demo I completely ignore . The results of the select call will include one or more sockets that are actually ready for reading or writing, which allows me to block until one or more sockets are ready for activity. I then process all the sockets ready for activity every pass ( but they've already been filtered down to just those which wouldn't block).
There's a ton still to be done here. I don't cleanup after myself, don't track closed connections, don't handle any exceptions, and so on. but without having to worry about threading and concurrency guarantees, it's pretty easy to start addressing these deficiencies.
Here it is "in action". Here for the client side I use netcat, which is perfect for layer 3 testing without layer 4+ protocols ( in other words, raw tcp so to speak). It simply opens a socket to the given destination and port and sends its stdin through the socket and sends its socket data to stdout, which makes it perfect for demoing this server app!
I also wanted to point out, coupling code between server and client is inadvisable because you won't be able to roll out changes to either without breaking the other. It's ideal to have a "contract" so to speak between server and client and maintain it. Even if you implement the behavior of server and client in the same code base, you should use the tcp communications contract to drive your implementation, not code sharing. Just my 2 cents, but once you start sharing code you often start coupling server/client versions in ways you didn't anticipate.
the server:
$ python ./svr.py
Connection from ('127.0.0.1', 52059)
Connection from ('127.0.0.1', 52061)
Message from ('127.0.0.1', 52061)
Message from ('127.0.0.1', 52059)
Message from ('127.0.0.1', 52059)
First client ( 52059):
$ nc localhost 17654
hello
From ('127.0.0.1', 52061): hello
From ('127.0.0.1', 52059): hello
From ('127.0.0.1', 52059): hello
Second client:
$ nc localhost 17654
From ('127.0.0.1', 52061): hello
hello
From ('127.0.0.1', 52059): hello
hello
From ('127.0.0.1', 52059): hello
If you need more convincing on why select is way more compelling than concurrent execution, consider this: Apache is based on a threading model, in other words, the connections each get a worker thread . nginx is based on a select model, so you can see how much faster that can potentially be. Not to say that nginx is inherently better, as Apache benefits from the threading model because of its heavy use of modules to extend capabilities ( mod_php for example), whereas nginx doesn't have this limitation and can handle all requests from any thread. But the raw performance of nginx is typically considered far higher and far more efficient, and a big reason for this is that it avoids almost all the cpu context switches inherent in apache. It's a valid approach!
A word on scaling. Obviously, this wouldn't scale forever. Neither would a threading model; eventually you run out of threads. A more distributed and high throughput system would likely use a Pub/Sub mechanism of some kind, offloading the client connection tracking and message queueing from the server to a pub/sub data tier and allowing connections to be restored and queued data to be sent, as well as adding multiple servers behind a load balancer. Just throwing it out there. You might be pleasantly surprised how well select can scale ( cpu is so much faster than network anyway that it's likely not the bottleneck).

Python long-lived socket connection weirdness

I've implemented some code that allows a client to connect to a socket server, introduces itself and the server then goes into an infinite loop which sends "commands" (strings) to the client from a Redis list. The server uses the Redis 'blpop' method to block until a string arrives which is then sent off to the client and the response awaited.
However, in testing (with a python client socket script on another local workstation) I find that if I break the client connection (Ctrl+c) to simulate an interruption in the connectivity, the server happily writes the next received string to the client, reports an empty response but ONLY throws the broken pipe exception when a second string is written :/ Thus, two writes are "lost" before anything is caught. Here's my code:
# Create global Redis resource
rds_cnx = redis.StrictRedis(host='localhost', port=6379, db=6)
def initialise_server():
""" Setup server socket """
try:
srv_skt = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
srv_skt.bind((IP, PORT))
srv_skt.listen(1)
print("Listening on:[{}]".format(IP, PORT))
return srv_skt
except socket.error as skt_err: # e.g. port in use
print("Could not initialise tcp server:[{}]".format(skt_err))
sys.exit(1)
except Exception as exp:
print("Unable to setup server socket:[{}]".format(exp))
sys.exit(1)
def main():
server_socket = initialise_server()
while True:
client_socket, remote_address = server_socket.accept()
try:
# Block and wait for connection and data
initial_data = client_socket.recv(1024).decode()
print("Connection from [{}] - Data:[{}]".format(remote_address, initial_data))
while True:
wait_for_queue_command(client_socket)
except (BrokenPipeError, socket.error, Exception) as sck_exp:
print("Exception in client loop:[{}]".format(sck_exp))
continue
except KeyboardInterrupt:
# Close client socket
client_socket.shutdown(2)
client_socket.close()
print('Caught Ctrl+c ... Shutting down.')
break
# Tear down context
server_socket.shutdown(2) # Param ref: 0 = done receiving, 1 = done sending, 2 = both
server_socket.close()
def wait_for_queue_command(client_skt):
""" Blocking while waiting for command for Redis list
:param client_skt: socket
:return: None
"""
print('Waiting for command...')
queue_cmd = rds_cnx.blpop('queuetest', 0)
print("Received something from the queue:")
pprint(queue_cmd)
try:
#client_skt.settimeout(15)
client_skt.send(queue_cmd[1])
# Block for response
response_data = client_skt.recv(1024).decode()
print("Response:[{}]".format(response_data))
except BrokenPipeError as brkn_p:
print('Outbound write detected "Broken Pipe":[{}]'.format(brkn_p))
''' Here one would decide to either re-schedule the command or
ignore the error and move on to the next command. A "pause"
(sleep) could also useful?
'''
raise
except socket.timeout as sck_tmo:
print('Socket timed out:[{}]'.format(sck_tmo))
except socket.error as sck_err:
print('Socket timed out:[{}]'.format(sck_err))
raise
print('Command handling complete.')
Is there any better way to handle such a situation? I've had a cursory look at Twisted but it seems very difficult to achieve the specific blocking behavior and other code that might be implemented to handle specific responses from the client.

Problems with socketing in IDLE

I'm trying to get a small socket communication set up on my own machine for testing purposes, but I keep getting errors like "[Errno 10053] An established connection was aborted by the software in your host machine" and "[Errno 10054] An existing connection was forcibly closed by the remote host"
The code for the server is
import socket, threading, Queue
class PiConn(threading.Thread, object):
def __init__(self, input_queue, output_queue):
threading.Thread.__init__(self)
self.input_queue = input_queue
self.output_queue = output_queue
self.HOST = ''
self.PORT = 8888
self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
try:
self.s.bind((self.HOST, self.PORT))
except socket.error, msg:
print "Binding socket failed, error message: " + msg[1]
def run(self):
self.s.listen(5)
while True:
try:
#trying to accept data
conn, addr = self.s.accept()
print "Connected to", addr
data = conn.recv(4096)
self.input_queue.put(data)
except Exception as e:
print e, "when trying to accept data"
break
try:
output = self.output_queue.get(False)
self.s.sendall(output)
print "Sent", output
except Queue.Empty:
pass
except socket.error as e:
print e, "when trying to send data"
input_queue = Queue.Queue()
output_queue = Queue.Queue()
conn = PiConn(input_queue, output_queue)
conn.start()
while True:
output_queue.put("This is sent by server")
try:
print input_queue.get(False)
except Queue.Empty:
pass
The code for the client is
import socket, threading, Queue
class GUIConn(threading.Thread, object):
def __init__(self, input_queue, output_queue):
threading.Thread.__init__(self)
self.input_queue = input_queue
self.output_queue = output_queue
self.PORT = 8888
self.PI_IP = "127.0.0.1"
try:
#Creates a socket
self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
except socket.error, msg:
print 'Socket creating failed, error message:' + str(msg[1])
self.s.connect((self.PI_IP, self.PORT))
def run(self):
while True:
try:
#Trying to send data
output = self.output_queue.get(False)
self.s.sendall(output)
except Queue.Empty:
pass
except socket.error as e:
print e
try:
#trying to accept data
data = self.s.recv(4096)
self.input_queue.put(data)
except Exception as e:
print e
break
input_queue = Queue.Queue()
output_queue = Queue.Queue()
conn = GUIConn(input_queue, output_queue)
conn.start()
while True:
output_queue.put("This is sent by client")
try:
print input_queue.get(False)
except Queue.Empty:
pass
To test it, I start 2 IDLE shells, run the server, and then the client.
Any clue as to what I'm doing wrong? I'm fairly new at sockets, and I've been struggling with this all day.
Thanks in advance!
Your initial problem is caused by known issues IDLE has when working with threads.
See here and here for example.
I'm not aware of any workaround. Try running your code from terminal instead.
As to the other errors you're getting, if you post them, we can try and assist.
warning, big wall of text, read all of it before commenting
there is a huge number of problem with this small amount of code
first, the most obvious is the 'busy' loops that will use up all 100% of the cpu, not only that, it will also slowly use up all the ram as well cause you set the blocking for the queue.get to be False
you could have set it to True and it would have waited until there something and once it get that, it would loop back to the top and put another one of "This is sent by client" thus solving both the busy loop and ram usage problem
while True:
output_queue.put("This is sent by client")
try:
print input_queue.get(False) # here
except Queue.Empty:
pass
second, the way you reply/send data from the server to the client isn't through the main listening socket but the socket that is return from the self.s.accept()
so self.s.sendall(output) in the server should have been conn.sendall(output)
third, in the client code, there a chance that self.output_queue.get(False) would error with Queue.Empty and thus pass using the try and except and ended up in the blocking recv
and both the server and client would both be listening and waiting for each other to send something
fourth, self.s.accept() is blocking, after one loop in the server, it would be stuck waiting for another client while the client would send the data then end up waiting for some data
lastly, about those error you said, i can't reproduce them at all, if i have to guess, i say those error are cause by your firewall or the server isn't running (fail to bind) or something else, see here: No connection could be made because the target machine actively refused it
also, you could try a different port and maybe the first two example on this site to check if there is something weird causing problem, if those example doesn't work then there is a problem with your computer, https://docs.python.org/release/2.5.2/lib/socket-example.html

Categories

Resources