I'm wanting to exit an UDP-server on KeyboardInterrupt, but this is not working immediately. when doing some research I stumbled on this question, where someone suggests that it is a issue with the GIL. Python processes calling into C APIs can block the Python interpreter from processing the interrupt. is there any way the unblock this in python?
the code I have is the following:
if __name__ == '__main__':
try:
main()
except KeyboardInterrupt: #works after new data is send to server
print('Interrupted', file=sys.stderr)
try:
sys.exit(0)
except SystemExit:
os._exit(0)
main:
def main():
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server_address = ('localhost', 2000)
s.bind(server_address)
running = True
print('Ready for connection')
while running:
try:
msg = ""
data, address = s.recvfrom(2000) #look for timeout and interrupt
msg = data.decode('utf-8')
if not data:
break
print(f'received {msg}')
if msg[:5] == "PING-":
print('sending PONG back to the client')
message = 'PONG-' + msg[5:len(msg)]
s.sendto(message.encode('utf-8'), address)
else:
print('bad format')
except socket.error as socketerror:
print(f'Error: {socketerror}', file=sys.stderr)
s.close()
When i'm not running the client and press ctrl-c nothing happens, the program just does nothing.
When I start up the client and send a UDP message, the interrupt is perfectly handled when the server gets a UDP message from the client. The UDP message then is dropped and the client handles it as a timeout. After the timeout the client has a lost connection.
The goal is to get the server exit even without the client running and without using ctrl-break.
Related
I am making a script that will allow multiple clients to see live camera footage from the server script, this all works fine until one of the client scripts is closed, then a ConnectionResetError is raised, To avoid this I used a try and except block to catch the ConnectionResetError but the same error is raised every time after a connection is lost. Using just socket.recv stops the ConnectionResetError but socket.recv does not return the senders address which is required for the script to send the video stream back to the client.
Server:
host = "0.0.0.0"
port = 5000
buffer_size = 1024
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(("", port))
listeners = list() # the addresses of the clients that want the video stream
def handle_queue(sock):
while True:
try:
message, address = sock.recvfrom(buffer_size) # block the thread until a packet arrives
print(address)
message = str(message, "utf-8") # decode the message
if message == "join":
listeners.append(address) # add the list of listeners
else:
print("unknown queue msg: ", message)
except ConnectionResetError:
print("The connection was forcefully quit")
queue_handler_thread = Thread(target=handle_queue, args=(sock,), daemon=True)
queue_handler_thread.start() # start the queue
the script then uses sock.sendto() for each address in the listeners list
Client:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(bytes("join","utf-8"), (host, port))
while True:
data, address = sock.recvfrom(max_length) # block main thread until a packet is received
I believe what you are looking for is socket.getpeername().
This will return the remote address to which the socket is connected.
Your approach in handling the data in the wire whilst the connection is closed/lost is the right way. Handling via try/catch or using a recv() or recvfrom() method to wait for a response before closing the socket.
Since posting this i have discovered that it is infact TCP and this will not work for the project I am trying to do. Any more guidance and help would be appreciated, I cant seem to find any examples of Multiclient UDP Servers that run through one socket.
I partially resolved my system, after using the Example provided by AztCrw4282.
I was able to find a solution, I instead used the socket.accept() aproach, I am unsure as to whether this is UDP or TCP (I want UDP but with the connect system I think that is the handshake for TCP, but I am not 100% sure), but it works for now.
The client has to connect to the server then the server will accept or deny the connection, upon accepting a thread will be created for that clients connection to be managed. If any errors are thrown when interacting with that client their connection will be closed.
Server
try:
ServerSocket.bind((host, port))
except socket.error as e:
print(str(e))
print('Waiting for a Connection..')
ServerSocket.listen(5)
connections = list()
def threaded_client(connection):
connection.send(str.encode('Welcome to the Server\n'))
while True:
try:
data = str(connection.recv(2048),"utf-8") # this needs to be try catched
print("Packet Recv: ", data)
if data == "join":
print("Client Joined")
if data == "quit":
break
if not data:
break
except ConnectionResetError:
break
print("Closing a connection") # need to handle leaving the stream
connection.close()
def handle_stream():
for connection in connections:
try:
connection.send(bytes(json.dumps(frame_info) ,"utf-8"))
except:
print("Packet send failure, kicking client")
connections.remove(connection)
while True:
Client, address = ServerSocket.accept()
print('Connected to: ' + address[0] + ':' + str(address[1]))
connections.append(Client)
Thread(target=threaded_client, args=(Client, ), daemon=True).start()
ThreadCount += 1
print('Thread Number: ' + str(ThreadCount))
The only part that changes for the client is the part where it connects to the server
try:
ClientSocket.connect((host, port))
except socket.error as e:
print(str(e))
I am having sockets code in which I am handling clients through threads. The use case is like this,
1. Server runs and clients start connecting to it.
2. After their interaction clients issue exit message and server closes the client connection.
3. However what i want is in case something happens on server and if i manually hit ctrl-c on server it should exit all threads and main program as well.
Below is what i was trying but unfortunately not everything gets cleaned up and i have to hit ctrl-c twice.
import socket
import sys
from threading import Thread
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# This is to prevent the socket going into TIME_WAIT status and OSError
# "Address already in use"
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
except socket.error as e:
print('Error occured while creating the socket {}'.format(e))
server_address = ('localhost', 50000)
sock.bind(server_address)
print('**** Server started on {}:{} ****'.format(*server_address))
sock.listen(5)
def client_thread(conn_sock, client_add):
while True:
client_msg = conn_sock.recv(1024).decode()
if client_msg.lower() != 'exit':
print('[{0}:{1}] {2}'.format(*client_add, client_msg))
serv_reply = 'Okay ' + client_msg.upper()
conn_sock.send(bytes(serv_reply, 'utf-8'))
else:
conn_sock.close()
print('{} exitted !!'.format(client_add[0]))
sys.exit()
try:
# Keep the server until there are incominmg connections
while True:
# Wait for the connctions to accept
conn_sock, client_add = sock.accept()
print('Recieved connection from {}:{}'.format(*client_add))
conn_sock.send(
bytes('***** Welcome to {} *****'.format(server_address[0]), 'utf-8'))
Thread(target=client_thread, args=(conn_sock, client_add)).start()
except Exception as e:
print('Some error occured \n {}'.format(e))
except KeyboardInterrupt as e:
print('Program execution cancelled by user')
sys.exit(0)
finally:
sock.close()
Below is the execution trace of my code
**** Server started on localhost:50000 ****
Recieved connection from 127.0.0.1:35910
[127.0.0.1:35910] ,msdasda
^CProgram execution cancelled by user
^CException ignored in: <module 'threading' from '/apps/nttech/rbhanot/anaconda3/lib/python3.6/threading.py'>
Traceback (most recent call last):
File "/apps/nttech/rbhanot/anaconda3/lib/python3.6/threading.py", line 1294, in _shutdown
t.join()
File "/apps/nttech/rbhanot/anaconda3/lib/python3.6/threading.py", line 1056, in join
self._wait_for_tstate_lock()
File "/apps/nttech/rbhanot/anaconda3/lib/python3.6/threading.py", line 1072, in _wait_for_tstate_lock
elif lock.acquire(block, timeout):
KeyboardInterrupt
So can anyone suggest how can i close everything when keyboardInterrupt occures.
I got it working by making the threads as daemon, so that main program will exit on receiving the KeyboardInterrupt and threads will keep on running in background.
I've implemented some code that allows a client to connect to a socket server, introduces itself and the server then goes into an infinite loop which sends "commands" (strings) to the client from a Redis list. The server uses the Redis 'blpop' method to block until a string arrives which is then sent off to the client and the response awaited.
However, in testing (with a python client socket script on another local workstation) I find that if I break the client connection (Ctrl+c) to simulate an interruption in the connectivity, the server happily writes the next received string to the client, reports an empty response but ONLY throws the broken pipe exception when a second string is written :/ Thus, two writes are "lost" before anything is caught. Here's my code:
# Create global Redis resource
rds_cnx = redis.StrictRedis(host='localhost', port=6379, db=6)
def initialise_server():
""" Setup server socket """
try:
srv_skt = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
srv_skt.bind((IP, PORT))
srv_skt.listen(1)
print("Listening on:[{}]".format(IP, PORT))
return srv_skt
except socket.error as skt_err: # e.g. port in use
print("Could not initialise tcp server:[{}]".format(skt_err))
sys.exit(1)
except Exception as exp:
print("Unable to setup server socket:[{}]".format(exp))
sys.exit(1)
def main():
server_socket = initialise_server()
while True:
client_socket, remote_address = server_socket.accept()
try:
# Block and wait for connection and data
initial_data = client_socket.recv(1024).decode()
print("Connection from [{}] - Data:[{}]".format(remote_address, initial_data))
while True:
wait_for_queue_command(client_socket)
except (BrokenPipeError, socket.error, Exception) as sck_exp:
print("Exception in client loop:[{}]".format(sck_exp))
continue
except KeyboardInterrupt:
# Close client socket
client_socket.shutdown(2)
client_socket.close()
print('Caught Ctrl+c ... Shutting down.')
break
# Tear down context
server_socket.shutdown(2) # Param ref: 0 = done receiving, 1 = done sending, 2 = both
server_socket.close()
def wait_for_queue_command(client_skt):
""" Blocking while waiting for command for Redis list
:param client_skt: socket
:return: None
"""
print('Waiting for command...')
queue_cmd = rds_cnx.blpop('queuetest', 0)
print("Received something from the queue:")
pprint(queue_cmd)
try:
#client_skt.settimeout(15)
client_skt.send(queue_cmd[1])
# Block for response
response_data = client_skt.recv(1024).decode()
print("Response:[{}]".format(response_data))
except BrokenPipeError as brkn_p:
print('Outbound write detected "Broken Pipe":[{}]'.format(brkn_p))
''' Here one would decide to either re-schedule the command or
ignore the error and move on to the next command. A "pause"
(sleep) could also useful?
'''
raise
except socket.timeout as sck_tmo:
print('Socket timed out:[{}]'.format(sck_tmo))
except socket.error as sck_err:
print('Socket timed out:[{}]'.format(sck_err))
raise
print('Command handling complete.')
Is there any better way to handle such a situation? I've had a cursory look at Twisted but it seems very difficult to achieve the specific blocking behavior and other code that might be implemented to handle specific responses from the client.
I am using a REQ/REP type socket for ZMQ communication in python. There are multiple clients that attempt to connect to one server. Timeouts have been added in the client script to prevent indefinite wait.
The problem is that when the server is not running, and a client attempts to establish connection, it's message gets added to the queue buffer, which should not even exist at this moment ideally. When the script starts running and a new client connects, the previous client's data is taken in first by the server. This should not happen.
When the server starts, it assumes a client is connected to it since it had tried to connect previously, and could not exit cleanly (since the server was down).
In the code below, when the client tries the first time, it gets ERR 03: Server down which is correct, followed by Error disconnecting. When server is up, I get ERR 02: Server Busy for the first client which connects. This should not occur. The client should be able to seamlessly connect with the server now that it's up and running.
Server Code:
import zmq
def server_fn():
context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind("tcp://192.168.1.14:5555")
one=1
while one == 1:
message = socket.recv()
#start process if valid new connection
if message == 'hello':
socket.send(message) #ACK
#keep session alive until application ends it.
while one == 1:
message = socket.recv()
print("Received request: ", message)
#exit connection
if message == 'bye':
socket.send(message)
break
#don't allow any client to connect if already busy
if message == 'hello':
socket.send ('ERR 00')
continue
#do all data communication here
else:
socket.send('ERR 01: Connection Error')
return
server_fn()
Client Code:
import zmq
class client:
def clientInit(self):
hello='hello'
#zmq connection
self.context = zmq.Context()
print("Connecting to hello world server...")
self.socket = self.context.socket(zmq.REQ)
self.socket.connect("tcp://192.168.1.14:5555")
#RCVTIMEO to prevent forever block
self.socket.setsockopt(zmq.RCVTIMEO, 5000)
#SNDTIME0 is needed since script may not up up yet
self.socket.setsockopt(zmq.SNDTIMEO, 5000)
try:
self.socket.send(hello)
except:
print "Sending hello failed."
try:
echo = self.socket.recv()
if hello == echo:
#connection established.
commStatus = 'SUCCESS'
elif echo == 'ERR 00':
#connection busy
commStatus = "ERR 00. Server busy."
else:
#connection failed
commStatus="ERR 02"
except:
commStatus = "ERR 03. Server down."
return commStatus
def clientQuit(self):
try:
self.socket.send('bye')
self.socket.recv()
except:
print "Error disconnecting."
cObj = client()
commStatus=cObj.clientInit()
print commStatus
cObj.clientQuit()
PS - I have a feeling the solution may lie in the correct usage of socket.bind and socket.connect.
Answering my own question-
The problem is that the first client sends a message which the server accepts when it starts running, regardless of the status of the client.
To prevent this, 2 things have to be done. The most important thing is to use socket.close() to close the client connection. Secondly, the LINGER parameter can be set to a low value or zero. This clears the buffer after the timeout value from the time the socket is closed.
class client:
def clientInit(self):
...
self.socket.setsockopt(zmq.LINGER, 100)
...
def clientQuit(self):
try:
self.socket.send('bye')
self.socket.recv()
except:
print "Error disconnecting."
self.socket.close()
I'm trying to get a small socket communication set up on my own machine for testing purposes, but I keep getting errors like "[Errno 10053] An established connection was aborted by the software in your host machine" and "[Errno 10054] An existing connection was forcibly closed by the remote host"
The code for the server is
import socket, threading, Queue
class PiConn(threading.Thread, object):
def __init__(self, input_queue, output_queue):
threading.Thread.__init__(self)
self.input_queue = input_queue
self.output_queue = output_queue
self.HOST = ''
self.PORT = 8888
self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
try:
self.s.bind((self.HOST, self.PORT))
except socket.error, msg:
print "Binding socket failed, error message: " + msg[1]
def run(self):
self.s.listen(5)
while True:
try:
#trying to accept data
conn, addr = self.s.accept()
print "Connected to", addr
data = conn.recv(4096)
self.input_queue.put(data)
except Exception as e:
print e, "when trying to accept data"
break
try:
output = self.output_queue.get(False)
self.s.sendall(output)
print "Sent", output
except Queue.Empty:
pass
except socket.error as e:
print e, "when trying to send data"
input_queue = Queue.Queue()
output_queue = Queue.Queue()
conn = PiConn(input_queue, output_queue)
conn.start()
while True:
output_queue.put("This is sent by server")
try:
print input_queue.get(False)
except Queue.Empty:
pass
The code for the client is
import socket, threading, Queue
class GUIConn(threading.Thread, object):
def __init__(self, input_queue, output_queue):
threading.Thread.__init__(self)
self.input_queue = input_queue
self.output_queue = output_queue
self.PORT = 8888
self.PI_IP = "127.0.0.1"
try:
#Creates a socket
self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
except socket.error, msg:
print 'Socket creating failed, error message:' + str(msg[1])
self.s.connect((self.PI_IP, self.PORT))
def run(self):
while True:
try:
#Trying to send data
output = self.output_queue.get(False)
self.s.sendall(output)
except Queue.Empty:
pass
except socket.error as e:
print e
try:
#trying to accept data
data = self.s.recv(4096)
self.input_queue.put(data)
except Exception as e:
print e
break
input_queue = Queue.Queue()
output_queue = Queue.Queue()
conn = GUIConn(input_queue, output_queue)
conn.start()
while True:
output_queue.put("This is sent by client")
try:
print input_queue.get(False)
except Queue.Empty:
pass
To test it, I start 2 IDLE shells, run the server, and then the client.
Any clue as to what I'm doing wrong? I'm fairly new at sockets, and I've been struggling with this all day.
Thanks in advance!
Your initial problem is caused by known issues IDLE has when working with threads.
See here and here for example.
I'm not aware of any workaround. Try running your code from terminal instead.
As to the other errors you're getting, if you post them, we can try and assist.
warning, big wall of text, read all of it before commenting
there is a huge number of problem with this small amount of code
first, the most obvious is the 'busy' loops that will use up all 100% of the cpu, not only that, it will also slowly use up all the ram as well cause you set the blocking for the queue.get to be False
you could have set it to True and it would have waited until there something and once it get that, it would loop back to the top and put another one of "This is sent by client" thus solving both the busy loop and ram usage problem
while True:
output_queue.put("This is sent by client")
try:
print input_queue.get(False) # here
except Queue.Empty:
pass
second, the way you reply/send data from the server to the client isn't through the main listening socket but the socket that is return from the self.s.accept()
so self.s.sendall(output) in the server should have been conn.sendall(output)
third, in the client code, there a chance that self.output_queue.get(False) would error with Queue.Empty and thus pass using the try and except and ended up in the blocking recv
and both the server and client would both be listening and waiting for each other to send something
fourth, self.s.accept() is blocking, after one loop in the server, it would be stuck waiting for another client while the client would send the data then end up waiting for some data
lastly, about those error you said, i can't reproduce them at all, if i have to guess, i say those error are cause by your firewall or the server isn't running (fail to bind) or something else, see here: No connection could be made because the target machine actively refused it
also, you could try a different port and maybe the first two example on this site to check if there is something weird causing problem, if those example doesn't work then there is a problem with your computer, https://docs.python.org/release/2.5.2/lib/socket-example.html