I am making a script that will allow multiple clients to see live camera footage from the server script, this all works fine until one of the client scripts is closed, then a ConnectionResetError is raised, To avoid this I used a try and except block to catch the ConnectionResetError but the same error is raised every time after a connection is lost. Using just socket.recv stops the ConnectionResetError but socket.recv does not return the senders address which is required for the script to send the video stream back to the client.
Server:
host = "0.0.0.0"
port = 5000
buffer_size = 1024
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(("", port))
listeners = list() # the addresses of the clients that want the video stream
def handle_queue(sock):
while True:
try:
message, address = sock.recvfrom(buffer_size) # block the thread until a packet arrives
print(address)
message = str(message, "utf-8") # decode the message
if message == "join":
listeners.append(address) # add the list of listeners
else:
print("unknown queue msg: ", message)
except ConnectionResetError:
print("The connection was forcefully quit")
queue_handler_thread = Thread(target=handle_queue, args=(sock,), daemon=True)
queue_handler_thread.start() # start the queue
the script then uses sock.sendto() for each address in the listeners list
Client:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(bytes("join","utf-8"), (host, port))
while True:
data, address = sock.recvfrom(max_length) # block main thread until a packet is received
I believe what you are looking for is socket.getpeername().
This will return the remote address to which the socket is connected.
Your approach in handling the data in the wire whilst the connection is closed/lost is the right way. Handling via try/catch or using a recv() or recvfrom() method to wait for a response before closing the socket.
Since posting this i have discovered that it is infact TCP and this will not work for the project I am trying to do. Any more guidance and help would be appreciated, I cant seem to find any examples of Multiclient UDP Servers that run through one socket.
I partially resolved my system, after using the Example provided by AztCrw4282.
I was able to find a solution, I instead used the socket.accept() aproach, I am unsure as to whether this is UDP or TCP (I want UDP but with the connect system I think that is the handshake for TCP, but I am not 100% sure), but it works for now.
The client has to connect to the server then the server will accept or deny the connection, upon accepting a thread will be created for that clients connection to be managed. If any errors are thrown when interacting with that client their connection will be closed.
Server
try:
ServerSocket.bind((host, port))
except socket.error as e:
print(str(e))
print('Waiting for a Connection..')
ServerSocket.listen(5)
connections = list()
def threaded_client(connection):
connection.send(str.encode('Welcome to the Server\n'))
while True:
try:
data = str(connection.recv(2048),"utf-8") # this needs to be try catched
print("Packet Recv: ", data)
if data == "join":
print("Client Joined")
if data == "quit":
break
if not data:
break
except ConnectionResetError:
break
print("Closing a connection") # need to handle leaving the stream
connection.close()
def handle_stream():
for connection in connections:
try:
connection.send(bytes(json.dumps(frame_info) ,"utf-8"))
except:
print("Packet send failure, kicking client")
connections.remove(connection)
while True:
Client, address = ServerSocket.accept()
print('Connected to: ' + address[0] + ':' + str(address[1]))
connections.append(Client)
Thread(target=threaded_client, args=(Client, ), daemon=True).start()
ThreadCount += 1
print('Thread Number: ' + str(ThreadCount))
The only part that changes for the client is the part where it connects to the server
try:
ClientSocket.connect((host, port))
except socket.error as e:
print(str(e))
Related
I know this topic is not new. There is various information out there although, the robust solution is not presented (at least I did not found). I have a P2P daemon written in python3 and the last element on the pie is to connect two clients behind the NAT via TCP. My references for this topic:
https://bford.info/pub/net/p2pnat/
How to make 2 clients connect each other directly, after having both connected a meeting-point server?
Problems with TCP hole punching
What I have done so far:
SERVER:
#!/usr/bin/env python3
import threading
import socket
MY_AS_SERVER_PORT = 9001
TIMEOUT = 120.0
BUFFER_SIZE = 4096
def get_my_local_ip():
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try:
# doesn't even have to be reachable
s.connect(('10.255.255.255', 1))
IP = s.getsockname()[0]
except Exception:
IP = '127.0.0.1'
finally:
s.close()
return bytes(IP, encoding='utf-8')
def wait_for_msg(new_connection, client_address):
while True:
try:
packet = new_connection.recv(BUFFER_SIZE)
if packet:
msg_from_client = packet.decode('utf-8')
client_connected_from_ip = client_address[0]
client_connected_from_port = client_address[1]
print("We have a client. Client advertised his local IP as:", msg_from_client)
print(f"Although, our connection is from: [{client_connected_from_ip}]:{client_connected_from_port}")
msg_back = bytes("SERVER registered your data. Your local IP is: " + str(msg_from_client) + " You are connecting to the server FROM: " + str(client_connected_from_ip) + ":" + str(client_connected_from_port), encoding='utf-8')
new_connection.sendall(msg_back)
break
except ConnectionResetError:
break
except OSError:
break
def server():
sock = socket.socket()
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
sock.bind((get_my_local_ip().decode('utf-8'), MY_AS_SERVER_PORT))
sock.listen(8)
sock.settimeout(TIMEOUT)
while True:
try:
new_connection, client_address = sock.accept()
if new_connection:
threading.Thread(target=wait_for_msg, args=(new_connection,client_address,)).start()
# print("connected!")
# print("")
# print(new_connection)
# print("")
# print(client_address)
msg = bytes("Greetings! This message came from SERVER as message back!", encoding='utf-8')
new_connection.sendall(msg)
except socket.timeout:
pass
if __name__ == '__main__':
server()
CLIENT:
#!/usr/bin/python3
import sys
import socket
import time
import threading
SERVER_IP = '1.2.3.4'
SERVER_PORT = 9001
# We don't want to establish a connection with a static port. Let the OS pick a random empty one.
#MY_AS_CLIENT_PORT = 8510
TIMEOUT = 3
BUFFER_SIZE = 4096
def get_my_local_ip():
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try:
# doesn't even have to be reachable
s.connect(('10.255.255.255', 1))
IP = s.getsockname()[0]
except Exception:
IP = '127.0.0.1'
finally:
s.close()
return bytes(IP, encoding='utf-8')
def constantly_try_to_connect(sock):
while True:
try:
sock.connect((SERVER_IP, SERVER_PORT))
except ConnectionRefusedError:
print(f"Can't connect to the SERVER IP [{SERVER_IP}]:{SERVER_PORT} - does the server alive? Sleeping for a while...")
time.sleep(1)
except OSError:
#print("Already connected to the server. Kill current session to reconnect...")
pass
def client():
sock = socket.socket()
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
#sock.bind((get_my_local_ip().decode('utf-8'), MY_AS_CLIENT_PORT))
sock.settimeout(TIMEOUT)
threading.Thread(target=constantly_try_to_connect, args=(sock,)).start()
while True:
try:
packet = sock.recv(BUFFER_SIZE)
if packet:
print(packet)
sock.sendall(get_my_local_ip())
except OSError:
pass
if __name__ == '__main__':
client()
Now the current code results:
./tcphole_server.py
We have a client. Client advertised his local IP as: 10.10.10.50
Although, our connection is from: [89.22.11.50]:32928
We have a client. Client advertised his local IP as: 192.168.1.20
Although, our connection is from: [78.88.77.66]:51928
./tcphole_client1.py
b'Greetings! This message came from SERVER as message back!'
b'SERVER registered your data. Your local IP is: 192.168.1.20 You are connecting to the server FROM: 89.22.11.50:32928'
./tcphole_client2.py
b'Greetings! This message came from SERVER as message back!'
b'SERVER registered your data. Your local IP is: 10.10.10.50 You are connecting to the server FROM: 78.88.77.66:51928'
As you can see the server has all information to connect two clients. We can send details about the other peer individually through the current server-client connection.
Now two questions remain in my head:
Assuming the SERVER sends information about CLIENT 1 and CLIENT 2 for each of the peers. And now the CLIENTS starts connecting like [89.22.11.50]:32928 <> [78.88.77.66]:51928 Does the SERVER should close the current connections with the CLIENTS?
How the CLIENT Router behaves? I assume it expecting the same EXTERNAL SERVER SRC IP [1.2.3.4], instead gets one of the CLIENTS EXT IP for instance [89.22.11.50] or [78.88.77.66]?
This is messier than I thought. Any help to move forward appreciated. Hope this would help other Devs/DevOps too.
Finally found the expected behavior! Don't want to give too much code here but I hope after this you will understand the basics of how to implement it. Best to have a separate file in each of the client's folder - nearby ./tcphole_client1.py and ./tcphole_client2.py. We need to connect fast after we initiated sessions with the SERVER. Now for instance:
./tcphole_client_connector1.py 32928 51928
./tcphole_client_connector2.py 51928 32928
Remember? We need to connect to the same ports as we initiated with SERVER:
[89.22.11.50]:32928 <> [78.88.77.66]:51928
The first port is needed to bind the socket (OUR). With the second port, we are trying to connect to the CLIENT. The other CLIENT doing the same procedure except it binds to his port and connects to yours bound port. If the ROUTER still has an active connection - SUCCESS.
I have a small python console that works over UDP.
The problem I'm having is that if I abruptly disconnect from the client, the connection stays open in netstat (because it's UDP).
I have tried implementing a threading.Timer in the server that asks if the connection is active, and if the client does not respond within that time, the connection closes from the server-side.
This is what I tried
def check_conn(sock: socket.socket):
# sock.settimeout()
sock.send(b"\nAwake?")
try:
ack = sock.recv(1024)
ack = ack.decode("UTF-8")
if ack.strip() == "Yea":
pass
else:
sock.close()
except:
sock.close()
so = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
so.connect((host, port))
t = threading.Timer(10.0, check_conn, [so])
t.start()
data = so.recv(1024)
#rest of the main program
The problem I'm having is that when it asks for the password, two so.recv(1024) are open, and it does not know how to handle them (that's my guess).
I started learning networking with python can somebody help me out with this code as I am unable to connect more than 5 clients at a time. Can someone please suggest me a solution for this?
def main():
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('', listening_port))
s.listen(5)
print "[*] Initializing Sockets ... Done"
print "[*] Sockets Binded Successfully ..."
print("[*] Server Started Successfully [%d]\n" % (listening_port))
except Exception, e:
print e
sys.exit(2)
while 1:
try:
conn, addr = s.accept()
data = conn.recv(buffer_size)
start_new_thread(conn_string, (conn, data, addr))
except KeyboardInterrupt:
s.close()
print "\n[*] Proxy Server Shutting Down ..."
sys.exit(1)
s.close()
def conn_string(conn, data, addr):
print conn
print addr
print data
As described in the python socket API:
socket.listen(backlog)
Listen for connections made to the socket. The
backlog argument specifies the maximum number of queued connections
and should be at least 0; the maximum value is system-dependent
(usually 5), the minimum value is forced to 0.
Increase the number from 5 to the number of simultaneous connections you wish to have to the server.
The accepted answer is incorrect. I'll try to explain you why.
listen takes an argument of queued connections. That means, how many new connections can be put into the queue. That's why even in the documentation it's specified 5, while obviously servers serves more than 5 concurrent clients usually.
Once a socket is created (accepted) it is being forwarded to another thread, which performs the actual work. This allows to the listening thread moves back to listening mode and wait for more clients.
The problem with your code is, your main thread which is the listening thread actually reads data from buffer, where it shouldn't. The recv() function blocks (read more about I/O stream blocking)
You need to move the reading process from the listening thread to the worker; and make sure to close() the socket when it's done. The code would look something like this:
def main():
try:
...
s.listen(5)
except Exception, e:
...
while 1:
try:
conn, addr = s.accept()
start_new_thread(conn_string, (conn, addr))
except KeyboardInterrupt:
s.close()
print "\n[*] Proxy Server Shutting Down ..."
sys.exit(1)
s.close()
def conn_string(conn, addr):
data = conn.recv(buffer_size)
print conn
print addr
print data
conn.close()
I've implemented some code that allows a client to connect to a socket server, introduces itself and the server then goes into an infinite loop which sends "commands" (strings) to the client from a Redis list. The server uses the Redis 'blpop' method to block until a string arrives which is then sent off to the client and the response awaited.
However, in testing (with a python client socket script on another local workstation) I find that if I break the client connection (Ctrl+c) to simulate an interruption in the connectivity, the server happily writes the next received string to the client, reports an empty response but ONLY throws the broken pipe exception when a second string is written :/ Thus, two writes are "lost" before anything is caught. Here's my code:
# Create global Redis resource
rds_cnx = redis.StrictRedis(host='localhost', port=6379, db=6)
def initialise_server():
""" Setup server socket """
try:
srv_skt = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
srv_skt.bind((IP, PORT))
srv_skt.listen(1)
print("Listening on:[{}]".format(IP, PORT))
return srv_skt
except socket.error as skt_err: # e.g. port in use
print("Could not initialise tcp server:[{}]".format(skt_err))
sys.exit(1)
except Exception as exp:
print("Unable to setup server socket:[{}]".format(exp))
sys.exit(1)
def main():
server_socket = initialise_server()
while True:
client_socket, remote_address = server_socket.accept()
try:
# Block and wait for connection and data
initial_data = client_socket.recv(1024).decode()
print("Connection from [{}] - Data:[{}]".format(remote_address, initial_data))
while True:
wait_for_queue_command(client_socket)
except (BrokenPipeError, socket.error, Exception) as sck_exp:
print("Exception in client loop:[{}]".format(sck_exp))
continue
except KeyboardInterrupt:
# Close client socket
client_socket.shutdown(2)
client_socket.close()
print('Caught Ctrl+c ... Shutting down.')
break
# Tear down context
server_socket.shutdown(2) # Param ref: 0 = done receiving, 1 = done sending, 2 = both
server_socket.close()
def wait_for_queue_command(client_skt):
""" Blocking while waiting for command for Redis list
:param client_skt: socket
:return: None
"""
print('Waiting for command...')
queue_cmd = rds_cnx.blpop('queuetest', 0)
print("Received something from the queue:")
pprint(queue_cmd)
try:
#client_skt.settimeout(15)
client_skt.send(queue_cmd[1])
# Block for response
response_data = client_skt.recv(1024).decode()
print("Response:[{}]".format(response_data))
except BrokenPipeError as brkn_p:
print('Outbound write detected "Broken Pipe":[{}]'.format(brkn_p))
''' Here one would decide to either re-schedule the command or
ignore the error and move on to the next command. A "pause"
(sleep) could also useful?
'''
raise
except socket.timeout as sck_tmo:
print('Socket timed out:[{}]'.format(sck_tmo))
except socket.error as sck_err:
print('Socket timed out:[{}]'.format(sck_err))
raise
print('Command handling complete.')
Is there any better way to handle such a situation? I've had a cursory look at Twisted but it seems very difficult to achieve the specific blocking behavior and other code that might be implemented to handle specific responses from the client.
My socket program hangs at clientsocket, address) = serversocket.accept() and doesn't spit our an error or anything.
I followed directions on https://docs.python.org/3/howto/sockets.html
I've been trying to figure it out for an hour now, but to no avail. I'm using python3 btw. What am i doing wrong? EDIT: My intedentation is all screwed up because I pasted it wrong, but other than that my code is as I have it in my file.
#import socket module
import socket
#creates an inet streaming socket.
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print('socket created')
#binds socket to a public host, and a well known port
serversocket.bind(('127.0.0.1', 1024))
#print(socket.gethostname())# on desktop prints 'myname-PC')
#become a server socket
serversocket.listen(5) # listens for up to 5 requests
while True:
#accept connections from outside
#print('In while true loop') This works, but we never get to the next print statement. Why the hell is it catching at line 20?
(clientsocket, address) = serversocket.accept()
#clientsocket = serversocket.accept()
print('Ready to serve')
#now we do something with client socket...
try:
message = clientsocket.recv(1024)
filename = message.split()[1]
f = open(filename[1:])
outputdata = f.read()
#send an http header line
clientsocket.send('HTTP/1.1 200 OK\nContent-Type: text/html\n\n')
for i in range(0, len(outputdata)):
clientsocket.send(outputdata[i])
clientsocket.close()
except IOERROR:
clientsocket.send('HTTP/1.1 404 File not found!')
clientsocket.close()
If you haven't written a client script / program to connect to the socket and send it data, it's also going to hang on serversocket.accept() due to there being nothing to accept. But assuming you have...
while True:
#accept connections from outside
#print('In while true loop') This works, but we never get to the next print statement. Why the hell is it catching at line 20?
(clientsocket, address) = serversocket.accept()
#clientsocket = serversocket.accept()
It hangs because the loop never exits due to True always being True. In the example provided, once a connection is accepted they pretend that the server is threaded and the idea is to create a separate thread to begin reading and processing data received allowing the socket to continue to listen for more connections.
while True:
# accept connections from outside
(clientsocket, address) = serversocket.accept()
# now do something with the clientsocket
# in this case, we'll pretend this is a threaded server
ct = client_thread(clientsocket)
ct.run()