Good evening,
I am trying to make a basic implementation of the clients-to-server model using sockets.
Basically, I want the client to retrieve a small piece of information (IE: MESSAGE key) to the server. To accomplish this, I've used threading.
When the client sends the request to receive the info, there is no error on the client side.
However, the server sends an error
Exception in thread Thread-2:
Traceback (most recent call last):
File "C:\Users\Stock\AppData\Local\Continuum\anaconda3\lib\threading.py", line 926, in _bootstrap_inner
self.run()
File "C:\Users\Stock\AppData\Local\Continuum\anaconda3\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "server.py", line 40, in listeningForRetriever
dataBytes = conn.recv(1024)
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
After doing some research, this thread seemed pretty relevant, https://stackoverflow.com/a/49289545/6902431, but I had already implemented the suggested fixes.
server.py
import socket
import threading
from Utility import *
HOST = '127.0.0.1' # Standard loopback interface address (localhost)
PORT = 65432 # Port to listen on (non-privileged ports are > 1023)
PORT_A = 22221
PORT_B = 22222
PORT_C = 22223
bank = {'me':'you'}
def listeningForRetriever():
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind((HOST, PORT_B))
s.listen()
conn, addr = s.accept()
with conn:
while True:
dataBytes = conn.recv(1024)
dataString = bytesToString(dataBytes)
try:
retrievedValueBytes = bytesToString(bank[cleanGET(dataString)])
print(f"Value: {retrievedValueBytes} has been retrieved with Key {cleanGET(dataString)}")
conn.sendall(retrievedValueBytes)
except:
print("SERVER: Errored out")
conn.sendall(stringToBytes("ERROR! Value retrieval for command: " + dataString + " doesn't exist!"))
retrieverThread = threading.Thread(target=listeningForRetriever)
retrieverThread.start()
retriever.py
#Retrieves values from server using keys
#retriever.py --key=username
import sys
import socket
from Utility import *
# total arguments
n = len(sys.argv)
print("Total arguments passed:", n)
# Arguments passed
print("\nName of Python script:", sys.argv[0])
print("\nArguments passed:", end = " ")
key = ''
value = ''
for i in range(1, n):
print(sys.argv[i], end = " ")
if '--key=' in sys.argv[i]:
data = sys.argv[i].replace('--key=','')
key = data
print()
if key == '':
print("ERROR! Invalid syntax.")
print("Try something like: Bob.py --key=username")
else:
print(f"Key: {key} {type(key)}")
#TODO: Send key to storage of server
HOST = '127.0.0.1' # The server's hostname or IP address
PORT = 22222 # The port used by the server
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((HOST, PORT))
s.sendall(stringToBytes("GET key"+"="+key))
print("DEBUG 1")
value = s.recv(1024)
print("DEBUG 2")
print(f"Retrieved Value: {value}")
print(f"Value Retrieved: {value}")
s.close()
What am I overlooking/misunderstanding?
I could (more or less...) reproduce and fix. But I may be wrong, because your code uses an Utility module that you failed to show so I could only guess from the names what bytesToString, stringToBytes and cleanGET were supposed to do.
First there is a misunderstanding about how TCP actually behaves. It is a streaming protocol, that guarantees that all sent bytes will be received an in same order. But there is no guarantee on the packets themselves: they may be splitted and/or reassembled by the network. So you must use a higher level way to delimit a message. It does not exhibit any problem here because short packets are often left unmodified, but the protocol does not guarantee that so you could experience weird errors later. Here as you send a single message, you could shut down the socket to signal the end of the message: the receiver will see a 0 size packet at that point.
Next (and the actual cause of your problem), you loop after accepting a connection. So you get a message, successfully process it, read again on a socket that will be closed by the peer and get a 0 size byte string. It probably causes a key error when looking in the bank dict, but you silently swallow that error message (which is BAD), so you try to write on a socket closed by the peer, which causes the connection to be aborted. And finally on next recv you get the error that you see.
What you should have learned from that:
never ever use a silent try: ... except: ... that swallows any exception without letting you know what has happened. Always limit the filtered exception to the smaller expected set, or (at least and at dev time) display the caught exception to make sure you have not inadvertently caught an unwanted one.
use a higher level protocol to delimit messages on TCP
Now for the fixes:
retriever.py
...
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((HOST, PORT))
s.sendall(stringToBytes("GET key"+"="+key))
s.shutdown(socket.SHUT_WR) # signals the end of the message
print("DEBUG 1")
...
server.py
...
s.listen()
conn, addr = s.accept()
with conn:
# read until the end of a message signaled by a 0 size packet
dataBytes = b''
while True:
b = conn.recv(1024)
if len(b) == 0:
break
dataBytes += b
dataString = bytesToString(dataBytes)
try:
retrievedValueBytes = stringToBytes(bank[cleanGET(dataString)])
print(f"Value: {retrievedValueBytes} has been retrieved with Key {cleanGET(dataString)}")
conn.sendall(retrievedValueBytes)
except Exception as e:
print(e)
print("SERVER: Errored out")
conn.sendall(stringToBytes("ERROR! Value retrieval for command: " + dataString + " doesn't exist!"))
...
Here the thread stops after one single message. If you want the server to be able to accept many clients, you can loop over accept:
s.listen()
while True:
conn, addr = s.accept()
...
Your server is coded to listen in a loop (while True / conn.recv), but your
client only listens once, then exits, which closes the connection.
So the message on the server side is actually not vague at all, it's perfectly to
the point: it's telling you that the connection has been closed. It has indeed,
by the client.
(To be very specific, your retriever code uses a socket as a context manager,
meaning that when you exit the 'with' bock, the socket gets automatically
closed - you can remove the s.close() call, BTW)
Your code works fine, you should be getting the data back (I did, but your post
doesn't say anything about that). The only thing missing is the exception
handling on the server side to gracefully handle the client disconnection.
Related
I am making a script that will allow multiple clients to see live camera footage from the server script, this all works fine until one of the client scripts is closed, then a ConnectionResetError is raised, To avoid this I used a try and except block to catch the ConnectionResetError but the same error is raised every time after a connection is lost. Using just socket.recv stops the ConnectionResetError but socket.recv does not return the senders address which is required for the script to send the video stream back to the client.
Server:
host = "0.0.0.0"
port = 5000
buffer_size = 1024
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(("", port))
listeners = list() # the addresses of the clients that want the video stream
def handle_queue(sock):
while True:
try:
message, address = sock.recvfrom(buffer_size) # block the thread until a packet arrives
print(address)
message = str(message, "utf-8") # decode the message
if message == "join":
listeners.append(address) # add the list of listeners
else:
print("unknown queue msg: ", message)
except ConnectionResetError:
print("The connection was forcefully quit")
queue_handler_thread = Thread(target=handle_queue, args=(sock,), daemon=True)
queue_handler_thread.start() # start the queue
the script then uses sock.sendto() for each address in the listeners list
Client:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(bytes("join","utf-8"), (host, port))
while True:
data, address = sock.recvfrom(max_length) # block main thread until a packet is received
I believe what you are looking for is socket.getpeername().
This will return the remote address to which the socket is connected.
Your approach in handling the data in the wire whilst the connection is closed/lost is the right way. Handling via try/catch or using a recv() or recvfrom() method to wait for a response before closing the socket.
Since posting this i have discovered that it is infact TCP and this will not work for the project I am trying to do. Any more guidance and help would be appreciated, I cant seem to find any examples of Multiclient UDP Servers that run through one socket.
I partially resolved my system, after using the Example provided by AztCrw4282.
I was able to find a solution, I instead used the socket.accept() aproach, I am unsure as to whether this is UDP or TCP (I want UDP but with the connect system I think that is the handshake for TCP, but I am not 100% sure), but it works for now.
The client has to connect to the server then the server will accept or deny the connection, upon accepting a thread will be created for that clients connection to be managed. If any errors are thrown when interacting with that client their connection will be closed.
Server
try:
ServerSocket.bind((host, port))
except socket.error as e:
print(str(e))
print('Waiting for a Connection..')
ServerSocket.listen(5)
connections = list()
def threaded_client(connection):
connection.send(str.encode('Welcome to the Server\n'))
while True:
try:
data = str(connection.recv(2048),"utf-8") # this needs to be try catched
print("Packet Recv: ", data)
if data == "join":
print("Client Joined")
if data == "quit":
break
if not data:
break
except ConnectionResetError:
break
print("Closing a connection") # need to handle leaving the stream
connection.close()
def handle_stream():
for connection in connections:
try:
connection.send(bytes(json.dumps(frame_info) ,"utf-8"))
except:
print("Packet send failure, kicking client")
connections.remove(connection)
while True:
Client, address = ServerSocket.accept()
print('Connected to: ' + address[0] + ':' + str(address[1]))
connections.append(Client)
Thread(target=threaded_client, args=(Client, ), daemon=True).start()
ThreadCount += 1
print('Thread Number: ' + str(ThreadCount))
The only part that changes for the client is the part where it connects to the server
try:
ClientSocket.connect((host, port))
except socket.error as e:
print(str(e))
This question already has an answer here:
Python Bidirectional TCP Socket Hanging on socket.recv
(1 answer)
Closed 3 years ago.
I'm trying to make a client-server application in which:
the client sends a message (name of a function) to the server
the server receives the message, calls the corresponding function and returns the results to the client.
I'm able to do this only if the message from the client is one; if I want to call two or more functions, and therefore send two or more messages, I get some problems because I need to add a loop inside the server.
In particular, what is not clear to me is the socket function recv(). Usually, to receive data I write something like this (without setblocking()) and I never had any problems:
while True:
data = sock.recv(BUFFER)
results += data.decode()
if not data:
break
But if I add a while True: to the server (in order to wait for the other functions), then the client never exits from sock.rev(). Why? I expect that function to be always blocking or always non-blocking based on how it was set sock.setblocking().
I already tried with settimeout() and it worked, but performances are really important so I would like to avoid every possible delay.
Anyway, the main question is why recv() is behaving differently; then if you had any suggestions to solve my task it would be really appreciated.
Here's a simple simulation of the real application.
Client:
import socket
BUFFER = 1024
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address= ('localhost', 14000)
sock.connect(bridge_address)
msg1 = "function 1".encode()
msg2 = "function 2".encode()
results1 = " "
results2 = " "
sock.sendall(msg1)
while True:
data = sock.recv(BUFFER)
results1 += data.decode()
if not data:
print('no more data')
break
sock.sendall(msg2)
while True:
data = sock.recv(BUFFER)
results2 += data.decode()
if not data:
print('no more data')
break
sock.close()
print('Results 1: ',results1)
print('Results 2: ',results2)
Server:
import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('localhost', 14000)
sock.bind(server_address)
sock.listen(1)
msg = ""
for i in range(4096):
msg += str(i) + " "
while True:
print('waiting for a connection')
client, client_address = sock.accept()
try:
while True:
data = client.recv(128)
if not data:
print('no more data from')
break
client.sendall(msg.encode())
finally:
print('connection closed.\n')
client.close()
You are explicitly forcing the server side socket to recieve the message from the client, even when the client is not sending anything.
data = client.recv(128)
if not data:
print('no more data from')
break
if case in the code snippet above will only execute when the client socket goes down.
One method to solve this would be to first send the size of the message and then call recv accordingly for the specified number of bytes.
I am currently building an automated trash bin using Raspberry Pi 3 B+ with Android application support where I would use a servo motor as an actuator for the lid and the Android application as a form of wireless remote control. Everything went on smoothly until I've encountered a problem that whenever I attempt to press a button on my Android application, the Python shell program has errors during testing. I've used a reference video (https://www.youtube.com/watch?v=t8THp3mhbdA&t=1s) and followed everything thoroughly until I've hit this roadblock.
The results to me that keeps appearing are:
Waiting for connection
...connected from :
Where the supposed result, according to the reference video, is:
Waiting for connection
...connected from : ('192.168.1.70', 11937)
Increase: 2.5
As you can see, the IP address, the port, and 'Increase' text doesn't appear, meaning there is something wrong with the code.
According to some comments that was made by the people who watched the video, this code is outdated, using Python 2, and the latest version we have now is Python 3, and that we need to use a ".encode()" line in a condition. However, as someone who is still new to Python, I'm afraid that I still don't have the knowledge to apply this on the code.
Here is the code that was used in the video:
import Servomotor
from socket import *
from time import ctime
import RPi.GPIO as GPIO
Servomotor.setup()
ctrCmd = ['Up','Down']
HOST = ''
PORT = 21567
BUFSIZE = 1024
ADDR = (HOST,PORT)
tcpSerSock = socket(AF_INET, SOCK_STREAM)
tcpSerSock.bind(ADDR)
tcpSerSock.listen(5)
while True:
print 'Waiting for connection'
tcpCliSock,addr = tcpSerSock.accept()
print '...connected from :', addr
try:
while True:
data = ''
data = tcpCliSock.recv(BUFSIZE)
if not data:
break
if data == ctrCmd[0]:
Servomotor.ServoUp()
print 'Increase: ',Servomotor.cur_X
if data == ctrCmd[1]:
Servomotor.ServoDown()
print 'Decrease: ',Servomotor.cur_X
except KeyboardInterrupt:
Servomotor.close()
GPIO.cleanup()
tcpSerSock.close();
I have already changed the text strings that used the ' ' format into the (" ") format since it also produced some errors in the code which I corrected immediately.
Any help will be greatly appreciated and thank you in advance!
Here's a Python3 version, edited a tiny bit for better clarity and good practice:
import Servomotor
import RPi.GPIO as GPIO
import socket
# Setup the motor
Servomotor.setup()
# Declare the host address constant - this will be used to connect to Raspberry Pi
# First values is IP - here localhost, second value is the port
HOST_ADDRESS = ('0.0.0.0', 21567)
# Declare the buffer constant to control receiving the data
BUFFER_SIZE = 4096
# Declare possible commands
commands = 'Up', 'Down'
# Create a socket (pair of IP and port) object and bind it to the Raspberry Pi address
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(HOST_ADDRESS)
# Set the socket to listen to an incoming connection (1 at a time)
server_socket.listen(1)
# Never stop the server once it's running
while True:
# Inform that the server is waiting for a connection
print("Waiting for connection to the following address: {}...".format(HOST_ADDRESS))
# Perform a blocking accept operation to wait for a client connection
client_socket, client_address = server_socket.accept()
# Inform that the client is connected
print("Client with an address {} connected".format(client_address))
# Keep exchanging data
while True:
try:
# Receive the data (blocking receive)
data = client_socket.recv(BUFFER_SIZE)
# If 0-byte was received, close the connection
if not data:
break
# Attempt to decode the data received (decode bytes into utf-8 formatted string)
try:
data = data.decode("utf-8").strip()
except UnicodeDecodeError:
# Ignore data that is not unicode-encoded
data = None
# At this stage data is correctly received and formatted, so check if a command was received
if data == commands[0]:
Servomotor.ServoUp()
print("Increase: {}".format(Servomotor.cur_X))
elif data == commands[1]:
Servomotor.ServoDown()
print("Decrease: {}".format(Servomotor.cur_X))
elif data:
print("Received invalid data: {}".format(data))
# Handle possible errors
except ConnectionResetError:
break
except ConnectionAbortedError:
break
except KeyboardInterrupt:
break
# Cleanup
Servomotor.close()
GPIO.cleanup()
client_socket.close()
# Inform that the connection is closed
print("Client with an address {} disconnected.".format(client_address))
To show you the code in action, I have hosted a local server on my machine and connected to it using Putty. Here are the commands I have entered:
Here is the output of the server (I have swapped the Servo-related functions to print statements):
Waiting for connection to the following address: ('0.0.0.0', 21567)...
Client with an address ('127.0.0.1', 61563) connected.
Received invalid data: Hello
Received invalid data: Let's try a command next
Running ServoUp
Increase: 2.5
Running ServoDown
Decrease: 2.5
Received invalid data: Nice!
Client with an address ('127.0.0.1', 61563) disconnected.
Waiting for connection to the following address: ('0.0.0.0', 21567)...
I've written a basic client/server interface using Python socket (quoted only relevant part of code, for full script: (SERVER: https://github.com/mydomo/ble-presence/blob/master/server.py)
(CLIENT: https://github.com/mydomo/ble-presence/blob/master/clients/DOMOTICZ/ble-presence/plugin.py)
The issue is when the script run from some hours and the result list is getting bigger sometimes the reply is exactly as it should be, other times it's cutted, not complete... it's random, like if the socket closed for no reason earlier or the reply is not fully read.
Can you please help me?
SERVER:
def client_thread(conn, ip, port, MAX_BUFFER_SIZE = 32768):
# the input is in bytes, so decode it
input_from_client_bytes = conn.recv(MAX_BUFFER_SIZE)
# MAX_BUFFER_SIZE is how big the message can be
# this is test if it's too big
siz = sys.getsizeof(input_from_client_bytes)
if siz >= MAX_BUFFER_SIZE:
print("The length of input is probably too long: {}".format(siz))
# decode input and strip the end of line
input_from_client = input_from_client_bytes.decode("utf8").rstrip()
res = socket_input_process(input_from_client)
#print("Result of processing {} is: {}".format(input_from_client, res))
vysl = res.encode("utf8") # encode the result string
conn.sendall(vysl) # send it to client
conn.close() # close connection
##########- END FUNCTION THAT HANDLE SOCKET'S TRANSMISSION -##########
def start_server():
global soc
soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# this is for easy starting/killing the app
soc.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
#print('Socket created')
try:
soc.bind((socket_ip, socket_port))
# print('Socket bind complete')
except socket.error as msg:
# print('Bind failed. Error : ' + str(sys.exc_info()))
sys.exit()
#Start listening on socket
soc.listen(10)
#print('Socket now listening')
# for handling task in separate jobs we need threading
#from threading import Thread
# this will make an infinite loop needed for
# not reseting server for every client
while (not killer.kill_now):
conn, addr = soc.accept()
ip, port = str(addr[0]), str(addr[1])
#print('Accepting connection from ' + ip + ':' + port)
try:
Thread(target=client_thread, args=(conn, ip, port)).start()
except:
print("Terible error!")
import traceback
traceback.print_exc()
soc.close()
CLIENT:
soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
SERV_ADDR = str(Parameters["Address"])
SERV_PORT = int(Parameters["Port"])
soc.connect((SERV_ADDR, SERV_PORT))
if BATTERY_REQUEST == True:
clients_input = str(BATTERY_DEVICE_REQUEST)
else:
clients_input = "beacon_data"
soc.send(clients_input.encode()) # we must encode the string to bytes
result_bytes = soc.recv(32768) # the number means how the response can be in bytes
result_string = result_bytes.decode("utf8") # the return will be in bytes, so decode
Method recv() does not guarantee receiving the full message in the first call so you have to try getting the full message by calling recv() multiple times.
If recv() does return an empty string, connection is closed in the client side.
Using this while loop you can get full stream from client into data:
data = b'' # recv() does return bytes
while True:
try:
chunk = conn.recv(4096) # some 2^n number
if not chunk: # chunk == ''
break
data += chunk
except socket.error:
conn.close()
break
TCP is a streaming protocol, meaning it has no concept of what constitutes a complete message. You have to implement your own message protocol layer on top of TCP to make sure you send and receive complete messages. You are responsible for buffering data received until you have a complete message, and you have to define what a complete message is. Some options:
Send fixed length messages.
Send a fixed number of bytes representing the length of the message, then the message.
Separate messages with a sentinel byte.
Then, call recv and accumulate the results until you have a complete message in the buffer.
My socket program hangs at clientsocket, address) = serversocket.accept() and doesn't spit our an error or anything.
I followed directions on https://docs.python.org/3/howto/sockets.html
I've been trying to figure it out for an hour now, but to no avail. I'm using python3 btw. What am i doing wrong? EDIT: My intedentation is all screwed up because I pasted it wrong, but other than that my code is as I have it in my file.
#import socket module
import socket
#creates an inet streaming socket.
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print('socket created')
#binds socket to a public host, and a well known port
serversocket.bind(('127.0.0.1', 1024))
#print(socket.gethostname())# on desktop prints 'myname-PC')
#become a server socket
serversocket.listen(5) # listens for up to 5 requests
while True:
#accept connections from outside
#print('In while true loop') This works, but we never get to the next print statement. Why the hell is it catching at line 20?
(clientsocket, address) = serversocket.accept()
#clientsocket = serversocket.accept()
print('Ready to serve')
#now we do something with client socket...
try:
message = clientsocket.recv(1024)
filename = message.split()[1]
f = open(filename[1:])
outputdata = f.read()
#send an http header line
clientsocket.send('HTTP/1.1 200 OK\nContent-Type: text/html\n\n')
for i in range(0, len(outputdata)):
clientsocket.send(outputdata[i])
clientsocket.close()
except IOERROR:
clientsocket.send('HTTP/1.1 404 File not found!')
clientsocket.close()
If you haven't written a client script / program to connect to the socket and send it data, it's also going to hang on serversocket.accept() due to there being nothing to accept. But assuming you have...
while True:
#accept connections from outside
#print('In while true loop') This works, but we never get to the next print statement. Why the hell is it catching at line 20?
(clientsocket, address) = serversocket.accept()
#clientsocket = serversocket.accept()
It hangs because the loop never exits due to True always being True. In the example provided, once a connection is accepted they pretend that the server is threaded and the idea is to create a separate thread to begin reading and processing data received allowing the socket to continue to listen for more connections.
while True:
# accept connections from outside
(clientsocket, address) = serversocket.accept()
# now do something with the clientsocket
# in this case, we'll pretend this is a threaded server
ct = client_thread(clientsocket)
ct.run()