I am writing a UDP server application that serves as a back end to Teltonika FMB630 car mounted devices.
I already took care of the protocol specifics and decoding, the problem I am facing relates to the UDP socket used.
My UDP server has to send an acknowledgement to the client device upon receiving a message (that is the protocol), however, if I send those ACKs, the server socket stops receiving data after a while.
The server's UDP socket object is passed to an concurrent.futures.ThreadPoolExecutor that fires a function (send_ack) that sends the ACK, however this is not the issue because I tried calling send_ack in the main thread, after receiving data and the same issue occurs.
I suspect the problem is the remote device somehow breaks the connection or the ISP or MNO doesn't route the reply packet (this is a GPRS device) and then the socket.send() method that is used to send the acknowledge, somehow freezes other socket operations, specifically recvfrom_into called in the main thread loop.
I wrote two scripts to illustrate the situation:
udp_test_echo.py :
#!/usr/env/bin python
import socket
import concurrent.futures
def send_ack(sock, addr, ack):
print("Sending ACK to {}".format(addr))
sock.connect(addr)
print("connected to {}".format(addr))
sock.send(ack)
print("ACK sent to {}".format(addr))
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind(("127.0.0.1", 1337))
data = bytearray([0] * 10)
executor = concurrent.futures.ThreadPoolExecutor(max_workers=4)
while True:
print("listening")
nbytes, address = s.recvfrom_into(data)
print("Socket Data received {} bytes Address {}".format(nbytes, address))
print("Data received: ", data, " Echoing back to client")
executor.submit(send_ack, s, address, data[:nbytes])
udp_test_client.py:
#!/usr/env/bin python
import socket
import time
import random
def get_random_bytes():
return bytearray([random.randint(0,255) for b in range(10)])
ip = "127.0.0.1"
port = 1337
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect((ip, port))
while True:
stuff_to_send = get_random_bytes()
print("Sending stuff", stuff_to_send)
s.sendall(stuff_to_send)
print("reply: ", s.recvfrom(10))
time.sleep(0.1)
Running udp_test_echo.py in one terminal and udp_test_client.py in another, we see normal operation but if you Ctrl+C the test client and re run it, you will see that the server doesn't respond until it is restarted.
Is there a way to timeout a specific sending operation from a specific call to socket.send() method without affecting other calls ? (I want my socket.recvfrom_into call to block on the main thread)
If I settimeout on the entire socket object, I am going to have to deal with many exceptions while waiting for data in the main thread and I don't like to have to rely on exceptions for proper program operation.
The culprit was the socket.connect() call in send_ack, when being called on the server's socket object it causes the socket to no longer be bound and listen on the port specified in the start of the program.
Instead the send_ack function was changed to be:
def send_ack(sock, addr, ack):
print("Sending ACK to {}".format(addr))
sock.sendto(ack, addr)
print("ACK sent to {}".format(addr))
socket.sendto(data, address) uses the existing connection instead of starting a new one.
Related
I'm transmitting some live data from a camera to a client over ethernet where i'm using a python socket and opencv to transmit the images. It works fine and i can read the video at the client side.
The issue is that when i start the server that listens for users that want the feed and a socket stream is started, then after i close the connection at the client side the server side keeps transmitting the data. Furthermore, after closing the client side connection i can only reconnect by restarting the server.
I've though about implementing a kind of heartbeat protocol by sending a message from client to server for every 100th image i recieve to tell i'm still listening - and if not recieved then go back to the loop to listen for a new connection. The only issue is that the recvfrom is non-blocking so a direct implementation into the while loop would probably not work.
How can i detect that a/the client isn't reading the data anymore?
Here is my server side:
import cv2, imutils, socket
import numpy as np
import time
import base64
BUFF_SIZE = 65536
server_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, BUFF_SIZE)
host_name = socket.gethostname()
host_ip = 'xxx.xxx.xxx.xxx'
port = 4001
socket_address = (host_ip, port)
server_socket.bind(socket_address)
print("listening at", socket_address)
vid = cv2.VideoCapture(0)
WIDTH=480
while True:
msg,client_addr = server_socket.recvfrom(BUFF_SIZE)
print(client_addr)
while (vid.isOpened()):
print("sending img")
_,frame = vid.read()
frame = imutils.resize(frame,width=WIDTH)
encoded,buffer = cv2.imencode(".jpg",frame,[cv2.IMWRITE_JPEG_QUALITY,80])
message = base64.b64encode(buffer)
print(len(message))
server_socket.sendto(message,client_addr)
key = cv2.waitKey(1) & 0xFF
if key == ord('q'):
server_socket.close()
Here is my client side:
def video_socket_stream_client():
BUFF_SIZE = 65536
client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
client_socket.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, BUFF_SIZE)
host_name = socket.gethostname()
host_ip = 'xxx.xxx.xxx.xxx'
port = 4001
client_socket.sendto("connect".encode(),(host_ip, port)) # send signal
while True:
packet,_ = client_socket.recvfrom(BUFF_SIZE)
data = base64.b64decode(packet,' /')
npdata = np.frombuffer(data,dtype=np.uint8)
frame = cv2.imdecode(npdata,1)
time.sleep(0.016)
yield(b'--frame\r\n'b'Content-Type: image/jpeg\r\n\r\n' + data + b'\r\n')
This is one of the tradeoffs with UDP vs TCP. With UDP, there is no long-term connection. The sender just blasts data and hopes someone is listening. The listener just picks up the phone and hopes someone is talking. There is no persistent state. If the sender stops sending, the listener will not hear anything. If you need to maintain a connection, then you need to use TCP.
You shouldn't ever have to restart the server. Neither side knows what the other side is doing. I wonder if you're getting bit by not using SO_REUSEADDR.
The running status of the server can be judged by the return value of getattr(socket,'_closed').True is closed state, False is running.
e.g.
import socket
ip = 'localhost'
port = 5003
ws = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ws.bind((ip, port))
ws.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
ws.listen()
# 关闭服务
# ws.close()
print("The status is as follows：")
if(getattr(ws, '_closed') == False):
print("Running")
elif(getattr(ws, '_closed') == True):
print("closed")
You can try.
You could have the client send a "goodbye" type of packet to the server just before the client exits, to let the server know the client is going away, but of course you can't rely on that packet actually making it to the server (the packet might get dropped, or the client might crash or get physically disconnected from the network or otherwise go away in an uncontrolled fashion), so in the end you'll need to implement some kind of heartbeat protocol.
In order to send() and recv() simultaneously, you can use non-blocking I/O and select(), or you can use multiple threads. I prefer the select() approach, but either way can be made to work. You might also consider upgrading your server's logic to support multiple clients simultaneously rather than just one client-at-a-time; even if you don't actually need to send to multiple clients at once, that would allow your server to handle the case where the user quits the client and then starts a new one (i.e. the server would be able to start sending packets to the new client even before it has timed-out the old/dead client).
The title explain everything. I made Java code that sends messgaes for the python server, but evry time, just the first message is sends because every time, java conneced again to server, and the server keeps waiting to next message from the first client that I send in the first time.
How can the server get message from all clients are connectd? and not just from one?
My python server:
server = socket.socket()
server.bind((socket.gethostname(), 4786))
server.listen(5)
(client, (ipNum, portNum)) = server.accept()
print("Client connected")
while True:
message = str(client.recv(64).decode()) # Check if client send message. I want to change it to check all clients
if(message != ""):
print("Client: " + message)
else:
time.sleep(0.1)
Summary
The server.accept() must be called inside the loop.
TLDR
The socket server returned from the call socket.socket() is a 'listening' socket. It is not used for any data transfer but just for listening incoming connections. When the server is willing to accept incoming connection then calls server.accept(). This call waits till a client connects. When a client connects the accept wakes up and returns a socket that represents a connection to one client. This socket is then used for data send and received and should be closed when the communication with this specific client is done.
When server wants to accept connection from another client it must call server.accept() again to wait for connection from another client and use the unique client socket for each connected client.
If it sufficient to handle client sequentially then you can just move the call accept onto the loop. Furthermore you should close the client socket when the communication with the client is done.
If multiple clients can be connected in parallel then slightly more complicated design is needed. You can start a new thread for each client after accepting the connection. The thread can call recv in a loop and terminates when the client disconnects. See Multi Threaded TCP server in Python for example.
I've a python Socket server running and also socket clients.
Now, for example say there are 3 clients connected to same server. Please find below the code of the server.
#!/usr/bin/python # This is server.py file
import socket # Import socket module
import threading
serversocket = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = 1234 # Reserve a port for your service.
serversocket.bind((host, port)) # Bind to the port
serversocket.listen(5)
print("Bound the port ",port,"on Machine : ",host,", and ready to accept connections.\n")
def clientThread(connection):
while True:
data=connection.recv(1024)
if not data:
break
connection.send("Thanks")
connection.close()
def sendMessage(connection, message):
connection.send(message)
while 1:
connection, address = serversocket.accept()
start_new_thread(clientthread, (connection,))
serversocket.close();
Now, I need to call sendMessage for a particular client, say out of clients A,B and C, send it to B. In this case, how do I identify the thread and call that function?
You can use Queues and multiple threads per connection to solve this problem.
Basic outline:
Each client connection spawns two threads - one to monitor client input and another which monitors a Queue. Items placed on the queue will be sent to the client. Each client connection will have its own output queue.
You'll also need a global dictionary to map a client name to their output queue.
To send a message to a particular client, find the client's output queue and add the message to it.
You'll also need a way to shutdown the output thread for a client. A common approach is to use a sentinel value (like None) on the queue to inform the output thread to exit its processing loop. When the client's input thread detects EOF it can place the sentinel value on the client's output queue and eventually the output thread will shut itself down.
I'm trying to set up a small server where when the client logs in gets some messages.
The server code
import socket
#Networking
s = socket.socket()
print("Network successfully created")
port = 3642
s.bind(('',port))
print("Network has been binded to %s" %(port))
s.listen(5)
print("Waiting for connections")
while True:
c, addr = s.accept()
print("Got a connection from",addr)
c.send(bytes("Thank you for connecting to me. Currently we","utf-8"))
c.send(bytes("Working on the server","utf-8"))
c.close()
This is the client code
# Import socket module
import socket
# Create a socket object
s = socket.socket()
# Define the port on which you want to connect
port = 3642
# connect to the server on local computer
s.connect(('MyIp..', port))
# receive data from the server
print(s.recv(1024))
# close the connection
s.close()
Everything works fine such as the connecting and the first message gets printed, however I can't get the second message to get printed. The one that says working on the server. I have just began learning about sockets and barely know anything about them so the solution probably is obvious it's just
I can't seem to figure it out. Thank you for any responses. (I would appreciate thorough responses)
If the two sent buffers happen to not get consolidated into a single buffer in the recv (which can happen based on timing, which OS you're running and other factors), then it makes sense that you would not see the second buffer because you're only making one recv call. If you want to receive everything the server sent, put the recv in a loop until it returns an empty string. (Empty string indicates end-of-file [i.e. socket closed by the other end].) – Gil Hamilton
I'm trying to implement the most basic python TCP server. Windows 8, Python 2.7, firewall is turned off. Code is from here: https://wiki.python.org/moin/TcpCommunication
If I do the client stuff (socket(...), connect(...), send(...)) via python repl, things work fine, ie the server correctly blocks when calling recv.
However if I run the exact same code via python script (both with and without explicitly calling python.exe at windows command line), the recv returns immediately with no data. I read elsewhere on SO this means it's an invalid socket, but I'm not sure what that means or how to check for it. I'm using the socket returned by accept() not the one used to initiate the connection.
I'm trying to block on recv so I can take advantage of the timeout (I don't want to use select module, which BTW also returns immediately) and process some keyboard stuff between attempts to recv, ie user presses 'q' to quit.
In various experiments I've shown that once this occurs, recv will always return immediately (as will select.select(...)) if I put it in a loop, so it's not like the client is sending a single "bad" packet initially. If the client happens to have sent something, then the recv returns with that data, but it certainly doesn't block waiting for data when put in a tight loop.
Is this behavior expected?
Server code:
import sys
import socket
TCP_IP = '192.168.1.10'
TCP_PORT = 5005
BUFFER_SIZE = 20 # Normally 1024, but we want fast response
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((TCP_IP, TCP_PORT))
s.listen(1)
conn, addr = s.accept()
print 'Connection address:', addr
while 1:
data = conn.recv(BUFFER_SIZE) # This returns immediately with no data, when client connection is run from script and doesn't send() anything, just connects.
if not data:
print "broken"
break
print "received data:", data
conn.send(data) # echo
conn.close()
sys.exit()
Client code:
import sys
import socket
TCP_IP = '192.168.1.10'
TCP_PORT = 5005
BUFFER_SIZE = 1024
MESSAGE = "Hello, World!"
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
# Commenting out the following to prove the recv() call on the other
#end returns with nothing instead of blocking indefinitely. If I
#type the rest of this at the REPL the server behaves correctly,
#ie, the recv call blocks forever until socket.send("bla") from client.
#s.send(MESSAGE) data = s.recv(BUFFER_SIZE)
#s.close()
#print "received data:", data
sys.exit()
Yes, this is expected behavior.
The client does not send anything. And it exit as soon as it connect to the server; cause disconnection.
socket.recv returns an empty string if the peer performed shutdown (disconnect).
While, in the REPL, the socket is not closed until you issue sys.exit() or you quit the interactive shell.