I'm transmitting some live data from a camera to a client over ethernet where i'm using a python socket and opencv to transmit the images. It works fine and i can read the video at the client side.
The issue is that when i start the server that listens for users that want the feed and a socket stream is started, then after i close the connection at the client side the server side keeps transmitting the data. Furthermore, after closing the client side connection i can only reconnect by restarting the server.
I've though about implementing a kind of heartbeat protocol by sending a message from client to server for every 100th image i recieve to tell i'm still listening - and if not recieved then go back to the loop to listen for a new connection. The only issue is that the recvfrom is non-blocking so a direct implementation into the while loop would probably not work.
How can i detect that a/the client isn't reading the data anymore?
Here is my server side:
import cv2, imutils, socket
import numpy as np
import time
import base64
BUFF_SIZE = 65536
server_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, BUFF_SIZE)
host_name = socket.gethostname()
host_ip = 'xxx.xxx.xxx.xxx'
port = 4001
socket_address = (host_ip, port)
server_socket.bind(socket_address)
print("listening at", socket_address)
vid = cv2.VideoCapture(0)
WIDTH=480
while True:
msg,client_addr = server_socket.recvfrom(BUFF_SIZE)
print(client_addr)
while (vid.isOpened()):
print("sending img")
_,frame = vid.read()
frame = imutils.resize(frame,width=WIDTH)
encoded,buffer = cv2.imencode(".jpg",frame,[cv2.IMWRITE_JPEG_QUALITY,80])
message = base64.b64encode(buffer)
print(len(message))
server_socket.sendto(message,client_addr)
key = cv2.waitKey(1) & 0xFF
if key == ord('q'):
server_socket.close()
Here is my client side:
def video_socket_stream_client():
BUFF_SIZE = 65536
client_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
client_socket.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, BUFF_SIZE)
host_name = socket.gethostname()
host_ip = 'xxx.xxx.xxx.xxx'
port = 4001
client_socket.sendto("connect".encode(),(host_ip, port)) # send signal
while True:
packet,_ = client_socket.recvfrom(BUFF_SIZE)
data = base64.b64decode(packet,' /')
npdata = np.frombuffer(data,dtype=np.uint8)
frame = cv2.imdecode(npdata,1)
time.sleep(0.016)
yield(b'--frame\r\n'b'Content-Type: image/jpeg\r\n\r\n' + data + b'\r\n')
This is one of the tradeoffs with UDP vs TCP. With UDP, there is no long-term connection. The sender just blasts data and hopes someone is listening. The listener just picks up the phone and hopes someone is talking. There is no persistent state. If the sender stops sending, the listener will not hear anything. If you need to maintain a connection, then you need to use TCP.
You shouldn't ever have to restart the server. Neither side knows what the other side is doing. I wonder if you're getting bit by not using SO_REUSEADDR.
The running status of the server can be judged by the return value of getattr(socket,'_closed').True is closed state, False is running.
e.g.
import socket
ip = 'localhost'
port = 5003
ws = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ws.bind((ip, port))
ws.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
ws.listen()
# 关闭服务
# ws.close()
print("The status is as follows:")
if(getattr(ws, '_closed') == False):
print("Running")
elif(getattr(ws, '_closed') == True):
print("closed")
You can try.
You could have the client send a "goodbye" type of packet to the server just before the client exits, to let the server know the client is going away, but of course you can't rely on that packet actually making it to the server (the packet might get dropped, or the client might crash or get physically disconnected from the network or otherwise go away in an uncontrolled fashion), so in the end you'll need to implement some kind of heartbeat protocol.
In order to send() and recv() simultaneously, you can use non-blocking I/O and select(), or you can use multiple threads. I prefer the select() approach, but either way can be made to work. You might also consider upgrading your server's logic to support multiple clients simultaneously rather than just one client-at-a-time; even if you don't actually need to send to multiple clients at once, that would allow your server to handle the case where the user quits the client and then starts a new one (i.e. the server would be able to start sending packets to the new client even before it has timed-out the old/dead client).
Related
I'm building a Client-Server connection for an assignment. The server basically runs everything and the client only receives messages from the server and responds when the server needs it to.
For some reason, I can't build a client that repeatedly listens and then responds.
I thought that I could just listen with socket.recv() and it ended up just stopping everything.
this is what I ended up with after some tweaking (for the client) :
import socket
HOST = 'localhost'
PORT = 65432
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.connect((HOST, PORT))
conn.setblocking(False)
while True :
try :
msg_lgn = len(conn.recv(1024,socket.MSG_PEEK))
server_message = conn.recv(1024).decode()
print('Server Sent:\n' + server_message)
if server_message == 'exit':
break
except :
conn.send(input().encode())
conn.close()
I set it to non-blocking so it won't hang on the conn.recv() and move on to input in the exception, but then it just freezes on the input.
does the server close the connection every time recv() gets nothing ? why is this happening ?
I just want the client to receive messages whenever the server sends them, and when the server doesn't send, the client will send the server it's input.
would appreciate any help!
Lidor
Edit : the server file is much bigger, so i'll show the important parts.
#imported some classes for the game itself also (that i've created), but has nothing to do with the problem
import socket
import time
import threading
def startGame(conn):
#this is where the servers sends the questions and receives answer (pretty basic send and recv)
# Establish Client-Server Connection
HOST = 'localhost' # Standard loopback interface address (localhost)
PORT = 65432 # Port to listen on (non-privileged ports are > 1023)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(3)
# Use Threading to start the game for an new player
while True:
if playerCount < 3:
(conn, addr) = s.accept()
playerCount += 1
x = threading.Thread(target=startGame, args=(conn,))
x.start()
# Close Server Socket
s.close()
I am writing a UDP server application that serves as a back end to Teltonika FMB630 car mounted devices.
I already took care of the protocol specifics and decoding, the problem I am facing relates to the UDP socket used.
My UDP server has to send an acknowledgement to the client device upon receiving a message (that is the protocol), however, if I send those ACKs, the server socket stops receiving data after a while.
The server's UDP socket object is passed to an concurrent.futures.ThreadPoolExecutor that fires a function (send_ack) that sends the ACK, however this is not the issue because I tried calling send_ack in the main thread, after receiving data and the same issue occurs.
I suspect the problem is the remote device somehow breaks the connection or the ISP or MNO doesn't route the reply packet (this is a GPRS device) and then the socket.send() method that is used to send the acknowledge, somehow freezes other socket operations, specifically recvfrom_into called in the main thread loop.
I wrote two scripts to illustrate the situation:
udp_test_echo.py :
#!/usr/env/bin python
import socket
import concurrent.futures
def send_ack(sock, addr, ack):
print("Sending ACK to {}".format(addr))
sock.connect(addr)
print("connected to {}".format(addr))
sock.send(ack)
print("ACK sent to {}".format(addr))
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind(("127.0.0.1", 1337))
data = bytearray([0] * 10)
executor = concurrent.futures.ThreadPoolExecutor(max_workers=4)
while True:
print("listening")
nbytes, address = s.recvfrom_into(data)
print("Socket Data received {} bytes Address {}".format(nbytes, address))
print("Data received: ", data, " Echoing back to client")
executor.submit(send_ack, s, address, data[:nbytes])
udp_test_client.py:
#!/usr/env/bin python
import socket
import time
import random
def get_random_bytes():
return bytearray([random.randint(0,255) for b in range(10)])
ip = "127.0.0.1"
port = 1337
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect((ip, port))
while True:
stuff_to_send = get_random_bytes()
print("Sending stuff", stuff_to_send)
s.sendall(stuff_to_send)
print("reply: ", s.recvfrom(10))
time.sleep(0.1)
Running udp_test_echo.py in one terminal and udp_test_client.py in another, we see normal operation but if you Ctrl+C the test client and re run it, you will see that the server doesn't respond until it is restarted.
Is there a way to timeout a specific sending operation from a specific call to socket.send() method without affecting other calls ? (I want my socket.recvfrom_into call to block on the main thread)
If I settimeout on the entire socket object, I am going to have to deal with many exceptions while waiting for data in the main thread and I don't like to have to rely on exceptions for proper program operation.
The culprit was the socket.connect() call in send_ack, when being called on the server's socket object it causes the socket to no longer be bound and listen on the port specified in the start of the program.
Instead the send_ack function was changed to be:
def send_ack(sock, addr, ack):
print("Sending ACK to {}".format(addr))
sock.sendto(ack, addr)
print("ACK sent to {}".format(addr))
socket.sendto(data, address) uses the existing connection instead of starting a new one.
Good afternoon everyone reading this, I am new to programming with sockets, as well as new to asynchronous coding (I feel async may be part of the solution to my problem), so forgive me for any silly mistakes I make.
To start, I have a UDP Echo server that acts as a game server. Anytime it gets a ping sent to it, it adds the source ip and port to a list of "connected clients", and sends that exact ping out to everyone on the list, excluding the sender. This works fairly well, because it reacts upon receiving a message, so it can always just listen. The problem with the client however, is that I need to be constantly sending pings, while also listening.
This is currently what my client looks like:
import socket
from time import sleep
from contextlib import contextmanager
UDP_IP_ADDRESS = "127.0.0.1"
UDP_PORT_NO = 14004
Message = b"Hello World, From Client B"
#contextmanager
def socket_ctx():
""" Context Manager for the socket. Makes sure the socket will close regardless of why it exited."""
my_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# Assign IP address and a RANDOM available port number to socket
my_socket.bind(('127.0.0.1', 0))
try:
# Let the rest of the app use the socket and wait for it to finish
yield my_socket
finally:
my_socket.close()
def send_data(client_sock):
client_sock.sendto(Message, (UDP_IP_ADDRESS, UDP_PORT_NO))
def listen(client_sock):
print(client_sock.recvfrom(100))
with socket_ctx() as sock:
while True:
send_data(sock)
listen(sock)
sleep(2)
Currently, it sends a ping once, then just idles as it presumably is listening. If it does happen to get a ping back, say, another client send a ping to the server, and the server sent the ping to this client, it hears it, prints it, and starts the loop again. The issue is, without another client sending something to jolt this one out of the listen, it doesn't send it's pings.
I think async might be my solution, but I would have no clue how to go about that. Does anyone have a solution for this problem?
Here's how I would implement a server with "receive and handle incoming UDP sockets, plus do some packet-sending once per second" behavior. Note that this uses the select() function to multiplex the two tasks, rather than asynchronous I/O; hopefully that is okay.
import socket
import select
import time
UDP_IP_ADDRESS = "127.0.0.1"
UDP_PORT_NO = 14004
Message = b"Hello World, From Client B"
udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
udp_socket.bind(('127.0.0.1', 0))
print "UDP socket is listening for incoming packets on port", udp_socket.getsockname()[1]
# When we want to send the next periodic-ping-message out
nextPingTime = time.time()
while True:
secondsUntilNextPing = nextPingTime - time.time();
if (secondsUntilNextPing < 0):
secondsUntilNextPing = 0
# select() won't return until udp_socket has some data
# ready-for-read, OR until secondsUntilNextPing seconds
# have passed, whichever comes first
inReady, outReady, exReady = select.select([udp_socket], [], [], secondsUntilNextPing)
if (udp_socket in inReady):
# There's an incoming UDP packet ready to receive!
print(udp_socket.recvfrom(100))
now = time.time()
if (now >= nextPingTime):
# Time to send out the next ping!
print "Sending out scheduled ping at time ", now
udp_socket.sendto(Message, (UDP_IP_ADDRESS, UDP_PORT_NO))
nextPingTime = now + 1.0 # we'll do it again in another second
I am working with a relay that is controlled via TCP. As far as I understood the following code is supposed to work:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('192.168.0.200', 17494))
s.send(chr(101))
s.close()
However, I noticed that the socket gets closed before the package is actually send, and the relay does not do anything. As dirty solution I now put a sleep statement before closing the connection and it works properly.
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('192.168.0.200', 17494))
s.send(chr(101))
time.sleep(0.01)
s.close()
Is there something more clever one can do to ensure that the package got actually send before closing the connection?
You could set the SO_LINGER option using s.setsockopt. The linger option makes the socket wait (internally) and close only after sending all the pending data upto the specified timeout value. Something like:
linger_enabled = 1
linger_time = 10 #This is in seconds.
linger_struct = struct.pack('ii', linger_enabled, linger_time)
s.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, linger_struct)
I'm writing a multithreaded distributed networking algorithm.
I've one thread that listens to new connections. Every time a new connection is established a separate thread is started for listening to messages from that connection.
My problem is that the socket I open works perfectly in both directions inside the connection listener. After I pass the socket object for that connection to the message listener I can read data from the socket, but sending data through it doesn't reach the remote host.
Here's the essential snip from my code:
def connection_listener(port, start_e, terminate_e):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.settimeout(1)
s.bind(('', port))
s.listen(1)
while (not start_e.isSet()):
try:
conn, addr = s.accept()
msg_in = conn.recv(1024).split(":")
if (msg_in[1]=="hello"):
# If addr sends us a 'id:hello', we reply with a 'my_id:welcome'
conn.send(str(my_id)+":welcome")
t = Thread(target=message_listener, args=(conn, addr[0], terminate_e, ))
t.start()
except:
pass # timeout
def message_listener(conn, address, terminate_e):
while (not terminate_e.isSet()):
try:
msg_in = conn.recv(1024)
# Here I can receive everything that I send from the other end of conn,
# but conn.send("any data") doesn't reach the remote host
What I'd like to do is send acknowledgement-like messages from the message listener thread using the conn. Is this possible somehow or am I thinking and doing it wrong?
I sorted this out myself, so I'll share my answer.
I made the protocol exchange fixed size messages by padding with zeroes up to the desired length. I used a length of 32 bytes, which might be quite tiny from the hardware's point of view. Nevertheless it seems to work as supposed.
Pragmatically my solution looks like:
def send_everyone(message):
for i in range(len(peers)):
chunk = (str(my_id)+":"+message).rjust(32, '0')
peers[i].send(chunk)
And on the receiving side we want only 32 bytes at a time:
def message_listener(conn, address, terminate_e):
while (not terminate_e.isSet()):
try:
msg_in = conn.recv(32)
...