I have a UDP socketserver program that I use to demonstrate how UDP works (code for the server and client are below). I run this on a server, then have the client.py program send a message and receive a reply. I am unfortunately running into an issue that seems to only occur on campus Wifi. On campus wifi, the client does not receive a response.
Troubleshooting with Wireshark shows the issue. For some reason the UDP server is responding with two UDP messages - one empty, and one containing the response message. These messages are recorded in Wireshark as coming in approximately 0.000002 seconds apart. On a wired network, the one with the response consistently comes first, and on Wifi, the empty message consistently comes first. Since the client is waiting for a single messages response, when the empty message returns, the client prints and exits, and the actual response is never seen.
I know I could write the client to listen for both messages and print out whichever one has the data, but I would rather try to figure out what's going on. Why is the socketserver responding with two messages in the first place, and how can I get it to only send one? OR at least to send the data first.
server.py:
import socketserver
class MyUDPRequestHandler(socketserver.DatagramRequestHandler):
def handle(self):
data = self.request[0].strip()
socket = self.request[1]
# just send back the same data, but lower-cased
socket.sendto(data.lower(), self.client_address)
if __name__ == "__main__":
with socketserver.UDPServer(("0.0.0.0", 9091), MyUDPRequestHandler) as server:
server.serve_forever()
client.py:
import socket
HOST, PORT = "localhost", 9091
message = "NOW I AM SHOUTING" # The UDP server will lowercase the message
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(bytes(message + "\n", "utf-8"), (HOST, PORT))
received = str(sock.recv(1024), "utf-8")
print("Sent: {}".format(message))
print("Received: {}".format(received))
I've repeated the problem and it's socketserver. Notice the definition of DatagramRequestHandler below:
class DatagramRequestHandler(BaseRequestHandler):
"""Define self.rfile and self.wfile for datagram sockets."""
def setup(self):
from io import BytesIO
self.packet, self.socket = self.request
self.rfile = BytesIO(self.packet)
self.wfile = BytesIO()
def finish(self):
self.socket.sendto(self.wfile.getvalue(), self.client_address)
The packet is put into a buffer as rfile and should be read from there, then written back to the wfile buffer. finish sends the packet. The handler shouldn't call sendto itself:
import socketserver
class MyUDPRequestHandler(socketserver.DatagramRequestHandler):
def handle(self):
data = self.rfile.read()
self.wfile.write(data.strip().lower())
if __name__ == "__main__":
with socketserver.UDPServer(("0.0.0.0", 9091), MyUDPRequestHandler) as server:
server.serve_forever()
But just using a simple socket as the server works fine too:
import socket
s = socket.socket(type=socket.SOCK_DGRAM)
s.bind(('', 9091))
while True:
data, client = s.recvfrom(2048)
s.sendto(data.strip().lower(), client)
Note that UDP packets are not guaranteed to be delivered or delivered in the same order, so the original code's issue with the two packets changing order isn't surprising.
Related
So in other words it's just stuck after 1st response from the browser.
I'm new to socket programming, but here is stupid question, do I need
to close client socket after the response from the server is sent? In
handleData after self.client.send(). Or what could be the reason
of this behaviour? Tried to use select here to make it asynchronous
but it didn't help. Same issue. After I get request from the client
with headers, I go to the next iteration of the loop, exactly to the
recv() and it is stuck. The client is not sending info anymore and
doesn't show the info that is sent from the server. In this example it
is "Info/Message was retrieved!".
P.S. DEFAULT_ENCODING = "utf-8"
DEFAULT_CLIENTS_AMOUNT = 5
SERVER_IP = "127.0.0.1"
SERVER_PORT = 8080
class Server:
def __init__(self, socket_family: int = socket.AF_INET, socket_type: int = socket.SOCK_STREAM, ip_address: str = SERVER_IP, port: int = SERVER_PORT):
self.addr: Tuple[str, int] = ip_address, port
self.serv_heart: socket.socket = socket.socket(socket_family, socket_type)
self.client: Optional[socket.socket] = None
def _launchSyncServer(self, clients: int = DEFAULT_CLIENTS_AMOUNT):
print("Starting server...")
self.serv_heart.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True)
self.serv_heart.bind(self.addr)
self.serv_heart.listen(clients)
print(f"Server started at {str(self.addr)}")
print(f"Visible: http://{self.addr[0]}:{self.addr[1]}")
def _acceptConnections(self):
client_socket, client_addr = self.serv_heart.accept()
print(f"Client connected from {str(client_addr)}")
self.client = client_socket
def _handleData(self):
while True:
rawData: str = self.client.recv(128).decode(DEFAULT_ENCODING)
if not rawData:
self.client.close()
break
self.client.send("Info/Message was retrieved!".encode(DEFAULT_ENCODING))
self.client.close()
def serverLoop(self, clients: int = DEFAULT_CLIENTS_AMOUNT):
self._launchSyncServer(clients=clients)
while True:
self._acceptConnections()
self._handleData()
it's just stuck after 1st response from the browser.
Your server doesn't implement HTTP nor any other browser protocol, so a browser is not an appropriate client for it; better use something like nc localhost 8080.
do I need to close client socket after the response from the server is sent?
That depends on how you want the communication to be conducted. If you want the client to only send one message, you can close the server's connection right after responding. If you want the client to be able to send multiple messages, the server as it is is prepared for that, and the client needs to indicate that it's finished by closing its socket or at least shutting down writing.
Note that in general, you can't be sure that a message is reveived in whole with a single recv call - it might be split and returned piece by piece through several calls of recv.
I am writing a UDP server application that serves as a back end to Teltonika FMB630 car mounted devices.
I already took care of the protocol specifics and decoding, the problem I am facing relates to the UDP socket used.
My UDP server has to send an acknowledgement to the client device upon receiving a message (that is the protocol), however, if I send those ACKs, the server socket stops receiving data after a while.
The server's UDP socket object is passed to an concurrent.futures.ThreadPoolExecutor that fires a function (send_ack) that sends the ACK, however this is not the issue because I tried calling send_ack in the main thread, after receiving data and the same issue occurs.
I suspect the problem is the remote device somehow breaks the connection or the ISP or MNO doesn't route the reply packet (this is a GPRS device) and then the socket.send() method that is used to send the acknowledge, somehow freezes other socket operations, specifically recvfrom_into called in the main thread loop.
I wrote two scripts to illustrate the situation:
udp_test_echo.py :
#!/usr/env/bin python
import socket
import concurrent.futures
def send_ack(sock, addr, ack):
print("Sending ACK to {}".format(addr))
sock.connect(addr)
print("connected to {}".format(addr))
sock.send(ack)
print("ACK sent to {}".format(addr))
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind(("127.0.0.1", 1337))
data = bytearray([0] * 10)
executor = concurrent.futures.ThreadPoolExecutor(max_workers=4)
while True:
print("listening")
nbytes, address = s.recvfrom_into(data)
print("Socket Data received {} bytes Address {}".format(nbytes, address))
print("Data received: ", data, " Echoing back to client")
executor.submit(send_ack, s, address, data[:nbytes])
udp_test_client.py:
#!/usr/env/bin python
import socket
import time
import random
def get_random_bytes():
return bytearray([random.randint(0,255) for b in range(10)])
ip = "127.0.0.1"
port = 1337
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect((ip, port))
while True:
stuff_to_send = get_random_bytes()
print("Sending stuff", stuff_to_send)
s.sendall(stuff_to_send)
print("reply: ", s.recvfrom(10))
time.sleep(0.1)
Running udp_test_echo.py in one terminal and udp_test_client.py in another, we see normal operation but if you Ctrl+C the test client and re run it, you will see that the server doesn't respond until it is restarted.
Is there a way to timeout a specific sending operation from a specific call to socket.send() method without affecting other calls ? (I want my socket.recvfrom_into call to block on the main thread)
If I settimeout on the entire socket object, I am going to have to deal with many exceptions while waiting for data in the main thread and I don't like to have to rely on exceptions for proper program operation.
The culprit was the socket.connect() call in send_ack, when being called on the server's socket object it causes the socket to no longer be bound and listen on the port specified in the start of the program.
Instead the send_ack function was changed to be:
def send_ack(sock, addr, ack):
print("Sending ACK to {}".format(addr))
sock.sendto(ack, addr)
print("ACK sent to {}".format(addr))
socket.sendto(data, address) uses the existing connection instead of starting a new one.
I just started programming Python.
My goal is to built a digital Picture Frame with three Screens. Therefore I use 3 Raspis, one for each Monitor.
For the communication of these Raspis I need to program a server and a Client.
For a first test I want to built a server which is able to send and receive messages to/from multiple clients.
So I started with a few socket tutorials an created the following program.
Server Class (TcpServer.py)
class TcpServer:
clients = []
serverIsRunning = 0
port = 0
def __init__(self, port):
self.port = port
self.serverIsRunning = 0
self.serverRunning = 0
def startServer (self):
print("start Server...")
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.server.bind(("", self.port))
self.server.listen(1)
self.serverRunning = 1
while self.serverRunning:
read, write, oob = select.select([self.server] + self.clients, [], [])
for sock in read:
if sock is self.server:
client, addr = self.server.accept()
self.clients.append(client)
print ("+++ Client ", addr[0], " verbunden")
else:
nachricht = sock.recv(1024)
ip = sock.getpeername()[0]
if nachricht:
print (ip, nachricht)
else:
print ("+++ Verbindung zu ", ip , " beendet")
sock.close()
self.clients.remove(sock)
for c in self.clients:
c.close()
self.clients.remove(c)
self.server.close()
def send(self, message):
message = message.encode()
self.server.send(message)
Client class (TcpClient.py)
import socket
class TcpClient:
def __init__(self, ip, port):
self.serverAdress = (ip, port)
self.connected = 0
self.connection = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.connection.connect(self.serverAdress)
print ("connectet to ", self.serverAdress)
def send(self, message):
message = message.encode()
self.connection.send(message)
Server:
import threading
import TcpServer
tcpServer = TcpServer.TcpServer(50000)
threadTcpServer = threading.Thread(target = tcpServer.startServer)
threadTcpServer.start()
while True:
tcpServer.send(input("Nachricht eingeben: "))
Client:
import threading
import TcpClient
tcpClient = TcpClient.TcpClient("192.168.178.49", 50000)
while True:
tcpClient.send(input("Nachricht eingeben: "))
I can send messages from the Client to the server, but when I want to send a Message from the server to the client it generates the following error:
BrokenPipeError: [Errno 32] Broken pipe
I assume it is because the server thread blocks the socket while waiting of a incoming message. But I have no idea how to handle this.
How can I program a server who can send and receive messages? Can you recommend a tutorial? I didn't found a tutorial who describes a solution for my problem.
Edit:
Now I tried to solve the problem with the socketserver library, but I still can't solve may problem.
here is my new code for the server:
import socketserver
import threading
import time
class MyTCPHandler(socketserver.BaseRequestHandler):
"""
The RequestHandler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
self.data = self.request.recv(1024).strip()
print("{} wrote:".format(self.client_address[0]))
print(self.data)
# just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
# Create the server, binding to localhost on port 9999
server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
threadTcpServer = threading.Thread(target = server.serve_forever)
threadTcpServer.start()
print("server started")
time.sleep(10)
print("sending Data")
server.request.sendall("Server is sending...")
it generates the error:
AttributeError: 'TCPServer' object has no attribute 'request'
My goal is to write a server with a thread who receives Data and still be able to send data from a other thread.
Is this even possible with only one socket?
You should use the provided socketserver rather than writing all the handling of sockets and select etc.
There are multiple problems with your code -
1 - The server is trying to write to the listening socket!! The client communication socket is the one that you get from the accept() call and that is the one you have to use for reading and writing.
2 - The client is sending the data and completing immediately, but it should really wait for getting a response. Otherwise, the python / OS will close the client socket as soon as the program completes and it will mostly be before the server gets a chance to respond.
I believe with the Handler code you are able to receive the data sent by the client on the server and are also able to send some data back from the Handler to the client? You must have understood that the server cannot send any data back unless there is a client connected to it?
Now, to send data to the client (or clients) from "another" thread, you will need a way to make the handler objects or the client sockets (available inside the Handler object as self.request) available to the "another" thread.
One way is to override the def __init__(self, request, client_address, server): method and save this object's reference in a global list. Remember to do the below as the last line of the overridden init -
# BaseRequestHandler __init__ must be the last statement as all request processing happens in this method
socketserver.BaseRequestHandler.__init__(self, request, client_address, server)
Once you have all the client handlers in the global list, you can easily write to all the clients from any thread as per your needs. You must read about synchronization (Locks) and understand that using same object / socket from multiple threads can create some logical / data issues with your application.
Another thing that you have to worry about and code for is cleaning up this global list whenever a client closes the connection.
I have a device that continually outputs data and I would like to send that data to a client on the same network as it is produced and I'm not finding a good solution. Here is what I'm trying.
Server:
import SocketServer
from subprocess import Popen,PIPE
class Handler(SocketServer.BaseRequestHandler):
def handle(self):
if not hasattr(self, 'Proc'):
self.Proc = Popen('r.sh', stdout=PIPE)
socket = self.request[1]
socket.sendto(self.Proc.stdout.readline(),self.client_address)
if __name__ == "__main__":
HOST, PORT = "192.168.1.1", 6001
server = SocketServer.UDPServer((HOST, PORT), Handler)
server.serve_forever()
Client:
import socket
data = " ".join(sys.argv[1:])
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(data + "\n", ("192.168.1.1", 6001))
try:
received = sock.recv(1024)
while True:
print "Sent: {}".format(data)
print "Received: {}".format(received)
sock.sendto('more' + "\n", ("192.168.1.1", 6001))
received = sock.recv(1024)
except:
print "No more messages"
arg[1] for the client is a program that outputs lines of data for several minutes that I need to process as it is created. The problem seems to be that every time the client sends another request, a new Handler object is created, so I loose Proc. How can I stream Proc.stdout?
Edit: The device is a Korebot2, so I have limited access to other python libraries due to space.
Using UDP you get a new "connection" each time you send a datagram, which is the reason you notice that a new object instance is created each time you send something. You're probably using the wrong kind of protocol here... UDP is used mostly for sending distinct "datagrams", or when a longer connection is not needed. TCP is also called a "streaming" protocol, and is often used for data that has no fixed end.
Also remember that UDP is not a reliable protocol, if used over a network it is almost guaranteed that you will loose packets.
I have a SocketServer.StreamRequestHandler server that calls self.rfile.readline() to read a request and then calls self.wfile.write(data) to send back some data:
class FileServerHandler(SocketServer.StreamRequestHandler):
def handle(self):
# self.rfile is a file-like oject created by the handler
data = self.rfile.readline()
if data == "msg":
self.wfile.write(someOtherData)
I want my client to be able to send the request and receive the "someOtherData" from the server:
# Create a socket (SOCK_STREAM means a TCP socket)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST, PORT))
sock.send("msg")
print sock.recv(1024)
sock.close()
But the client hangs when I try this. Where am I going wrong? Also is it necessary to know how much data the socket recv's or is there a way to just receive all the data the server writes?
As your server is doing a self.rfile.readline() it is constantly reading until it receives a newline ("\n") character. Thus your client needs to send sock.send("msg\n") for it to terminate the read command.
beside Jan answers I like to mention if you want to receive your exact message, you need to use strip to drop '\n' that you have put at the end of your string .