Develop Reference

Python Socket UDP WinError 10040

I'm trying to stream JSON data over UDP to another computer. For this i created this simple class. It does work but after a while i get the OS error:
OSError: [WinError 10040] A message sent on a datagram socket was larger than the internal message buffer or some other network limit, or the buffer used to receive a datagram into was smaller than the datagram itself
My packet size are around 128 bytes but is send them +- 20 time a second. The bigger the packet the faster i get this error. The also happends when on the recieving side nothing is connected.
What am i doing wrong?
import socket
import json
class JSON_UDP:
def __init__(self,send_ip='127.0.0.1', send_port=7002):
self.send_ip = send_ip
self.send_port = send_port
self.sender = socket.socket(socket.AF_INET,socket.SOCK_DGRAM) # UDP
def send(self,input,send_ip='127.0.0.1', send_port=7002):
self.send_ip = send_ip
self.send_port = send_port
self.input = input
data = json.dumps(self.input)
data = bytes(data,encoding="utf-8")
print(len(data))
self.sender.sendto(data, (self.send_ip, self.send_port))
Tried it set a buffer size but without succes
self.buff_size = 1024
self.sender.setsockopt(socket.SOL_SOCKET,socket.SO_RCVBUF,self.buff_size)
self.sender.bind(('0.0.0.0',0))

Recvall with while loop doesn't work between two devices in python

I have the following problem: I want a sever to send the contents of a textfile
when requested to do so. I have writen a server script which sends the contents to the client and the client script which receives all the contents with a revcall loop. The recvall works fine when
I run the server and client from the same device for testing.
But when I run the server from a different device in the same wifi network to receive the textfile contents from the server device, the recvall doesn't work and I only receive the first 1460 bytes of the text.
server script
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("", 5000))
server.listen(5)
def send_file(client):
read_string = open("textfile", "rb").read() #6 kilobyte large textfile
client.send(read_string)
while True:
client, data = server.accept()
connect_data = client.recv(1024)
if connect_data == b"send_string":
send_file(client)
else:
pass
client script
import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(("192.168.1.10", 5000))
connect_message = client.send(b"send_string")
receive_data = ""
while True: # the recvall loop
receive_data_part = client.recv(1024).decode()
receive_data += receive_data_part
if len(receive_data_part) < 1024:
break
print(receive_data)

recv(1024) means to receive at least 1 and at most 1024 bytes. If the connection has closed, you receive 0 bytes, and if something goes wrong, you get an exception.
TCP is a stream of bytes. It doesn't try to keep the bytes from any given send together for the recv. When you make the call, if the TCP endpoint has some data, you get that data.
In client, you assume that anything less than 1024 bytes must be the last bit of data. Not so. You can receive partial buffers at any time. Its a bit subtle on the server side, but you make the same mistake there by assuming that you'll receive exactly the command b"send_string" in a single call.
You need some sort of a protocol that tells receivers when they've gotten the right amount of data for an action. There are many ways to do this, so I can't really give you the answer. But this is why there are protocols out there like zeromq, xmlrpc, http, etc...

working with NMEA data sent to URL/IP

i have a bunch of devices that send NMEA sentences to a URL/ip.
that look like this
"$GPGGA,200130.0,3447.854659,N,11014.636735,W,1,11,0.8,41.4,M,-24.0,M,,*53"
i want to read this data in, parse it and upload the key parts to a database. i know how to parse it and upload it to the DB but i am at a complete loss on how to "read"/accept/get the data into a python program so that i can parse and upload.
my first thought was to point it at a Django page and then have Djanog parse it and upload to the database (data will be accessed from Django site) but its a NMEA sentence not a HTTP request so Django rejects it as "message Bad request syntax"
what is the best (python) way to read NMEA sentences sent to a url/IP?
thanks

I assume you have some hardware that has an ethernet connection, and it pipes out the NMEA string over its ethernet connection. this probably defaults to having some random 192.168.0.x ip address and spitting out data over port 12002 or something
you would typically create a socket to listen for this incomming data
server.py
import socket
host = "" #Localhost
port = 12002
PACKET_SIZE=1024 # how many characters to read at a time
sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
sock.bind((host,port))
sock.listen(5) # we should never have more than one client
def work_thread(client):
while True: #continuously read and handle data
data = client.recv(PACKET_SIZE)
if not data:
break # done with this client
processData(data)
while True:
client,addr = sock.accept() # your script will block here waiting for a connection
t = threading.Thread(target=work_thread,args=(client,))
t.start()
sometimes however you need to ping the device to get the data
client.py
import socket
host = "192.168.0.204" #Device IP
port = 12002
PACKET_SIZE=1024 # how many characters to read at a time
sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
sock.connect((host,port)) #connect to the device
while True: #continuously read and handle data
data = sock.recv(PACKET_SIZE)
processData(data)

Receive all of the data when using python socket

I am using socket to connect to a simple server to receive some data:
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
host = "X.X.X.X"
port = Y
s.connect((host,port))
data = s.recv(512)
print data
Not all of the expected received data is received; some of it is cut off. So any operations that I need to perform on the received data is thrown off.
I've tried using s.settimeout(1) and putting the socket in non-blocking mode. Strangely, it works just fine in ipython.

Even if 512 bytes are sent from the server to the client, if they're sent through multiple packets, and that one packet has TCP Push flag set, the kernel will stop buffering and hand over the data to userland, ie, your application.
You should loop on data = s.recv(512) and print data. I'd even remove the '512', which is useless.
while True:
if not data: break
data = s.recv()
print data

Python sends malformed UDP Packets

I am having trouble receiving UDP packets on an Android device, so I want to find out if I am sending them properly. Using Wireshark, everytime I try to send a UDP packet to a remote address, the following error message occurs:
232646 311.898009000 172.56.16.78 192.168.0.3 UDP 64 Source port: 31947 Destination port: 5001 [ETHERNET FRAME CHECK SEQUENCE INCORRECT]
Frame check sequence: 0xf5b6d06d [incorrect, should be 0xb0c869e3]
Does anyone know how to fix this? Would this be the cause of why I could not receive UDP packets on my Android device?
Server Code:
import http.server
import socket
import threading
import socketserver
class ThreadedUDPRequestHandler(socketserver.BaseRequestHandler):
def handle(self):
data = self.request[0].strip().decode("utf-8")
print("{} Recieved: ".format(self.client_address) + data)
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
response = data.upper()
sock.sendto(bytes(response, "utf-8"), self.client_address)
print("{} Sent: {}".format(self.client_address,response))
if __name__ == "__main__":
udpserver = ThreadedUDPServer((HOST,PORT+1), ThreadedUDPRequestHandler)
udp_thread = threading.Thread(target=udpserver.serve_forever)
udp_thread.daemon = True
udp_thread.start()
print("UDP serving at port", PORT+1)
while True:
pass
udpserver.shutdown()

It seems like you're sending packets using regular userspace sockets. In that case, there's very little chance that the packets are being sent malformed since the FCS is generated physically by the network interface card.
What you're probably seeing is an FCS error due to completely different reasons, which can be safely disregarded.
I'd look for other reasons for why the other device doesn't receive the packet, like firewalls or NAT. Start by using netcat or a similar tool for sending and receiving the UDP packets between the two machines.