Develop Reference

Python UDP receiver raises timeout error with high package rate while wireshark receives data

I'm trying to make a simple UDP receiver, using python, to read a continuous data stream from a data aquisition device. This device can be configured to send data using UDP protocol to a certain IP and port. I configured it to send two data streams on port 4000 and 4001 both to ip: 192.168.0.10 (see wireshark screenshot below). The stream to port 4001 has a higher package rate. My laptop ip for the ethernet interface used is set statically at this ip in order to receive the data. I can receive the slow stream at port 4000 which works as expected. When I use the same code and set the port to 4001 I expected to read this data stream, instead I get a timeout exception.
I expected the biggest problem with high package rate udp would be loosing data because the buffer overflows which is not a big problem since loosing some packets is acceptable for my case. However if I get a timeout exception during the socket.recv(1024) call I thought that means there is no data in the buffer? Since wireshark does receive data I don't understand what is happening.
Here is a minimal example of what I am trying to do.
import socket
for i in range(3): # try multiple times
try:
socket_i = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
socket_i.settimeout(5)
socket_i.bind(('192.168.0.10', 4001)) #4000 works, 4001 does not
raw = socket_i.recv(1024)
socket_i.close()
print(raw)
except Exception as ex: # Exception is raised when using 4001: timed out
socket_i.close()
print('failed')
print(ex)
Wireshark printscreen; only 1 packet from port 4000 is shown, since this stream is slow.
I read similar questions:
Receive an high rate of UDP packets with python
Here it was concluded that python was the bottle neck. Also they received data, I don't receive anything, instead the socket.recv() times out. I don't think my package rate is that high, anyone thinks otherwise?
Receive an high rate of UDP packets with python
Here most of the data was received and only after some time the algorithm 'hangs on socket.recv()'. I tried increasing the buffer as suggested, using:
socket_i.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 1024*1024*1024)
Various values for buffer size are tried without luck. Setting large time out did not help.
Finally I tried using 'socketserver' package (the example udp server script):
import socketserver
class MyUDPHandler(socketserver.BaseRequestHandler):
"""
This class works similar to the TCP handler class, except that
self.request consists of a pair of data and client socket, and since
there is no connection the client address must be given explicitly
when sending data back via sendto().
"""
def handle(self):
data = self.request[0].strip()
socket = self.request[1]
print("{} wrote:".format(self.client_address[0]))
print(data)
# socket.sendto(data.upper(), self.client_address) #don't want to send stuff back
if __name__ == "__main__":
with socketserver.UDPServer(('192.168.0.10', 4001), MyUDPHandler) as server:
server.serve_forever()
Using port 4000 this gives the expected behaviour, for port 4001 the code hangs without doing anything (probably large timeout by default?). I guess 'socketserver' package runs ontop of'socket' package?
I hope someone can help!
Update:
I fixed my problem by making the udp receiver using pyshark. This is a bit hacky and I'm not satisfied with the method however for now it works. I figured if wireshark receives data pyshark might work as well. Any suggestions using 'socket' package are welcome!
code for hacky udp receiver
import pyshark
ip = '192.168.0.10'
port = 4001
capture = pyshark.LiveCapture(bpf_filter=f'dst host {ip} and dst port {port}', use_json=True, include_raw=True)
for pkt in capture.sniff_continuously():
raw = bytearray.fromhex(pkt.udp.payload_raw[0])
print(raw)

How to use scapy srp or sendp function

I want to send a packet with scapy to another interface.
I have the wlan2 interface and i want my packet (that i generate) to be send there.
I've tried using send with iface but it has no effect.
I also tried using srp and just sendp but i am getting this strange result:
answer = srp(pkt[Ether]/ip/new_pkt/html1, iface="wlan2")
pkt[Ether] is a valid pkt that comes from the wlan2 interface and i can sniff it.
i am trying to generate an http response packet using its Ethernet layer.
But my response is always going to another interface and i think this is the problem.
Wireshark Ethernet II
The packets are grey...
The question is how to fix this? how do i send a legit packet to the wlan2 interface.

Scapy, RST packets and iptables

I wrote a script that basically does the following: Opens a socket, connects and then an object that parses this socket so I can play around with scapy. It turns out that the packets are arriving to my destination but the responses from the destination host back to me are lost "somewhere" "somehow". I can see from the dump of the destination host that it clearly replies with some RST packets back, however from my "attacking" host I cant see these RST packets. Is this related to my TCP/IP stack and the generation of these RST packets locally or what? Do I have to set up an iptables rule for solving this problem?
I have the following snippet from my script
for i in range (0,int(args.packets)):
time.sleep(5)
ans,unans=sr(IP(dst=args.dst)/TCP(dport=int(args.port))/fuzz(Raw()), timeout=0.5, inter=0, iface=args.interface)
unans.nsummary()
ans.nsummary()
Another issue is that supposedly this is using the fuzz() method from scapy and it fuzzes the actual payload of the packet (not the IP / TCP header ). However when I am listening with netcat for example nothing is getting printed and I can also see from the dump that the packets are getting interpreted to either FTP-DATA or something other...
Any help?

Receiving Broadcast Packets in Python

I have the following code which sends a udp packet that is broadcasted in the subnet.
from socket import *
s=socket(AF_INET, SOCK_DGRAM)
s.setsockopt(SOL_SOCKET, SO_BROADCAST, 1)
s.sendto('this is testing',('255.255.255.255',12345))
The following code is for receiving the broadcast packet.
from socket import *
s=socket(AF_INET, SOCK_DGRAM)
s.bind(('172.30.102.141',12345))
m=s.recvfrom(1024)
print m[0]
The problem is that its not receiving any broadcast packet. However, it is successfully receiving normal udp packets sent to that port.
My machine was obviously receiving the broadcast packet, which I tested using netcat.
$ netcat -lu -p 12345
this is testing^C
So, where exactly is the problem?

Try binding to the default address:
s.bind(('',12345))

I believe the solution outlined in the accepted answer solves the issue, but not in exactly the right way. You shouldn't use the normal interface IP, but the broadcast IP which is used to send the message. For example if ifconfig is:inet addr:10.0.2.2 Bcast:10.0.2.255 Mask:255.255.255.0
then the server should use
s.bind(('10.0.2.255',12345)), not 10.0.2.2 (in OP's case he should use 255.255.255.255).
The reason the accepted answer works is because ' ' tells the server to accept packets from all addresses, while specifying the IP address, filters it.' ' is the hammer, specifying the correct broadcast address is the scalpel. And in many cases, though possibly not OP's, it is important that a server listen only the specified IP address (e.g. you want to accept requests only from a private network - the above code would accept requests from any external network too), for security purposes if nothing else.

s=socket(AF_INET, SOCK_DGRAM)
s.bind(('',1234))
while(1):
m=s.recvfrom(4096)
print 'len(m)='+str(len(m))
print 'len(m[0])='+str(len(m[0]))
print m[0]
print 'len(m[1])='+str(len(m[1]))
print m[1]

Complete HTTP GET with scapy

I am trying to use scapy to run a complete HTTP session. That is to say, I want to manually perform the three way handshake, GET request, acknowledgements as necessary to receive the HTML file, and terminating the connection. Using [1] I have completed the three way handshake and the GET request, but I can't seem to capture the raw HTML packets sent from the server, and I obviously can't send an ack packet back for more. Any ideas?
Additionally, I'd ultimately like to be able to parse the raw packet for HTML. If anyone knows how to do that from a scapy packet I'd appreciate it.
[1] http://www.thice.nl/creating-ack-get-packets-with-scapy/

Gimbi,
I am at work and can only parse and not initiate connections in scapy right now. So i will address your second request. We are looking at something like I have provided here. The layer that contains the html as well as the http requests is (Raw).load if the packet contains html or an http request I would first test to see if the layer exists (haslayer) and then if the packet is a 'http packet" here is just check for 80 in the IF statement, however you could potentially just use the port in the sniff netfilter. I have included the option to sniff directly off the wire or pull in a pcap here in this snippet. (adjust ports etc if you are using non standards)
#!/usr/bin/python -tt
from scapy import *
import sys
def parse(pkt):
if pkt.haslayer(TCP) and pkt.getlayer(TCP).dport == 80 and pkt.haslayer(Raw):
print pkt.getlayer(Raw).load
if '-l' in sys.argv:
sniff(prn=parse)
else:
pkts = rdpcap(sys.argv[1])
for pkt in pkts:
parse(pkt)
Of course use this as a start you can adjust line 8 to pick up not just dport but also sport for example. Let me know if this helps at all and good luck!
P.S. change the following
from scapy import *
to
from scapy.all import *
depending on your version..