I have a client which is creating packets and sending packets to a destination in a network which has been created using mininet. Now I am writing a python program at the destination to count the number of packets which has arrived. Now I know for sure that the packets are arriving at the destination (used tcpdump to verify it)
How do I go about it?
I thought of using this -
s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_TCP)
print s.recvfrom(5001)
But this seems to be slow. Is there any other alternative?
You want socket.IPPROTO_UDP for UDP packets, but otherwise, that's basically what you must do. No matter what other things you try, it's going to have to do those things.
Oh, and you'll want to do a socket.bind(('',PORT)) to bind it to the port you want it to listen on.
Related
I'm having a strange problem with python udp sockets. When I send data over them, they open an udp port on 0.0.0.0 and I cannot find out why they do, what they do listen to and how to deactivate that behaviour. Our system administrators don't like ports to be listened on 0.0.0.0 (reasonably).
Minimum example:
import socket
fam = socket.AF_INET
family, _, _, _, addr = socket.getaddrinfo('localhost', 9999, fam, socket.SOCK_DGRAM)[0]
sock = socket.socket(family, socket.SOCK_DGRAM)
sock.sendto('foobar'.encode('ascii'), addr)
Right after the last method call, the python program listens to:
udp 0 0 0.0.0.0:41972 0.0.0.0:* 1000 308716 17777/python3
And this seems to stay until the python executable stops. So my question is, does anyone here have the same problem and how can I avoid it?
Thank you very much!
Which port do you expect your packets to come from?
Sockets have ports at both ends. Packets come from a port and go to a port. You didn't pick a port (with bind), so the operating system chose one for you.
It's not a problem, it's how sockets work.
0.0.0.0 means "any IP address" by the way.
just to elaborate on my comment, here's how I'd put the socket calls together:
import socket
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
sock.bind(('127.0.0.1', 0)) # optional
sock.connect(('localhost', 9999))
sock.send(b'foobar')
notes:
connecting a UDP socket should cause it to be bound to something more appropriate than 0.0.0.0, and hence why I put an optional comment
getaddrinfo doesn't help you much, hence I'm just passing names and letting Python resolve them internally
using a name for the connect call and dotted-quad notation for bind looks a little strange. I'd suggest using just one format for consistency, or just using connect and not using bind
getaddrinfo is really useful when you want to be able to transparently handle IPv6 along with IPv4 addresses, especially for hosts that resolve to multiple addresses. see the Happy Eyeballs algorithm for an example
So, I've found a workaround, even if I think it's not the cleanest way to do that, I couldn't find another one.
I used the method bind(addr: tuple) on the udp socket to make it listen to only a specific IP. In my case this looks like:
import socket
fam = socket.AF_INET
family, _, _, _, addr = socket.getaddrinfo('localhost', 9999, fam, socket.SOCK_DGRAM)[0]
sock = socket.socket(family, socket.SOCK_DGRAM)
sock.bind(('127.0.0.1', 0)) # This is the new line where I bind only to 127.0.0.1
sock.sendto('foobar'.encode('ascii'), addr)
Thanks to #user253751 for leading me in the right direction.
I currently have a raspberry pi that is set up as a wifi to ethernet bridge. The raspberry pi acts as an access point to the entire ethernet subnet that I have. The subnet and the network bridge work perfectly but when I try to get my python program on the raspberry pi to listen to requests on the ethernet interface/subnet it doesn't seem to do that. It is set to bind the socket to ('',10000) but it never receives any messages. However, it is able to send messages via sockets to the subnet just fine, just not receive. I think it is listening to the wifi interface rather than the ethernet one but I'm not sure how to specify which interface the socket is suppose to listen to.
here is my receiving code
receive_group = ('',10000)
receive = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
receive.bind(receive_group)
while(True):
data, address = receive.recv(65536)
print(data)
The bind part should be correct. The receive part is wrong because recv only return the (bytes) data, you should use recvfrom to also get the sender address. But this should work:
import socket
receive = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
receive.bind(('', 10000))
while True:
data, address = receive.recvfrom(64)
print(data, address)
I used this code for the send part:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.sendto(b'foo', (addr, 10000))
where addr is one of the (reachable) addresses of the receiver, and the receiver could successfully receive data from any of its interfaces.
To get something from socket.recv something must connect to this socket and send something. Are you sure that some program on the network is doing this?
For listening/sniffing to packets and network traffic, better use pyshark.
Turns out it wasn't anything with python. When I created the access point on the pi it created a firewall rule that blocked that port even though I never configured the firewall that way. Adding an exception to that port fixed my problem
An eye-tracking application I use utilizes UDP to send packets of data. I made a python socket on the same computer to listen and dump the data into a .txt file. I already have this much working.
A separate application also written in python (what the eye-tracked subject is seeing) is running on a separate computer. Because the eye-tracking application is continuous and sends unnecessary data, so far I've had to manually parse out the instances when the subject is looking at desired stimuli. I did this based on a manually synchronized start of both the stimuli and eye-tracking applications and then digging through the log file.
What I want to do is have the second computer act as a second UDP client, sending a packet of data to the socket on the eye-tracking computer everytime the subject is looking at stimuli (where a marker is inserted into the .txt file previously mentioned). Is it possible to have a socket listening to two IP addresses at one time?
Here's my socket script:
#GT Pocket client program
import datetime
import socket
now = datetime.datetime.now()
filename = 'C:\gazelog_' + now.strftime("%Y_%m_%d_%H_%M") + '.txt'
UDP_IP = '127.0.0.1' # The remote host (in this case our local computer)
UDP_PORT = 6666 # The same port as used by the GT server by default
sock = socket.socket(socket.AF_INET, #internet
socket.SOCK_DGRAM) #UDP
sock.bind( (UDP_IP, UDP_PORT) )
while True:
data, addr = sock.recvfrom( 1024) #I assume buffer size is 1024 bytes.
print "Received Message:", data
with open(filename, "a") as myfile:
myfile.write(str(data + "\n"))
sock.close()
myfile.close()
EDIT:
#abarnert I was able to bind to the host address on the Ethernet interface and send a message from computer B to computer A, but computer A was no long able to receive packets from itself. When I specified UDP_IP = '0.0.0.0' computer B was no longer able to send data across the Ethernet. When I specified UDP_IP = '' I received the `error: [Errno 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted
This have to do with the script I used on the Computer B to send the data:
import socket
UDP_IP = "169.254.35.231" # this was the host address I was able to send through.
UDP_PORT = 6666
MESSAGE = "Start"
print ("UDP target IP:"), UDP_IP
print ("UDP target port:"), UDP_PORT
print ("message:"), MESSAGE
sock = socket.socket(socket.AF_INET,
socket.SOCK_DGRAM)
sock.sendto(MESSAGE, (UDP_IP, UDP_PORT) )
I didn't know where (or if at all) I needed to specify INADDR_ANY, so I didn't. But I did try once where import socket.INADDR_ANY but got ImportError: No module named INADDR_ANY
Seems like a simple issue based on your response, so I'm not sure where I'm messing up.
EDIT2: I just reread your answer again and understand why socket.INADDR_ANY doesn't work. Please disregard that part of my previous edit
EDIT3: Okay so the reason that I wasn't picking up data when specifying the host IP was that the application I was collecting data from on Computer A was still specified to send to 127.0.0.1. So I figured it out. I am still curious why 0.0.0.0 didn't work though!
No. A socket can only be bound to a single address at a time.*
If there happens to be a single address that handles both things you want, you can use a single socket to listen to it. In this case, the INADDR_ANY host (0.0.0.0) may be exactly what you're looking for—that will handle any (IPv4) connections on all interfaces, both loopback and otherwise. And even if there is no pre-existing address that does what you want, you may be able to set one up via, e.g., an ipfilter-type interface.
But otherwise, you have to create two sockets. Which means you need to either multiplex with something like select, or create two threads.
In your case, you want to specify a host that can listen to both the local machine, and another machine on the same Ethernet network. You could get your host address on the Ethernet interface and bind that. (Your machine can talk to itself on any of its interfaces.) Usually, getting your address on "whatever interface is the default" works for this too—you'll see code that binds to socket.gethostname() in some places, like the Python Socket Programming HOWTO. But binding to INADDR_ANY is a lot simpler. Unless you want to make sure that machines on certain interfaces can't reach you (which is usually only a problem if you're, e.g., building a server intended to live on a firewall's DMZ), you'll usually want to use INADDR_ANY.
Finally, how do you bind to INADDR_ANY? The short answer is: just use UDP_IP = '', or UDP_IP = '0.0.0.0' if you want to be more explicit. Anyone who understands sockets, even if they don't know any Python, will understand what '0.0.0.0' means in server code.(You may wonder why Python doesn't have a constant for this in the socket module, especially when even lower-level languages like C do. The answer is that it does, but it's not really usable.**)
* Note that being bound to a single address doesn't mean you can only receive packets from a single address; it means you can receive packets from all networks where that single address is reachable. For example, if your machine has a LAN connection, where your address is 10.0.0.100, and a WAN connection, where your address is 8.9.10.11, if you bind 10.0.0.100, you can receive packets from other LAN clients like 10.0.0.201 and 10.0.0.202. But you can't receive packets from WAN clients like 9.10.11.12 as 10.0.0.100.
** In the low-level sockets API, dotted-string addresses like '0.0.0.0' are converted to 32-bit integers like 0. Python sometimes represents those integers as ints, and sometimes as 4-byte buffers like b'\0\0\0\0'. Depending on your platform and version, the socket.INADDR_ANY constant can be either 0 or b'\0\0\0\0'. The bind method will not take 0, and may not take b'\0\0\0\0'. And you can't convert to '0.0.0.0' without first checking which form you have, then calling the right functions on it. This is ugly. That's why it's easier to just use '0.0.0.0'.
I believe you can bind a raw socket to an entire interface, but you appear to be using two different interfaces.
It's probably best to use two sockets with select().
Is there a way python can distinguish between packets being sent ? e.g.
python receives data
it process data
clients sends first packet
client sends second packet
python receives data, can i receive the first packet rather then all info in the buffer
I know i can set it up up so it sends data i confirm and the client wont send more data it i have confirmed that have a processed the last piece but i'd rather not
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(("", 2000))
sock.listen(5)
all the relevant socket data
There are basically two approaches:
At the start of each packet, send an integer specifying how long that packet will be. When you receive data, read the integer first, then read that many more bytes as the first packet.
Send some sort special marker between packets. This only works if you can guarantee that the marker cannot occur within a packet.
As S. Lott points out, you could instead use UDP (which is packet-based) instead of TCP (which is stream-based), but then you give up the other features that TCP provides (retransmission of dropped packets, sequential packets, and congestion control). It's not too hard to write your own code for retransmission, but congestion control is difficult to get right.
Is there a way python can distinguish between packets being sent ?
Yes. Use UDP instead of TCP.
Netstring is a simple serialization
format used to send data packets. Each
data packet is of the form
'length:data'.
http://en.wikipedia.org/wiki/Netstring
Python networking frameworks like
twisted has direct support for
netstring.
I have some code that listens for "announcements" via UDP multicast. I can get the IP address of the sender, but what I really need is the MAC address of the sender (since the IP address can and will change).
Is there an easy way to do this in Python?
A code snippet is included for reference, but likely unnecessary.
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
# Allow multiple sockets to use the same PORT number
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
# Bind to the port that we know will receive multicast data
sock.bind((self.interface, MCAST_PORT))
# Tell API we are a multicast socket
sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 255)
# Tell API we want to add ourselves to a multicast group
# The address for the multicast group is the third param
status = sock.setsockopt(socket.IPPROTO_IP,
socket.IP_ADD_MEMBERSHIP,
socket.inet_aton(MCAST_ADDR) + socket.inet_aton(self.interface));
data, addr = sock.recvfrom(1024)
...
You cannot, in general, get the mac address. You might succeed using ARP on a LAN, but across the Internet it's not possible.
Consider the case where the packet you receive has the IP address of the sender's NATting router. The packet may have traversed any number of intermediate machines along the way, each of which have mac addresses, too. Whose responsibility should it be to support the kind of lookup you're after? For all the machines along the way, the sender's mac address is completely useless, so why bother supporting that kind of lookup?
And, btw, changing the mac address is trivial on many network cards, so using it as some kind of unique ID is not a wise idea.
To do this, you need to capture the raw Ethernet frame, not just the UDP packet.
import socket
ETH_P_ALL=3
sock = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.htons(ETH_P_ALL))
sock.bind((interface_name, 0))
data = sock.recv(2000)
dmac = data[:6]
smac = data[6:12]
udp_offset = 14
ethertype = data[12:14]
if ethertype == [0x81, 0x00]: # 802.1Q VLAN-tagged
udp_offset += 4
udp_pkt = data[udp_offset:]
Some notes:
An Ethertype value of 0x88a8 would indicate 802.1ad QinQ or "stacked VLAN" and would need to be handled appropriately to correctly find the UDP data.
On most Linux systems you need to be root (or have the CAP_NET_RAW capability) to do this. Not sure what you need on Windows, but assume something similar.
In practice this will be something of a firehose as it will receive all UDP packets. You can either parse the UDP header yourself to narrow it down to the ones you are interested in, or (on Linux / BSD etc) investigate Berkeley Packet Filter to get the kernel to do it for you. The latter is much more CPU efficient but is rather a pain to implement.
Answers suggesting ARP or similar might do what you want but they might not. They will tell you what association your OS has between IP addresses and MACs; this won't help for layer 2 protocols, multicast, broadcast etc or if something is lying in response to ARP requests.
The protocol you need is ARP. Check this question/answer for details
I'm not sure that it is possible to get the sender's MAC address since the MAC address is a link level address and not a network level address like IP. The MAC address will change at each hop in the network as the packet containing the UDP message is routed from the sender to the receiver.
I don't know how to do it in python but it is possible to get MAC address. For example by using tcpdump I put all packets into file:
sudo tcpdump -i enp0s31f6 -w file_name port 6665
then in python read it with:
packetlist = rdpcap("./file_name")
for pkt in packetlist:
print pkt.src, pkt.load
you can see the mac address
edit:
I found one way to do this:
sniff all packages with scapy with the help of function sniff, then filter the packages to get only what you need. There you can use mac address
for example from my project:
sniff(prn=self._pkt_callback, store=0)
def _pkt_callback(self, pkt):
if not self.sniffer_on:
return
if Ether not in pkt or pkt[Ether].type != 0x800:
return
if pkt[IP].proto != 17: # 17 means UDP package
return
if pkt[UDP].dport != 6665:
return
print pkt.src, pkt.load #src is mac address