Yes, I did already try to find information on this.
The Python socket documentation has this list of what I believe are protocols:
SO_*
socket.SOMAXCONN
MSG_*
SOL_*
IPPROTO_*
IPPORT_*
INADDR_*
IP_*
IPV6_*
EAI_*
AI_*
NI_*
TCP_*
What exactly do they do? Let's say I used
s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP)
What does this do? I understand it's a raw socket, but does the IPPROTO_IP mean I have to construct everything? (i.e. the IP header down to the TCP to the data?)
The Python documentation says I can find information on the above in the Unix documentation on sockets, but I couldn't find the document. Anyone know where it is?
There are a lot of Linux manual pages describing socket:
SOCKET
IP
TCP
UDP
UNIX
In general, we use these arguments for socket:
Address family: AF_INET for internet domain address family, AF_UNIX for UNIX domain address family.
Socket type: SOCK_STREAM for TCP, SOCK_DGRAM for UDP. Of course you can use SOCK_RAW for directly access IP protocol.
Protocol: when using TCP or UDP, leave it to 0 is just fine; when using RAW, you can specify protocol to 0, IPPROTO_TCP for TCP sockets, IPPROTO_UDP for UDP sockets.
And, SO_ means "socket option", SOL_ means "socket option level", which are used to set socket options through setsockopt (also mentioned in SOCKET).
In fact, you can find more pages at the bottom of these pages in the SEE ALSO section. Note that the page of 2 or 3 is a concrete system call or library function, pages of 7 is what you need.
Related
Following is a python socket snippet:
import socket
socket.socket(socket.AF_INET, socket.SOCK_STREAM)
My question is: does the line state whethet socket connection will be transported via TCP/IP? So far I was programming TCP connections only, using above line, but probably I was unaware of the fact. Am I able to program UDP connections using python sockets? How can I differentiate the transport layer?
The question isn't strictly connected to python, explanations are welcome as well in c++ or anything else.
The second argument determines the socket type; socket.SOCK_DGRAM is UDP, socket.SOCK_STREAM is a TCP socket. This all provided you are using a AF_INET or AF_INET6 socket family.
Before you continue, perhaps you wanted to go and read the Python socket programming HOWTO, as well as other socket programming tutorials. The difference between UDP and TCP sockets is rather big, but the differences translate across programming languages.
Some information on sockets on the Python Wiki:
UDP Communication
TCP Communication
The general syntax for creating a socket is:
socket(socket_family, socket_type, protocol=0)
We can use either AF_INET (for IPv4) or AF_INET6 (IPv6) as the first argument i.,e for socket_family.
The socket_type is the argument that determines whether the socket to be created is TCP or UDP. For TCP sockets it will be SOCK_STREAM and for UDP it will be SOCK_DGRAM (DGRAM - datagram). Finally, we can leave out the protocol argument which sets it to the default value of 0.
For TCP sockets you should have used bind(), listen() and accept() methods for server sockets and connect() or connect_ex() for client sockets. Whereas for UDP sockets you won't need listen(), accept() and connect() methods (as TCP sockets are connection-oriented sockets while UDP sockets are connection less sockets).
There are specific methods available for UDP to send and receive packets recvfrom() and sendto() respectively while recv() and send() are for TCP. Refer to this documentation for socket for more information on respective methods for TCP and UDP. Also, Core Python Applications Programming by Wesley Chun is a better book for some pretty basics on socket programming.
The main difference is that TCP sockets are connection-based. You can't send or receive anything until you are connected to another TCP socket on the remote machine. Once connected, a TCP socket can only send and receive to/from the remote machine. This means that you'll need one TCP socket for each client in your application.
UDP is not connection-based, you can send and receive to/from anyone at any time with the same socket.
IP multicast gives you the ability to send a single packet, which is picked up by multiple interfaces if they are subscribed to that multicast. If I understand it correctly.
Now if I want to use UDP, in combination with IP multicast, I am obligated to assign a port to listen to. But now I understand that I only receive UDP packets on a multicast specifically sent to that port. But I would like to intercept all UDP packets send to a certain multicast IP address, regardless of the port and receive them at my single socket.
Is something like this possible?
Preferably accompanied by a python example, if this is possible.
This isn't possible using the BSD socket API (which is roughly the API Python exposes in its socket module) - except by creating 2 ** 16 - 1 sockets and using them to bind to all ports.
It's possible using lower-level interfaces such as the TUN/TAP system offered by Linux.
This is a method of DatagramProtocol class in Twisted. As I understand UDP protocol doesn't guarantee that someone is listening on the given port even using ConnectedDatagramProtocol.
Can someone explain to me, when this method is called and how I suppose to check if there is someone listening to my transmission using UDP?
If the datagram socket is connected, it can receive ICMP Port Unreachable messages via the Sockets API, which presumably maps into calling this method. Note that I am not speaking of the TCP connect operation here, but the Sockets connect() method, which can be called on a UDP socket, and which presumably maps into some method in the API you are using.
Why does python specify IPv4 or IPv6 with SOCK_STREAM or SOCK_DGRAM when it also accepts protocol number arguments in the instantiation of the socket object (nrs 4 and 41 ,see list of protocol numbers)? (as proto in socket.socket([family[, type[, proto]]])) Doesn't that make the family argument useless?
proto in this context refers to the Layer 4 (L4) protocol to use. Most likely this will be IPPROTO_TCP (TCP) or IPPROTO_UDP (UDP). In practice it's unlikely you will need to pass these parameters explicitly very often, as for IP generally SOCK_STREAM implies TCP and SOCK_DGRAM implies UDP.
It could come in handy for instance, however, if say you wanted to implement ICMP support in your program using raw sockets. Then you'd pass SOCK_RAW and IPPROTO_ICMP. Also, keep in mind this is based on the Berkeley sockets API which is designed to handle much more than just IP networking (take a look at all the other AF_* from the socket module to get an idea of what it supports).
In contrast, IPv4/IPv6 is a Layer 3 (L3) protocol and works independently from TCP/UDP.
EDIT:
As to why you see IPv4/IPv6 encapsulation in that list, it's to support protocols like IP in IP or 6in4. It's not likely you will deal directly with these mechanisms in your program.
Nope. Number 4 and 41 in protocol list you linked to clearly state "IPv4 encapsulation" and "IPv6 encapsulation" You would use IPv4 with IPV6 encapsulation for example to generate packet which are then de-capsulated (unless that's not actually a word) upstream e.g. at an IPv6 over IPv4 tunnel
How to reassemble TCP packets in Python? Is there any existing tools for this?
Thanks! :-)
To do perform TCP reassembly you'll need to use something like pynids http://jon.oberheide.org/pynids/.
You can also build your own using pylibpcap, dpkt or scapy.
TCP reassembly is very tricky with a LOT of edge cases. I wouldn't recommend doing it yourself if you need a robust solution.
Yes... the TCP protocol guarantees that the application layer will only see packets assembled and in order. Now if you are talking about building some low level interface parsing the IP packet itself, you can take a stab at it with RAW sockets which should give you access to IP header information. Here's an example:
import socket
# the public network interface
HOST = socket.gethostbyname(socket.gethostname())
# create a raw socket and bind it to the public interface
s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP)
s.bind((HOST, 0))
# Include IP headers
s.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)
# receive all packages
s.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)
# receive a package
print s.recvfrom(65565)
# disabled promiscuous mode
s.ioctl(socket.SIO_RCVALL, socket.RCVALL_OFF)
lifted shamelessly from the python socket module documentation:
http://docs.python.org/library/socket.html