Develop Reference

How to bind ports for TCP clients when using raw packet sending TCP packets in Python? [duplicate]

Ok, I realize this situation is somewhat unusual, but I need to establish a TCP connection (the 3-way handshake) using only raw sockets (in C, in linux) -- i.e. I need to construct the IP headers and TCP headers myself. I'm writing a server (so I have to first respond to the incoming SYN packet), and for whatever reason I can't seem to get it right. Yes, I realize that a SOCK_STREAM will handle this for me, but for reasons I don't want to go into that isn't an option.
The tutorials I've found online on using raw sockets all describe how to build a SYN flooder, but this is somewhat easier than actually establishing a TCP connection, since you don't have to construct a response based on the original packet. I've gotten the SYN flooder examples working, and I can read the incoming SYN packet just fine from the raw socket, but I'm still having trouble creating a valid SYN/ACK response to an incoming SYN from the client.
So, does anyone know a good tutorial on using raw sockets that goes beyond creating a SYN flooder, or does anyone have some code that could do this (using SOCK_RAW, and not SOCK_STREAM)? I would be very grateful.
MarkR is absolutely right -- the problem is that the kernel is sending reset packets in response to the initial packet because it thinks the port is closed. The kernel is beating me to the response and the connection dies. I was using tcpdump to monitor the connection already -- I should have been more observant and noticed that there were TWO replies one of which was a reset that was screwing things up, as well as the response my program created. D'OH!
The solution that seems to work best is to use an iptables rule, as suggested by MarkR, to block the outbound packets. However, there's an easier way to do it than using the mark option, as suggested. I just match whether the reset TCP flag is set. During the course of a normal connection this is unlikely to be needed, and it doesn't really matter to my application if I block all outbound reset packets from the port being used. This effectively blocks the kernel's unwanted response, but not my own packets. If the port my program is listening on is 9999 then the iptables rule looks like this:
iptables -t filter -I OUTPUT -p tcp --sport 9999 --tcp-flags RST RST -j DROP

You want to implement part of a TCP stack in userspace... this is ok, some other apps do this.
One problem you will come across is that the kernel will be sending out (generally negative, unhelpful) replies to incoming packets. This is going to screw up any communication you attempt to initiate.
One way to avoid this is to use an IP address and interface that the kernel does not have its own IP stack using- which is fine but you will need to deal with link-layer stuff (specifically, arp) yourself. That would require a socket lower than IPPROTO_IP, SOCK_RAW - you need a packet socket (I think).
It may also be possible to block the kernel's responses using an iptables rule- but I rather suspect that the rules will apply to your own packets as well somehow, unless you can manage to get them treated differently (perhaps applying a netfilter "mark" to your own packets?)
Read the man pages
socket(7)
ip(7)
packet(7)
Which explain about various options and ioctls which apply to types of sockets.
Of course you'll need a tool like Wireshark to inspect what's going on. You will need several machines to test this, I recommend using vmware (or similar) to reduce the amount of hardware required.
Sorry I can't recommend a specific tutorial.
Good luck.

I realise that this is an old thread, but here's a tutorial that goes beyond the normal SYN flooders: http://www.enderunix.org/docs/en/rawipspoof/
Hope it might be of help to someone.

I can't help you out on any tutorials.
But I can give you some advice on the tools that you could use to assist in debugging.
First off, as bmdhacks has suggested, get yourself a copy of wireshark (or tcpdump - but wireshark is easier to use). Capture a good handshake. Make sure that you save this.
Capture one of your handshakes that fails. Wireshark has quite good packet parsing and error checking, so if there's a straightforward error it will probably tell you.
Next, get yourself a copy of tcpreplay. This should also include a tool called "tcprewrite".
tcprewrite will allow you to split your previously saved capture files into two - one for each side of the handshake.
You can then use tcpreplay to play back one side of the handshake so you have a consistent set of packets to play with.
Then you use wireshark (again) to check your responses.

I don't have a tutorial, but I recently used Wireshark to good effect to debug some raw sockets programming I was doing. If you capture the packets you're sending, wireshark will do a good job of showing you if they're malformed or not. It's useful for comparing to a normal connection too.

There are structures for IP and TCP headers declared in netinet/ip.h & netinet/tcp.h respectively. You may want to look at the other headers in this directory for extra macros & stuff that may be of use.
You send a packet with the SYN flag set and a random sequence number (x). You should receive a SYN+ACK from the other side. This packet will have an acknowledgement number (y) that indicates the next sequence number the other side is expecting to receive as well as another sequence number (z). You send back an ACK packet that has sequence number x+1 and ack number z+1 to complete the connection.
You also need to make sure you calculate appropriate TCP/IP checksums & fill out the remainder of the header for the packets you send. Also, don't forget about things like host & network byte order.
TCP is defined in RFC 793, available here: http://www.faqs.org/rfcs/rfc793.html

Depending on what you're trying to do it may be easier to get existing software to handle the TCP handshaking for you.
One open source IP stack is lwIP (http://savannah.nongnu.org/projects/lwip/) which provides a full tcp/ip stack. It is very possible to get it running in user mode using either SOCK_RAW or pcap.

if you are using raw sockets, if you send using different source mac address to the actual one, linux will ignore the response packet and not send an rst.

Sending raw IP Traffic with Python: Detect MTU

How can I implement a manual MTU discovery with Python?
From: https://networkengineering.stackexchange.com/a/28988/23983
Send a ping to a target, in my example, I'll use Google's DNS server (8.8.8.8). Set your DF bit in your ping to on, to prevent your ping from being fragmented. Set your packet size to some large number, or the standard MTU of 1500. Note that some ping implementations set the size of just the payload, which means you have to account for the 8 byte ICMP header, and 20 byte IP header.

I'd like to add that you could to this with raw sockets as well.
scapy is a nice abstraction layer that does much of the magic for you, but in all fairness if you're going to go low you could go all the way for the learning experience. (Note that raw sockets require elevated permissions in most modern OS'es, and as you go deeper implementations may vary from Windows and Linux.)
import socket
s = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0003))
This will give you a raw packet based socket that literally delivers all the frames for you. It varies a bit between Windows and Linux but I'll stick to Linux in this answer. Also note that all outgoing packets might not get picked up by this socket, if you need that functionality (sniffing stuff) consider searching for promiscuous mode related hits.
All you need to do now is treat each packet as the segments they come or go in, for instance - unpacking a ethernet and IP frame would look something like this:
frame, meta = s.recvfrom(65565)
print(frame, meta)
ethernet = frame[0:14]
ethernet_segments = struct.unpack("!6s6s2s", ethernet)
mac_source, mac_dest = (binascii.hexlify(mac) for mac in ethernet_segments[:2])
ip = frame[14:34]
ip_segments = struct.unpack("!12s4s4s", ip)
ip_source, ip_dest = (socket.inet_ntoa(section) for section in ip_segments[1:3])
print('MAC Source:', b':'.join(mac_source[i:i+2] for i in range(0, len(mac_source), 2)))
print('MAC Dest:', b':'.join(mac_dest[i:i+2] for i in range(0, len(mac_dest), 2)))
print('IP Source:', ip_source)
print('IP Dest:', ip_dest)
Payloading would be "easy", considering you're building the packets yourself.
All be it not the most conventional of ways or the initially fastest way, but you'd be able to implement whatever you wanted.
Sending is just as easy, use struct and have a look at the many ICMP examples out there, including those with checksum calculations:
https://www.g-loaded.eu/2009/10/30/python-ping/
https://gist.github.com/pklaus/856268
Regarding MTU, this is logic you'd have to implement yourself because there's no pre-built library that does this that I'm aware of.
But this is my contribution to sending raw IP traffic with Python.

Sending anything raw in python, you are looking at using at using scapy.
For info on how to send ping messages:
http://www.secdev.org/projects/scapy/doc/usage.html#icmp-ping
To set the DF Flag:
IP(flags='DF')
For how to adjust the particular size so that you can simulate fragmentation:
Adding payload in packet (scapy)
Putting it all together:
data = "x" * 1473
ans,unans = sr(IP(dst="<target_ip>", flags='DF' )/ICMP() / Raw(load=data))
If you aren't actually that interested in creating these things raw, this question is a dup of: How to find mtu value of network through code(in python)?

Scapy waiting for multiple packets

In order to perform a HTTP GET, I need to send a packet (the GET / HTTP/1.0\n\n) and wait for 3 packets:
The ACK of my GET
The GET answer: HTTP/1.0 200 OK
and the FIN ACK of the transmission
I found 2 ways:
=> use sr() with multi option
=> use sniff just after sending my GET request
For sr() function, the problem is to stop the sniffing, the only option is to set a timeout, but my script will test many different sites, so many different of time's answer, it could be hard to choose a static timeout value where I'm sure that no site exceed it anytime.
For sniff, there is no the same problem because I can set "count" argument to take only the 3 packets. But it's hard to make a filter good enough to be sure the 3 packets recorded are the 3 that I want (and no ARP, DNS or anything else).
But the main problem is sometimes the fist answer packet come before "sniff" is launched (between send(GET_PACKET) and answers=sniff(...)). In this case, I lost some information and all my post-treatment is corrupted.
The perfect way would be to use sr() function with "count=3" option to only get 3 packets, but that option doesn't exist with sr().
Anynone have an idea?
Thanks a lot
Sorry for my language, I'm French

Use Sniff and set the filter to TCP port 80
and for delay problem you can use a thread, first start your sniffer in thread then send the packets :
def sniffer():
packets=sniff(filter="tcp port 80" , count=5)
wrcap("test.cap" , packets) #save packets in .cap file
t = threading.Thread(target=sniffer)
t.start()
But you can use a better way that explained HERE. send your packets manually.

This is more of a hint than an answer, but the problem might be that you want to inspect transport layer packets for a application layer request. You could split up your HTTP GET down to transport layer by sending SYN, waiting for and answer and then send ACK, GET. Here is a link describing what you might want.

Can the python client tcp socket be used for multiple sends?

I tried using a socket for 2 sends. The first one succeeds and the next one does not.
From the http://docs.python.org/howto/sockets.html
it would appear that multiple sends should be allowed. For Better or worse, I don't really need to read from the socket.
I have used twisted, but for the present purpose, I would like to stick to a socket, if I can help it(partly because I am using it within an application already using twisted to communicate.. this is a seperate connection).
"When the connect completes, the socket s can be used to send in a request for the text of the page. The same socket will read the reply, and then be destroyed. That’s right, destroyed. Client sockets are normally only used for one exchange (or a small set of sequential exchanges)."
return value for the send that succeeds = 35
return value for the send that FAILS = 32
code with some minor editing to remove any business logic.
self._commandSock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
def sendPrereqs(self,id,prereqs):
self._commandSock.connect(self._commandConnection)
#parse prereqs
temp = prereqs.split(',')
for pair in temp:
tup = pair.partition(':')
try:
command = 'some command'
logging.info('sending command: ' + command)
ret = self._commandSock.send(command)
if ret == None:
logging.info('send called successfully: ' + command)
else:
logging.info('socket returned non-None: ' + str(ret))
except:
print 'Unexpected Exception ', sys.exc_info()[0]()
print sys.exc_info()
#logging.info('Unexpected Exception '+ str(sys.exc_info()[0]()))
#logging.info(' ' + str(sys.exc_info()))
self._commandSock.close()`

return value for the send that succeeds = 35 return value for the send that FAILS = 32
Documentation says that successful send should return None.
No it doesn't. Documentation says:
Returns the number of bytes sent. Applications are responsible for checking that all data has been sent; if only some of the data was transmitted, the application needs to attempt delivery of the remaining data. For further information on this concept, consult the Socket Programming HOWTO.
You still haven't explained what you mean by "FAILS". The send call is returning successfully, and it's almost certainly placed 32 bytes into the socket write buffer.
If the only reason you think it's failing is that it returns the correct value, then the answer is obviously that it's not failing.
If something else is going wrong, there are all kinds of things that could be wrong at a higher level. One likely one is this: The server (especially if it was coded by someone who doesn't understand sockets well) is coded to expect one recv() of 35 bytes, and one recv() of 32 bytes. But it's actually getting a single recv() of 67 bytes, and waiting forever for the second, which never comes. There is no rule guaranteeing that each send() on a stream (TCP) socket corresponds to one recv() on the other side. You need to create some kind of stream protocol that demarcates the separate messages.
Meanwhile, the quote you're referring to is irrelevant. It's describing how client sockets are used by simple web browsers: They make a connection, do one send, receive all the data, then destroy the connection. I can see why it misled you, and you may want to file a documentation bug to get it improved. But for now, just ignore it.
If you want to make absolutely sure that the client is sending the right data, and the problem is in the server, there are two easy ways to do that:
Use netcat as a trivial substitute server (e.g., nc -kl 6000, replacing the "6000" with the actual port) and making sure it logs what you think the server should be seeing.
Use Wireshark to watch the connection between the client and server.
Once you've verified that the problem is on the server side, you need to debug the server. If you need help with that, that's probably best done in a new question, where you can post the server code instead of the client, and explain (with a link here) that you're sure the client is sending the right information.

The documentation is only referring to a common scenario. You can call send, sendall, and sendto on all sockets as often as you want - as long as the socket is not closed.
Note that these methods return the number of bytes sent, 32 and 35 simply mean you sent 32 bytes the first and 35 bytes the second time.
The fact that socket.send returns without an exception means that the data got handed to the operating system, but not that it actually reached the endpoint (or has been read correctly by an application there).

Packet spoofing using python

I'm trying to write a proof of concept code, which will automatically spoof packets to a thick-client application.
I've chosen python as the language of choice. I have found resources to help me monitor for packets using scapy or other similar libraries.
How do I go about spoofing the packet.
Eg. Scenario :
Client C, Server S
C sends get request R(HTTP) to S
Proxy_python intercepts request R
Proxy_python crafts a HTTP response (r)
Proxy_python sends r to C
Essentially a MiTm on C. Is this possible. One condition to note is that, the proxy i'm writing should not need any configuration to get redirected to. It should ubiquotously listen for all packets.
Which makes me ask me another question : Can I make the python proxy listen to a particular PID ?

At least to answer the question regarding whether you can tie a PID to the packets being sent, this is not something that is explicitly included within the packet data. However, you can determine which port the process is sending traffic on and associate packets on that port to the process. I would reference this question for some information on how to get that port info. Hope this helps a little, not sure exactly what else you are looking for at the moment.