Develop Reference

How to bind ports for TCP clients when using raw packet sending TCP packets in Python? [duplicate]

Ok, I realize this situation is somewhat unusual, but I need to establish a TCP connection (the 3-way handshake) using only raw sockets (in C, in linux) -- i.e. I need to construct the IP headers and TCP headers myself. I'm writing a server (so I have to first respond to the incoming SYN packet), and for whatever reason I can't seem to get it right. Yes, I realize that a SOCK_STREAM will handle this for me, but for reasons I don't want to go into that isn't an option.
The tutorials I've found online on using raw sockets all describe how to build a SYN flooder, but this is somewhat easier than actually establishing a TCP connection, since you don't have to construct a response based on the original packet. I've gotten the SYN flooder examples working, and I can read the incoming SYN packet just fine from the raw socket, but I'm still having trouble creating a valid SYN/ACK response to an incoming SYN from the client.
So, does anyone know a good tutorial on using raw sockets that goes beyond creating a SYN flooder, or does anyone have some code that could do this (using SOCK_RAW, and not SOCK_STREAM)? I would be very grateful.
MarkR is absolutely right -- the problem is that the kernel is sending reset packets in response to the initial packet because it thinks the port is closed. The kernel is beating me to the response and the connection dies. I was using tcpdump to monitor the connection already -- I should have been more observant and noticed that there were TWO replies one of which was a reset that was screwing things up, as well as the response my program created. D'OH!
The solution that seems to work best is to use an iptables rule, as suggested by MarkR, to block the outbound packets. However, there's an easier way to do it than using the mark option, as suggested. I just match whether the reset TCP flag is set. During the course of a normal connection this is unlikely to be needed, and it doesn't really matter to my application if I block all outbound reset packets from the port being used. This effectively blocks the kernel's unwanted response, but not my own packets. If the port my program is listening on is 9999 then the iptables rule looks like this:
iptables -t filter -I OUTPUT -p tcp --sport 9999 --tcp-flags RST RST -j DROP

You want to implement part of a TCP stack in userspace... this is ok, some other apps do this.
One problem you will come across is that the kernel will be sending out (generally negative, unhelpful) replies to incoming packets. This is going to screw up any communication you attempt to initiate.
One way to avoid this is to use an IP address and interface that the kernel does not have its own IP stack using- which is fine but you will need to deal with link-layer stuff (specifically, arp) yourself. That would require a socket lower than IPPROTO_IP, SOCK_RAW - you need a packet socket (I think).
It may also be possible to block the kernel's responses using an iptables rule- but I rather suspect that the rules will apply to your own packets as well somehow, unless you can manage to get them treated differently (perhaps applying a netfilter "mark" to your own packets?)
Read the man pages
socket(7)
ip(7)
packet(7)
Which explain about various options and ioctls which apply to types of sockets.
Of course you'll need a tool like Wireshark to inspect what's going on. You will need several machines to test this, I recommend using vmware (or similar) to reduce the amount of hardware required.
Sorry I can't recommend a specific tutorial.
Good luck.

I realise that this is an old thread, but here's a tutorial that goes beyond the normal SYN flooders: http://www.enderunix.org/docs/en/rawipspoof/
Hope it might be of help to someone.

I can't help you out on any tutorials.
But I can give you some advice on the tools that you could use to assist in debugging.
First off, as bmdhacks has suggested, get yourself a copy of wireshark (or tcpdump - but wireshark is easier to use). Capture a good handshake. Make sure that you save this.
Capture one of your handshakes that fails. Wireshark has quite good packet parsing and error checking, so if there's a straightforward error it will probably tell you.
Next, get yourself a copy of tcpreplay. This should also include a tool called "tcprewrite".
tcprewrite will allow you to split your previously saved capture files into two - one for each side of the handshake.
You can then use tcpreplay to play back one side of the handshake so you have a consistent set of packets to play with.
Then you use wireshark (again) to check your responses.

I don't have a tutorial, but I recently used Wireshark to good effect to debug some raw sockets programming I was doing. If you capture the packets you're sending, wireshark will do a good job of showing you if they're malformed or not. It's useful for comparing to a normal connection too.

There are structures for IP and TCP headers declared in netinet/ip.h & netinet/tcp.h respectively. You may want to look at the other headers in this directory for extra macros & stuff that may be of use.
You send a packet with the SYN flag set and a random sequence number (x). You should receive a SYN+ACK from the other side. This packet will have an acknowledgement number (y) that indicates the next sequence number the other side is expecting to receive as well as another sequence number (z). You send back an ACK packet that has sequence number x+1 and ack number z+1 to complete the connection.
You also need to make sure you calculate appropriate TCP/IP checksums & fill out the remainder of the header for the packets you send. Also, don't forget about things like host & network byte order.
TCP is defined in RFC 793, available here: http://www.faqs.org/rfcs/rfc793.html

Depending on what you're trying to do it may be easier to get existing software to handle the TCP handshaking for you.
One open source IP stack is lwIP (http://savannah.nongnu.org/projects/lwip/) which provides a full tcp/ip stack. It is very possible to get it running in user mode using either SOCK_RAW or pcap.

if you are using raw sockets, if you send using different source mac address to the actual one, linux will ignore the response packet and not send an rst.

Get TCP response from server

I have a program (.exe) that gets certain info from a server, and I wanted to be able to get that info from the command line too. I started netcat and listened on the port the program uses to communicate with its target, because I wanted to know what "requests" to make from a script, and I received this over netcat, plain text:
net.tcp://[my ip address]:41012/Lapis.Btouch/ServerInfo
I tried sending exactly that to the target (replacing my IP for its IP) using socket.send() but it wouldn't return anything. What does that mean and how can I get the data?
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('[server ip address]',41012))
while True:
s.send('net.tcp://[server ip address]:41012/Lapis.Btouch/ServerInfo')
response = s.recv(1024)
print response
s.close()
That's the code I'm using to send the request to the target server. It won't return anything, so I think I'm not making the request in the right way. Thanks for any help.

Try capture your network packet on port 41012, via tcpdump or wireshark or ...
Then check:
does your code send the request?
does the response returned to you?
If the answer for question (1) is False, problem is side of you and your machine. so go and solve it (review your code, check your firewall and ...)
If the answer of question (1) is True, but server doesn't send any response (this case often not happened) review your request format (check you URL or test it via browser or ...). also based on the server you connect to, maybe needed to your machine's IP recognized for server (I mean some server only give response to requests, that come from known IP addresses, in this case you need introduce yourself to server (e.g. add your IP to trusted list on server), before sending request)
Last case, if answer of both questions are correct (I guess it's your problem), you most correct your code (response = s.recv(1024)) why do you used 1024 for your response length?. Use correct and accurate parameters.
In python exist several methods for receiving response via socket (from server), search stackoverflow and you can find useful tips and commands. commands like use non-blocking ways, ascync ways and ...

Only allow connections from custom clients

I'm writing a Socket Server in Python, and also a Socket Client to connect to the Server.
The Client interacts with the Server in a way that the Client sends information when an action is invoked, and the Server processes the information.
The problem I'm having, is that I am able to connect to my Server with Telnet, and probably other things that I haven't tried yet. I want to disable connection from these other Clients, and only allow connections from Python Clients. (Preferably my custom-made client, as it sends information to communicate)
Is there a way I could set up authentication on connection to differentiate Python Clients from others?
Currently there is no code, as this is a problem I want to be able to solve before getting my hands dirty.

When a new connection is made to your server, your protocol will have to specify some way for the client to authenticate. Ultimately there is nothing that the network infrastructure can do to determine what sort of process initiated the connection, so you will have to specify some exchange that allows the server to be sure that it really is talking to a valid client process.

#holdenweb has already given a good answer with basic info.
If a (terminal) software sends the bytes that your application expects as a valid identification, your app will never know whether it talks to an original client or anything else.
A possible way to test for valid clients could be, that your server sends an encrypted and authenticated question (should be different at each test!), e.g. something like "what is 18:37:12 (current date and time) plus 2 (random) hours?"
Encryption/Authentication would be another issue then.
If you keep this algorithm secret, only your clients can answer it and validate themselves successfully. It can be hacked/reverse engineered, but it is safe against basic attackers.

Identifying services listening on remote ports

I'd like to elaborate a little bit on this question and, particularly, on this answer. Let's suppose I have a fixed list of services to check, say: ('ftp', 'ssh', 'http'). Opening a socket to port 22 of a remote server:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
result = s.connect_ex(('my_remote_server', 22))
if(result == 0) :
print s.recv(256)
s.close()
I get the following output:
'SSH-2.0-OpenSSH_6.1p1 Debian-4\r\n'
So I can guess there's a ssh service listening on that port. Therefore my question is: where or how can I find the welcome messages (or similar) for different kind of services? Are these regulated in some way? In the case of ssh, does it always start with SSH-?

The response you get from a server depens both on the exact protocol and server configuration.
HTTP has a Server header for example, but many server out on the internet lies about what server is responding, to throw off malicious hackers trying to exploit server-specific flaws. Moreover, the server remains silent until the client has sent a request; your code has to send something before there is any response.
The best you can do is try out different protocols and see if the other side responds sensibly. Trying to send an SSH handshake to a HTTP port is likely to result in a HTTP/1.0 400 Bad Request response, for example.
In other words, there is no ready-made solution for this, you'll have to create your own heuristics based on what protocols you are trying to detect.

The best you can do is probably search for the string ssh(/ftp/http) and conclude based on that.
Even that would be a guess, of course, as this can usually be changed(see here) with a config parameter and it is considered good security practice to change it so as to not expose the server program/version.

Using pcapy or scapy to monitor self-generated (HTTP) network traffic

I need to monitor how long it takes for a certain website to respond when addressed. I would like to sniff the traffic on port 80 but only when there is traffic being exchanged with the targeted site. I have searched SO and it seems like pcapy or scapy is the right tool for the job, but they seem deeper than I need. I have studying the following script:
Network traffic monitor with pcapy in python
and I think I need to change the
def __handle_packet(self, header, data):
# method is called for each packet by dispatch call (pcapy)
self._dispatch_bytes_sum += header.getlen() #header.getlen() #len(data)
logger.debug("header: (len:{0}, caplen:{1}, ts:{2}), d:{3}".format(header.getlen(), header.getcaplen(), header.getts(), len(data)))
#self.dumper.dump(header, data)
to somehow only unpack/handle packets that are destined for the target site. Note that this is for a Windows XP machine on a LAN and it is critical that the browser initiate the traffic.
Any pointers appreciated?

The problem with scapy is it doesn't handle reassembling TCP streams. Your HTTP that you're looking for is likely to be embedded in a TCP stream. To quote the docs:
Scapy is based on a stimulus/response model. This model does not work well for a TCP stack. On the other hand, quite often, the TCP stream is used as a tube to exchange messages that are stimulus/response-based.
Like you said scapy is more ideal for lower-layer things. You could, for instance, probably track IP packets on DHCP requests. Like many network tools, the complexities and stream-based nature of TCP means once you cross that layer it gets harder to reassemble everything and deal with all the retransmission and what not edge cases and coherently pull data out.
Could you use something like curl or urllib and see how long it takes for the response to come back?