Develop Reference

Measuring packet per second for connected clients IP addresses

How can we determine the packet rate of clients connected to our server in case of multi client server using Winsock. The idea I came up with is keeping a frequency map for IP addresses of all the clients and storing the packets count for some arbitrary amount k seconds. Now after k seconds we traverse the map and see what IP addresses have more than 100*k packets, now we block these IP addresses. After every k seconds we empty the map and start again.
PSEUDO CODE: (k = 10)
map<string,int> map;
void calculate() {
for(auto &ip : map){
if(ip.second>10000) blacklist(ip.first);
}
map.clear();
Sleep(10000);
calculate();
}
int s = socket(AF_INET,SOCK_STREAM, IPPROTO_TCP);
// bind(), listen()
calculate();
while(1) {
if(recv(s,buff,len)>0) map[client.ip]++;
}

Per comments:
If someone is sending too fast, I'd like to block him permanently rather than receiving his messages less frequently. Something like this is what I'm trying to achieve
If this was UDP, I'd 100% be onboard with what you are trying to do and give the code I have. But this is TCP and your assumptions are flawed.
Let's say the sender invokes this:
send(sock, buffer, 1000, 0);
And then on the other side, you invoke this:
recv(sock, buffer, 1000, 0)
Did you know that recv may do any of the following:
It may return any value less than or equal to 1000. It could return 1 and expect you to invoke it another 999 times to consume the entire message. One of the biggest confusions with TCP socket is assuming that each send call mirrors a recv call in a 1:1 fashion. Lots of buggy apps have shipped that way.
More probably, you'll get 1 or 2 recv calls because of IP fragmentation and/or TCP segmentation. How fast you invoke recv also But this is never guaranteed or expected to be consistent. What you observe with local testing on your on LAN will not resemble actual internet behavior.
How many recv calls you get has nothing to do with how many actual IP packets or TCP segments, because "the packets" will get coalesced anyway by the TCP stack on the recv side.
Similarly, how many bytes you pass to send doesn't influence the packet count. TCP, including any number of routers and gateways in between, may split up this 1000 byte stream into additional fragments and segments.
I'm going to offer two suggestions:
Detect flood attacks by counting application protocol messages and/or the size of these application protocol messages - but not individual recv calls. That is, as you recv data, you'll accumulate this data stream into logical protocol messages based on a fixed size of bytes or a delimiter based message structure and pass it up to a higher part of your application for processing. Do the incremental count then.
Instead of trying to thwart flood attacks at the message level, it's probably simpler to just throttle clients to a fixed rate of data. That is, each time you recv data, count how many bytes it returns and use with a timer to measure an incoming bytes/second rate. If the remote side exceeds your limit, insert sleep statements in between recv calls. This will implicitly make TCP slow the other side down from sending too fast.

Why does Python sock.recv receives data from 2 different packets?

The behavior is random.
I made pcaps with wireshark and tcpdump. Both show the packets length correctly.
When I do sock.recv, I randomly receive data from 2 consecutive packets. The behaviour is rare. From aprx 100 packets, 1-2 recv contains data from 2 consecutive packets.
The packets are sent very fast. Some are received bellow 1ms. However this is not a good indicator because other packets received in similar time diff are read correctly.
The socket is AF_INET, SOCK_STREAM, non blocking and it is implemented using selectors.
The socket is a client

As #jasonharper says, TCP is a protocol that provides you with a stream of bytes. It doesn't provide you with packets.
If you have a protocol that runs over TCP, and that has a notion of individual packets, there is no guarantee that a single chunk of bytes delivered on a TCP socket will begin at the beginning of a higher-level packet or will end at the end of the same higher-level packet. A packet may be broken up between two chunks, and a chunk may include bits of more than one packet. The only guarantee you get from TCP is that the data you get is received in the order in which it's transmitted.
As noted in the comment, protocols that run atop TCP generally either use some form of terminator to mark the end of a packet or put a byte count at the beginning of a packet to indicate how many bytes are in the packet.

Reliable udp using tcp retransmit [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 3 years ago.
Improve this question
I am client receiver of UDP multicast data sent by Sender server (Stock Exchange data). I am continuously receiving udp multicast packet flow sequentially numbered 1 to approximately 35,000,000 sent uniformly over a period of 6 hours . I need to ensure all packets upto say N are received before the set of N packets is periodically processed after every say ~ 256 packets. i.e. I need reliable UDP.
Reliable UDP is mimicked using TCP retransmit. If any udp packet(s) is lost/not received, it is requested by using tcp protocol by specifying the desired missing packet range (starting number, ending number).
Sender keeps record of all the packets (stock exchange data) it has sent via UDP multicast so far. So Sender will resend by TCP only those packets numbers that the receiver specifically requests for via TCP. This is how UDP reliability is achieved by receiver. The UDP drop ratio is very small (less than 0.001%) except when starting the UDP multicast in the middle of the day, in which case all previously sent UDP packets from 1 to some N will need to be resent on TCP, while live transmission of UDP multicast data packet number N+1 onward is being received.) I can't request Sender (Stock Exchange) to change its protocol--it is fixed.
What is the efficient algorithm to implement this in terms of CPU?
The issue is speed BigOh. I can make a naive algorithm using several nested loops and methods, but it not necessarily the best.
I am thinking of maintaining a number N which confirms I have received UDP
packets 1 through N, and any packet no. M which is not the next expected packet no. N+1 will be buffered, for say 256 packets, and then TCP will be used to request the missing numbers. Then normal UDP reception resumes over from the last confirmed received number after TCP request is filled.
Example:
Suppose UDP packets received by receiver are in the following sequence {1,2,3,6,7,8,9,10 ...}
After packet No. 3, the next packet is No. 6. Packets 4 through 5 are missing.
So the missing packets {4,5} are requested using TCP request({4 through 5}), and {6,7,8,9,10} are buffered. There is enough space on the 10GBaseT LAN card for buffering 35,000,000 packets.
So: receive UDP {1,2,3}, refill by TCP request {4,5}, continue receive UDP {6,7,8,9,10, ...}

I assume since you are using multicast that there are going to be multiple receivers of this data? (Because if not, you'd probably be using unicast instead)
Therefore, if the receivers are going to have the option of requesting TCP retransmission of packets they didn't get, that means that the transmitting program will need to keep a copy of recently-sent UDP packets in memory, so that when it receives a retransmit-request, it will have the requested data available to retransmit. Assuming you're stamping each packet with a unique ID, it can store this data in a std::map or std::unordered_map or similar for quick lookup.
The real question becomes, how much of this old-packet data should the transmitter retain? ideally it would retain all of it, because you never know how much a given receiver might have missed and might want to request; but that would require infinite memory so that's not a realistic option. Probably the best you can do is decide how much RAM you're willing to tie up for this purpose, and keep a count of the total number of bytes you have in your table, and when it reaches the limit, start dropping the oldest packets from the table in order to keep its size under the limit.
I wrote an open-source library that uses essentially the technique you describe (multicast UDP + TCP-retransmit-to-recover-from-packet-loss) to synchronize databases across multiple hosts as quickly as possible; some things I learned while implementing it include:
If/when you can, pack your data-messages together into larger packets, up to the MTU of the network you are transmitting over (e.g. 1388 bytes for IPv4/Ethernet). Very small packet-sizes (like 48-bytes/packet) are inefficient, since the fixed-sized packet-headers make up a greater percentage of the total data sent/received.
Only try to send when your sending-socket indicates it is ready-for-write. (i.e. don't assume that you will never fill up the socket's outgoing-data-buffer; if your traffic is "bursty", you probably will at some point)
Minimize UDP packet loss by making your UDP sockets' send and receive buffers as large as you can get away with
Further minimize UDP packet loss by doing all the UDP receiving in a dedicated, high-priority thread (which can then route the received UDP data back to a normal-priority thread for further processing -- the main thing is to avoid allowing the receiving UDP-socket's incoming-data-buffer to overflow if possible)
For the TCP retransmission part, keep in mind that TCP streams can potentially slow down to nearly zero bytes-per-second in the worst case scenario, which makes it important to ensure that poor TCP performance to client A doesn't block the TCP communications to/from clients B, C, D, etc. This can be accomplished either via non-blocking I/O and select() (or poll() or similar), or asynchronous networking, or via multiple threads; avoid blocking I/O unless you are implementing a thread-per-socket model (and probably avoid that model as well, since a thread that is indefinitely-blocked-inside-recv() is difficult to shut down cleanly)
Think about under what circumstances (if any) it is acceptable for a client to never receive a particular packet at all; are there situations where that is okay? Or must the entire system grind to a halt until every receiver has received every packet in the group, regardless of how long that might take?
If you want to get really fancy, you can look into Forward Error Correction algorithms that encode data across packets, such that the receiver can still decode all of the data even if it never receives (up to a certain percentage of) the packets. This makes the need for a re-transmit request less likely, at the cost of making all of the packets slightly larger.

Stop tcp packets from concatenating

I have two apps sending tcp packages, both written in python 2. When client sends tcp packets to server too fast, the packets get concatenated. Is there a way to make python recover only last sent package from socket? I will be sending files with it, so I cannot just use some character as packet terminator, because I don't know the content of the file.

TCP uses packets for transmission, but it is not exposed to the application. Instead, the TCP layer may decide how to break the data into packets, even fragments, and how to deliver them. Often, this happens because of the unterlying network topology.
From an application point of view, you should consider a TCP connection as a stream of octets, i.e. your data unit is the byte, not a packet.
If you want to transmit "packets", use a datagram-oriented protocol such as UDP (but beware, there are size limits for such packets, and with UDP you need to take care of retransmissions yourself), or wrap them manually. For example, you could always send the packet length first, then the payload, over TCP. On the other side, read the size first, then you know how many bytes need to follow (beware, you may need to read more than once to get everything, because of fragmentation). Here, TCP will take care of in-order delivery and retransmission, so this is easier.

TCP is a streaming protocol, which doesn't expose individual packets. While reading from stream and getting packets might work in some configurations, it will break with even minor changes to operating system or networking hardware involved.
To resolve the issue, use a higher-level protocol to mark file boundaries. For example, you can prefix the file with its length in octets (bytes). Or, you can switch to a protocol that already handles this kind of stuff, like http.

First you need to know if the packet is combined before it is sent or after. Use wireshark to check it the sender is sending one packet or two. If it is sending one, then your fix is to call flush() after each write. I do not know the answer if the receiver is combining packets after receiving them.
You could change what you are sending. You could send bytes sent, followed by the bytes. Then the other side would know how many bytes to read.

Normally, TCP_NODELAY prevents that. But there are very few situations where you need to switch that on. One of the few valid ones are telnet style applications.
What you need is a protocol on top of the tcp connection. Think of the TCP connection as a pipe. You put things in one end of the pipe and get them out of the other. You cannot just send a file through this without both ends being coordinated. You have recognised you don't know how big it is and where it ends. This is your problem. Protocols take care of this. You don't have a protocol and so what you're writing is never going to be robust.
You say you don't know the length. Get the length of the file and transmit that in a header, followed by the number of bytes.
For example, if the header is a 64bits which is the length, then when you receive your header at the server end, you read the 64bit number as the length and then keep reading until the end of the file which should be the length.
Of course, this is extremely simplistic but that's the basics of it.
In fact, you don't have to design your own protocol. You could go to the internet and use an existing protocol. Such as HTTP.

Python: Sending large object over UDP

I am new to socket programming and recently picked up Python for it. I have a few questions in mind which I can't seems to find a definite answer for.
I am looking into sending data over UDP and have written a simple python script to do just that. Works fine sending small objects (Small pickled objects to be exact) across but how should I handle objects that are too large to be fitted in one UDP packet?
I've thought of first sizing up the object in bytes. Nothing will be done if the object is small enough to be fitted in a UDP packet, but if the object is too huge, the object will then be split up evenly (if possible) into many smaller chunks so that it can be fitted into multiple UDP packets and be sent across to the client. Once the client receive the chunks, the client will reassemble the multiple UDP packets into the original state.
I immediately hit my first brick wall when trying to implement the mentioned above.
From my research done, it doesn't seems like there is any 'effective' way in getting the byte size of an object. This means I am unable to determine if an object is too large to fit in a UDP packet.
What happen if I insist on sending an large object across to the client? Will it get fragmented automatically and be reassembled on the client side or will the packet be dropped by the client?
What is the right way to handle large object over UDP? Keeping in mind that the large object could be a file that is 1GB in size or a byte object that is 25MB in size.
Thanks in advance.
Side Notes:
I do understand that UDP packets may not always come in order and
therefore I have already implemented countermeasure for it which is
to tag a sequence number to the UDP packets sent out to the client.
I do understand that there is no assurance that the client will receive all of the UDP packets. I am not concerned about packet loss for now.
I do understand that TCP is the right candidate for what I am trying to do but I am focusing on understanding UDP and on how to handle situations where acknowledgement of packets from client is not possible for now.
I do understand the usage of pickle is insecure. Will look into it at later stage.

A UDP packet can be as large as approximately 64k. So if you want to send a file that is larger than that you can fragment yourself into packets of 64k. That is the theoretical maximum. My advice is to use fragments of smaller chunks of 500 bytes.
IP is responsible for fragmentation and reassembly of the packets if you do use 64k packets. Smaller packets of 500 bytes are not likely to be fragmented because the mtu is usually around 1500 bytes. If you use larger packets that are fragmented, IP is going to drop them if one of those fragments is lost.
You are right that using TCP is probably better to use for something like this or even an existing protocol like TFTP. It implements a per packet acking mechanism and sequence numbers just like you did.

Most applications dealing with sockets might store data in memory until it is all sent. Bad idea! I have a product application that has to send very large files over the web and have used chunking methods in the past. I just rewrote some of my code in Python ( binfileio ). In my applications, I have sent chuck files to a reserved folder and, once all chunked files were tucked in bed, I reassembled them. I've never trusted sending large files across a wire that could get cut at any time. Hope this helps.

Pytho 3.0
Code for Sending Data Through UDP Communication
import socket
UDP_IP = "127.0.0.1"
UDP_PORT = 5005
MESSAGE = "Hello, World!"
print("UDP target IP:",UDP_IP)
print("UDP target port:",UDP_PORT)
print("message:",MESSAGE)
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(bytes(MESSAGE, "utf-8"), (UDP_IP, UDP_PORT))