Develop Reference

Download several files from a local server to a client

The following codes let me download from server to client three files called tmp.bsp, tmp.seq and tmp.dms. However, just the first file tmp.dms is completely downloaded. The other one tmp.seq is filled up with the informations of tmp.bsp and tmp.bsp stay 0KB.
client:
import socket
import socket
skClient = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
skClient.connect(("127.0.0.1",2525))
sData = "Temp"
sData2 = "Temp"
sData3 = "Temp"
while True:
sData = skClient.recv(1024)
fDownloadFile = open("tmp.dms","wb")
sData2 = skClient.recv(1024)
fDownloadFile2 = open("tmp.seq","wb")
sData3 = skClient.recv(1024)
fDownloadFile3 = open("tmp.bsp","wb")
while sData:
fDownloadFile.write(sData)
sData = skClient.recv(1024)
fDownloadFile.close()
fDownloadFile2.write(sData2)
sData2 = skClient.recv(1024)
fDownloadFile2.close()
fDownloadFile3.write(sData3)
sData3 = skClient.recv(1024)
fDownloadFile3.close()
print "Download over"
break
skClient.close()
n is a counter and the prints are for debugging.
sFileName is to download one file, and used to work but since I want three files I just commented it.
server:
import socket
host = ''
skServer = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
skServer.bind((host,2525))
skServer.listen(10)
print "Server currently active"
while True:
Content,Address = skServer.accept()
print Address
files = "C:\Users\Name_user\Desktop\Networking\Send_Receive/"
fUploadFile = open(files+str('tmp.dms'),"rb")
sRead = fUploadFile.read(1024)
fUploadFile2 = open(files+str('tmp.seq'),"rb")
sRead2 = fUploadFile2.read(1024)
fUploadFile3 = open(files+str('tmp.bsp'),"rb")
sRead3 = fUploadFile3.read(1024)
while sRead:
Content.send(sRead)
sRead = fUploadFile.read(1024)
Content.send(sRead2)
sRead2 = fUploadFile2.read(1024)
# Content.send(sRead3)
# sRead3 = fUploadFile3.read(1024)
Content.close()
print "Sending is over"
break
skServer.close()
files I'm using:
server2.py is my server
Execution

The main issue with your code is that you're sending / receiving an arbitrary number of data. If your buffer (1024) is smaller than the file size then the client's file will contain less information, and if it's larger the file may contain more information (data from the next file).
You could solve this issue by sending a value that signifies the end of a file. The problem with this method is that this value can't be contained in any file, and the client must be scanning the received data for this value.
Another possible solution is to calculate the file size and send that infomation in front of the file data. This way the cilent will know how many data to expect for each file.
Using struct.pack we can create a minimal four bytes header with the file size.
def send_file(soc, path):
with open(path, 'rb') as f:
data = f.read()
size = struct.pack('!I', len(data))
soc.send(size + data)
Tthen the client can get the file size by reading four bytes and unpacking to int.
def recv_file(soc, path):
size_header = soc.recv(4)
size = struct.unpack('!I', size_header)[0]
data = soc.recv(size)
with open(path, 'wb') as f:
f.write(data)
Note that sending/receiving files with one call may raise a socket error if the file size is larger than the socket buffer. In that case you'll have to read the data in smaller chunks in a loop, or increase the buffer size with socket.setsockopt.
Here is a modified version of the above functions that can handle large files:
import struct
import os.path
def send_file(soc, path):
file_size = os.path.getsize(path)
size_header = struct.pack('!Q', file_size)
soc.send(size_header)
with open(path, 'rb') as f:
while True:
data = f.read(1024)
if not data:
break
soc.send(data)
def recv_file(soc, path):
size_header = soc.recv(8)
file_size = struct.unpack('!Q', size_header)[0]
chunks = [1024 for i in range(file_size / 1024)]
with open(path, 'wb') as f:
for chunk in chunks:
f.write(soc.recv(chunk))
f.write(soc.recv(file_size % 1024))
I haven't tested this code thoroughly, but it should work for files of any size.
An example using the send_file function in your server:
host = ''
skServer = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
skServer.bind((host,2525))
skServer.listen(10)
print "Server currently active"
Content,Address = skServer.accept()
print Address
files = ['tmp.bsp', 'tmp.seq', 'tmp.dms']
for file in files:
send_file(Content, file)
Content.close()
print "Sending is over"
skServer.close()
Using recv_file in the client:
skClient = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
skClient.connect(("127.0.0.1",2525))
files = ['tmp.bsp', 'tmp.seq', 'tmp.dms']
for file in files:
recv_file(skClient, file)
print "Download over"
skClient.close()

Yes you are right, I did run your program and found exactly same issue. I dont have enough time to work more on this issue but I found few key points which might lead you to the right work around.
https://docs.python.org/2/howto/sockets.html
The above official doc says:
When a recv returns 0 bytes, it means the other side has closed (or is in the process of closing) the connection. You will not receive any more data on this connection. Ever. You may be able to send data successfully
This is what it is happening when the third file returns 0 bytes.
But why 2nd and 3rd file is merged, I guess its because sockets are just buffered files and we might need to try making sure buffer is clear before sending another.
Read this,
Now there are two sets of verbs to use for communication. You can use send and recv, or you can transform your client socket into a file-like beast and use read and write. The latter is the way Java presents its sockets. I’m not going to talk about it here, except to warn you that you need to use flush on sockets. These are buffered “files”, and a common mistake is to write something, and then read for a reply. Without a flush in there, you may wait forever for the reply, because the request may still be in your output buffer.
But if you plan to reuse your socket for further transfers, you need to realize that there is no EOT on a socket. I repeat: if a socket send or recv returns after handling 0 bytes, the connection has been broken. If the connection has not been broken, you may wait on a recv forever, because the socket will not tell you that there’s nothing more to read (for now). Now if you think about that a bit, you’ll come to realize a fundamental truth of sockets: messages must either be fixed length (yuck), or be delimited (shrug), or indicate how long they are (much better), or end by shutting down the connection. The choice is entirely yours, (but some ways are righter than others).
Hope this helps.

I'm not totally fluent in Python, but I think your while statement should be something like:
while: sData or sData2 or sData3
I may have the syntax wrong, but currently it looks like you will stop when "sData" is done and stop downloading sData2 and aData3 at that time even if they haven't finished.
Hmm--either that or the "While" isn't looping at all and it's just being used as an "if"? hard to tell without knowing the API.

Sockets Python 3.5: Socket server hangs forever on file receive

I'm trying to write a Python program that can browse directories and grab files w/ sockets if the client connects to the server. The browsing part works fine, it prints out all directories of the client.
Here's a part of the code:
with clientsocket:
print('Connected to: ', addr)
while True:
m = input("Command > ")
clientsocket.send(m.encode('utf-8'))
data = clientsocket.recv(10000)
if m == "exit":
clientsocket.close()
if m.split()[0] == 'get':
inp = input("Filename > ")
while True:
rbuf = clientsocket.recv(8192)
if not rbuf:
break
d = open(inp, "ab")
d.write(rbuf)
d.close()
elif data.decode('utf-8').split()[0] == "LIST":
print(data.decode('utf-8'))
if not data:
break
However, the problem lies in here:
if m.split()[0] == 'get':
inp = input("Filename > ")
while True:
rbuf = clientsocket.recv(8192)
if not rbuf:
break
It seems to be stuck in an infinite loop. What's more interesting is that the file I'm trying to receive is 88.3kb, but what the file returns is 87kb while it's in the loop, which is very close...
I tried receiving a python script at one time as well (without the loop) and it works fine.
Here's some of the client code:
while True:
msg = s.recv(1024).decode('utf-8')
if msg.split()[0] == "list":
dirs = os.listdir(msg.split()[1])
string = ''
for dira in dirs:
string += "LIST " + dira + "\n"
s.send(string.encode('utf-8'))
elif msg == "exit":
break
else:
#bit that sends the file
with open(msg.split()[1], 'rb') as r:
s.sendall(r.read())
So my question is, why is it getting stuck in an infinite loop if I have it set up to close when there is no data, and how can I fix this?
I'm sort of new to network programming in general, so forgive me if I miss something obvious.
Thanks!

I think I know what's the problem, but I may be wrong. It happened to me several times, that the entire message is not received in one recv call, even if I specify the correct length. However, you don't reach the end of stream, so your program keeps waiting for remaining of 8192 bytes which never arrives.
Try this:
Sending file:
#bit that sends the file
with open(msg.split()[1], 'rb') as r:
data = r.read()
# check data length in bytes and send it to client
data_length = len(data)
s.send(data_length.to_bytes(4, 'big'))
s.send(data)
s.shutdown(socket.SHUT_RDWR)
s.close()
Receiving the file:
# check expected message length
remaining = int.from_bytes(clientsocket.recv(4), 'big')
d = open(inp, "wb")
while remaining:
# until there are bytes left...
# fetch remaining bytes or 4094 (whatever smaller)
rbuf = clientsocket.recv(min(remaining, 4096))
remaining -= len(rbuf)
# write to file
d.write(rbuf)
d.close()

There are several issues with your code.
First:
clientsocket.send(m.encode('utf-8'))
data = clientsocket.recv(10000)
This causes the file to be partially loaded to data variable when you issue get statement. That's why you don't get full file.
Now this:
while True:
rbuf = clientsocket.recv(8192)
if not rbuf:
break
...
You indeed load full file but the client never closes the connection (it goes to s.recv() after sending the file) so if statement is never satisfied. Thus this loop gets blocked on the clientsocket.recv(8192) part after downloading the file.
So the problem is that you have to somehow notify the downloader that you've sent all the data even though the connection is still open. There are several ways to do that:
You calculate the size of the file and send it as a first few bytes. For example, say the content of the file is ala ma kota. These are 11 bytes and thus you send \x11ala ma kota. Now receiver knows that first byte is size and it will interpret it as such. Of course one byte header isn't much (you would only be able to send max 256 byte files) so normally you go for for example 4 bytes. So now your protocol between client and server is: first 4 bytes is the size of the file. Thus our initial file would be sent as \x0\x0\x0\x11ala ma kota. The drawback of this solution is that you have to know the size of the content before sending it.
You mark the end of the stream. So you pick a particular character, say X and you read the straem until you find X. If you do then you know that the other side sent everything it has. The drawback is that if X is in the content then you have to escape it, i.e. additional content processing (and interpretation on the other side) is needed.

I am creating a Twitch-focused IRC bot in Python, however it is getting the responses slowly. What am I doing wrong?

As the title says, although this is also the first time I have used Python to really do anything big. I'm not all that used to the language yet, so this is probably my missing something. The code is fairly short and is as followed, with username and private pass removed:
import re
import socket
import sys
import time
import string
HOST = "irc.twitch.tv"
PORT = 6667
NICK = ""
PASS = ""
CHAN = ""
RATE = (20/30) # messages per second
CHAT_MSG=re.compile(r"^:\w+!\w+#\w+\.tmi\.twitch\.tv PRIVMSG #\w+ :")
def chat(sock, msg):
sock.send("PRIVMSG #{} :{}".format(cfg.CHAN, msg))
public = socket.socket()
public.connect((HOST, PORT))
public.send("PASS {}\r\n".format(PASS).encode("utf-8"))
public.send("NICK {}\r\n".format(NICK).encode("utf-8"))
public.send("JOIN {}\r\n".format(CHAN).encode("utf-8"))
private = socket.socket()
private.connect((HOST, PORT))
private.send("PASS {}\r\n".format(PASS).encode("utf-8"))
private.send("NICK {}\r\n".format(NICK).encode("utf-8"))
private.send("CAP REQ :twitch.tv/tags twitch.tv/commands {}\r\n".format(CHAN).encode("utf-8"))
while True:
channelResponse = public.recv(1024).decode("utf-8")
privateResponse = private.recv(1024).decode("utf-8")
if privateResponse == "PING :tmi.twitch.tv\r\n":
private.send("PONG :tmi.twitch.tv\r\n".encode("utf-8"))
else:
privateResponseUsername = re.search(r"\w+", privateResponse).group(0) # return the entire match
privateResponseMessage = CHAT_MSG.sub("", privateResponse)
print(privateResponseUsername + ": " + privateResponseMessage)
if channelResponse == "PING :tmi.twitch.tv\r\n":
public.send("PONG :tmi.twitch.tv\r\n".encode("utf-8"))
else:
username = re.search(r"\w+", channelResponse).group(0) # return the entire match
message = CHAT_MSG.sub("", channelResponse)
print(username + ": " + message)
time.sleep(1 / RATE)
One thing to mention is that I was following a basic template style, however it did not cover implementing whispers into the bot - so I'm having to guess by doing research on how to do that, and it seems to be that the most recommended way is two connections, one for public, one for private.

As you've structured your code, you can't get anything from the private socket until you've gotten something from the public socket. If IRC didn't send PING messages occasionally, this would work even worse.
The way to handle this is to use select, and give it your two sockets. Right as soon as one has stuff that can be read, select will return, and indicate which socket has bytes available for reading.
This answer has some general code. You might want to modify it to look something like:
while True:
# this will block until at least one socket is ready
ready_socks,_,_ = select.select([private, public], [], [])
if private in ready_socks:
privateResponse += private.recv()
if public in ready_socks:
channelResponse += public.recv()
# check privateResponse buffer, do stuff
# check channelResponse buffer, do stuff
There are a few other things you should keep in mind:
The network doesn't have to deliver entire IRC messages at the same time, nor does it have to deliver a single one at a time. You could get "PI", "NG :t", "mi.twitch.tv", "\r\n" as separate messages. So you should accumulate bytes in a buffer, and then when you've got at least one entire message, process it, and remove it from the buffer.
UTF-8 characters can span multiple bytes, and might be split up by the network. Don't decode UTF-8 until you're sure you've got an entire message to work with.

Python Client/Server send big size files

I am not sure if this topic have been answered or not, if was I am sorry:
I have a simple python script the sends all files in one folder:
Client:
import os,sys, socket, time
def Send(sok,data,end="292929"):
sok.sendall(data + end);
def SendFolder(sok,folder):
if(os.path.isdir(folder)):
files = os.listdir(folder);
os.chdir(folder);
for file_ in files:
Send(sok,file_);#Send the file name to the server
f = open(file_, "rb");
while(True):
d = f.read();#read file
if(d == ""):
break;
Send(sok, d, "");#Send file data
f.close();
time.sleep(0.8);#Problem here!!!!!!!!!!!
Send(sok,"");#Send termination to the server
time.sleep(1);#Wait the server to write the file
os.chdir("..");
Send(sok,"endfile");#let the server know that we finish sending files
else:
Send("endfile")#If not folder send termination
try:
sok1 = socket.socket();
sok1.connect(("192.168.1.121",4444))#local ip
time.sleep(1);
while(True):
Send(sok1,"Enter folder name to download: ");
r = sok1.recv(1024);
SendFolder(sok1,r);
time.sleep(0.5);
except BaseException, e:
print "Error: " + str(e);
os._exit(1);
Server:
import sys,os,socket,time
# receive data
def Receive(sock, end="292929"):
data = "";
while(True):
if(data.endswith(end)):
break;
else:
data = sock.recv(1024);
return data[:-len(end)];#return data less termination
def FolderReceive(sok):
while(True):
r = Receive(sok);# recv filename or folder termination("endfile")
if(r == "endfolder"):
print "Folder receive complete.";
break;
else:
filename = r;#file name
filedata = Receive(sok);# receive file data
f = open(filename,"wb");
f.write(filedata);
f.close();#finish to write the file
print "Received: " + filename;
try:
sok1 = socket.socket();
sok1.bind(("0.0.0.0",4444));
sok1.listen(5);
cl , addr = sok1.accept();#accepts connection
while(True):
r = Receive(cl);
sys.stdout.write("\n" + r);
next = raw_input();
cl.sendall(next);#send folder name to the client
FolderReceive(cl);
except BaseException, e:
print "Error: " + str(e);
os._exit(1);
I know this not best server ever...but is what I know. This just work for a folder with small files because if I send big files(like 5mb...) it crashes because the time the client waits for the server is not enough.
So my question is how can I send the files to the server without client need to wait??or know exactly how many time the client needs to wait for the server to receive the file?? Some code that does the same but handling any file size, any help?

TCP sockets are byte streams, not message streams. If you want to send a series of separate messages (like your separate files), you need to define a protocol, and then write a protocol handler. There is no way around that; just guessing at the timing or trying to take advantage of packet boundaries cannot possibly work.
The blog post linked above shows one way to do it. But you can do it with string delimiters if you want. But you have to deal with two problems:
The delimiter can appear anywhere in a read packet, not just at the end.
The delimiter can be split on packet boundaries—you may get "2929" at the end of one read, and the other "29" at the start of the next.
The usually way you do that is to accumulate a buffer, and search for the delimiter anywhere in the buffer. Something like this:
def message(sock, delimiter):
buf = ''
while True:
data = sock.read(4096)
if not data:
# If the socket closes with no delimiter, this will
# treat the last "partial file" as a complete file.
# If that's not what you want, just return, or raise.
yield buf
return
buf += data
messages = buf.split(delimiter)
for message in messages[:-1]:
yield message
buf = message[-1]
Meanwhile, you have another problem with your delimiter: There's nothing stopping it from appearing in the files you're trying to transmit. For example, what if you tried to send your script, or this web page?
That's one of the reasons that other protocols are often better than delimiters, but this isn't hard to deal with: just escape any delimiters found in the files. Since you're sending the whole file at once, you can just use replace on right before the sendall, and the reverse replace right before the split.

Python Socket Receive Large Amount of Data

When I try to receive larger amounts of data it gets cut off and I have to press enter to get the rest of the data. At first I was able to increase it a little bit but it still won't receive all of it. As you can see I have increased the buffer on the conn.recv() but it still doesn't get all of the data. It cuts it off at a certain point. I have to press enter on my raw_input in order to receive the rest of the data. Is there anyway I can get all of the data at once? Here's the code.
port = 7777
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind(('0.0.0.0', port))
sock.listen(1)
print ("Listening on port: "+str(port))
while 1:
conn, sock_addr = sock.accept()
print "accepted connection from", sock_addr
while 1:
command = raw_input('shell> ')
conn.send(command)
data = conn.recv(8000)
if not data: break
print data,
conn.close()

TCP/IP is a stream-based protocol, not a message-based protocol. There's no guarantee that every send() call by one peer results in a single recv() call by the other peer receiving the exact data sent—it might receive the data piece-meal, split across multiple recv() calls, due to packet fragmentation.
You need to define your own message-based protocol on top of TCP in order to differentiate message boundaries. Then, to read a message, you continue to call recv() until you've read an entire message or an error occurs.
One simple way of sending a message is to prefix each message with its length. Then to read a message, you first read the length, then you read that many bytes. Here's how you might do that:
def send_msg(sock, msg):
# Prefix each message with a 4-byte length (network byte order)
msg = struct.pack('>I', len(msg)) + msg
sock.sendall(msg)
def recv_msg(sock):
# Read message length and unpack it into an integer
raw_msglen = recvall(sock, 4)
if not raw_msglen:
return None
msglen = struct.unpack('>I', raw_msglen)[0]
# Read the message data
return recvall(sock, msglen)
def recvall(sock, n):
# Helper function to recv n bytes or return None if EOF is hit
data = bytearray()
while len(data) < n:
packet = sock.recv(n - len(data))
if not packet:
return None
data.extend(packet)
return data
Then you can use the send_msg and recv_msg functions to send and receive whole messages, and they won't have any problems with packets being split or coalesced on the network level.

You can use it as: data = recvall(sock)
def recvall(sock):
BUFF_SIZE = 4096 # 4 KiB
data = b''
while True:
part = sock.recv(BUFF_SIZE)
data += part
if len(part) < BUFF_SIZE:
# either 0 or end of data
break
return data

The accepted answer is fine but it will be really slow with big files -string is an immutable class this means more objects are created every time you use the + sign, using list as a stack structure will be more efficient.
This should work better
while True:
chunk = s.recv(10000)
if not chunk:
break
fragments.append(chunk)
print "".join(fragments)

Most of the answers describe some sort of recvall() method. If your bottleneck when receiving data is creating the byte array in a for loop, I benchmarked three approaches of allocating the received data in the recvall() method:
Byte string method:
arr = b''
while len(arr) < msg_len:
arr += sock.recv(max_msg_size)
List method:
fragments = []
while True:
chunk = sock.recv(max_msg_size)
if not chunk:
break
fragments.append(chunk)
arr = b''.join(fragments)
Pre-allocated bytearray method:
arr = bytearray(msg_len)
pos = 0
while pos < msg_len:
arr[pos:pos+max_msg_size] = sock.recv(max_msg_size)
pos += max_msg_size
Results:

You may need to call conn.recv() multiple times to receive all the data. Calling it a single time is not guaranteed to bring in all the data that was sent, due to the fact that TCP streams don't maintain frame boundaries (i.e. they only work as a stream of raw bytes, not a structured stream of messages).
See this answer for another description of the issue.
Note that this means you need some way of knowing when you have received all of the data. If the sender will always send exactly 8000 bytes, you could count the number of bytes you have received so far and subtract that from 8000 to know how many are left to receive; if the data is variable-sized, there are various other methods that can be used, such as having the sender send a number-of-bytes header before sending the message, or if it's ASCII text that is being sent you could look for a newline or NUL character.

Disclaimer: There are very rare cases in which you really need to do this. If possible use an existing application layer protocol or define your own eg. precede each message with a fixed length integer indicating the length of data that follows or terminate each message with a '\n' character. (Adam Rosenfield's answer does a really good job at explaining that)
With that said, there is a way to read all of the data available on a socket. However, it is a bad idea to rely on this kind of communication as it introduces the risk of loosing data. Use this solution with extreme caution and only after reading the explanation below.
def recvall(sock):
BUFF_SIZE = 4096
data = bytearray()
while True:
packet = sock.recv(BUFF_SIZE)
if not packet: # Important!!
break
data.extend(packet)
return data
Now the if not packet: line is absolutely critical!
Many answers here suggested using a condition like if len(packet) < BUFF_SIZE: which is broken and will most likely cause you to close your connection prematurely and loose data. It wrongly assumes that one send on one end of a TCP socket corresponds to one receive of sent number of bytes on the other end. It does not. There is a very good chance that sock.recv(BUFF_SIZE) will return a chunk smaller than BUFF_SIZE even if there's still data waiting to be received. There is a good explanation of the issue here and here.
By using the above solution you are still risking data loss if the other end of the connection is writing data slower than you are reading. You may just simply consume all data on your end and exit when more is on the way. There are ways around it that require the use of concurrent programming, but that's another topic of its own.

A variation using a generator function (which I consider more pythonic):
def recvall(sock, buffer_size=4096):
buf = sock.recv(buffer_size)
while buf:
yield buf
if len(buf) < buffer_size: break
buf = sock.recv(buffer_size)
# ...
with socket.create_connection((host, port)) as sock:
sock.sendall(command)
response = b''.join(recvall(sock))

You can do it using Serialization
from socket import *
from json import dumps, loads
def recvall(conn):
data = ""
while True:
try:
data = conn.recv(1024)
return json.loads(data)
except ValueError:
continue
def sendall(conn):
conn.sendall(json.dumps(data))
NOTE: If you want to shara a file using code above you need to encode / decode it into base64

I think this question has been pretty well answered, but I just wanted to add a method using Python 3.8 and the new assignment expression (walrus operator) since it is stylistically simple.
import socket
host = "127.0.0.1"
port = 31337
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host,port))
s.listen()
con, addr = s.accept()
msg_list = []
while (walrus_msg := con.recv(3)) != b'\r\n':
msg_list.append(walrus_msg)
print(msg_list)
In this case, 3 bytes are received from the socket and immediately assigned to walrus_msg. Once the socket receives a b'\r\n' it breaks the loop. walrus_msg are added to a msg_list and printed after the loop breaks. This script is basic but was tested and works with a telnet session.
NOTE: The parenthesis around the (walrus_msg := con.recv(3)) are needed. Without this, while walrus_msg := con.recv(3) != b'\r\n': evaluates walrus_msg to True instead of the actual data on the socket.

Modifying Adam Rosenfield's code:
import sys
def send_msg(sock, msg):
size_of_package = sys.getsizeof(msg)
package = str(size_of_package)+":"+ msg #Create our package size,":",message
sock.sendall(package)
def recv_msg(sock):
try:
header = sock.recv(2)#Magic, small number to begin with.
while ":" not in header:
header += sock.recv(2) #Keep looping, picking up two bytes each time
size_of_package, separator, message_fragment = header.partition(":")
message = sock.recv(int(size_of_package))
full_message = message_fragment + message
return full_message
except OverflowError:
return "OverflowError."
except:
print "Unexpected error:", sys.exc_info()[0]
raise
I would, however, heavily encourage using the original approach.

For anyone else who's looking for an answer in cases where you don't know the length of the packet prior.
Here's a simple solution that reads 4096 bytes at a time and stops when less than 4096 bytes were received. However, it will not work in cases where the total length of the packet received is exactly 4096 bytes - then it will call recv() again and hang.
def recvall(sock):
data = b''
bufsize = 4096
while True:
packet = sock.recv(bufsize)
data += packet
if len(packet) < bufsize:
break
return data

This code reads 1024*32(=32768) bytes in 32 iterations from the buffer which is received from Server in socket programming-python:
jsonString = bytearray()
for _ in range(32):
packet = clisocket.recv(1024)
if not packet:
break
jsonString.extend(packet)
Data resides in jsonString variable

Plain and simple:
data = b''
while True:
data_chunk = client_socket.recv(1024)
if data_chunk:
data+=data_chunk
else:
break