I have been working on an image transfer server and I've run into a problem with saving multiple images. The server can only save multiple images (from more than one client) if the socket on the client end closes which only allows that client to send one image. I've also tried to send the filesize and stop saving when the filesize is met, but the image then becomes impossible to open for whatever reason.
Client Code
f = open(filename, "rb")
while True:
data = f.read(512)
if not data:
break
s.send(data)
f.close()
Server Code
for client in clients:
while True:
data = client[0].recv(512)
if data == "":
break
f.write(data)
Try to send an empty string from the client after the transfer of the image is completed. I believe that the problem lies in the fact that the server does not distinguish how many images are transmitted.
Client:
for file in files:
with open(filename, "rb") as f:
data = f.read()
while data:
s.send(data[:512])
data = data[512:]
s.send('')
Server:
from functools import partial
while True:
with open(filename, "rb") as f:
data = b"".join(iter(partial(client[0].recv, 512), b''))
f.write(data)
Related
I'm writing a file transfer program between two sockets.
The sender has the usual game:
initiate file send
pick file to send, send filesize over socket.
pick recipient
confirm
Then the server reaches out to the recipient:
do you want to accept file of this filesize?
wait for y or n.
Recipient:
yes
receive.
and all the edge cases therein.
I am running a send thread and receive thread on both socket and server.
on the server I'm doing something like this.
import threading
accept_thread = Thread(target=accept)
send_thread = Thread(target=send)
def accept():
while True:
data = client.recv(BUFF)
# break if no data
if not data: break
# if command is /sendfile
if data == b'/sendfile':
# get filesize
filesize = client.recv(BUFF)
# get recipient
recipient = client.recv(BUFF)
# check if recipient is in address list.
if recipient_exists(recipient):
sender_socket.send(b'valid')
ask = recipient_socket.send(b'Do you want to accept?')
if ask == 'n':
sender_socket.send(b'rejected')
else:
data = sender_socket.recv(BUFF)
recipient_socket.send(data)
msg = data
clients.send(msg.encode())
Needless to say, things somehow go out of whack. I keep getting my sendfile tagged onto the end of whatever I send. For instance, if I send a text file, the last bytes will always be '/sendfile'.
And if the sender types anything when they're not supposed to, then that gets added to the file transfer as well.
What do I need to use in order to make this work? I tried putting a thread lock before the receive.
Edit: Here is my send and receive:
def xfer_file(self, path, server_socket):
"""Opens file at path, sends to server_socket."""
with open(path, 'rb') as f:
server_socket.sendfile(f, 0, self.filesize)
def receive_file(self, data, BUFFSIZE, filesize, client):
"""Download file transfer from server."""
try:
data = client.recv(BUFFSIZE)
bytes_recd = len(data)
print("STARTING=========")
# target_path = self._set_target_path()
with open('image(2).jpg', 'wb') as f:
while bytes_recd < int(filesize):
f.write(data)
data = client.recv(BUFFSIZE)
bytes_recd = bytes_recd + len(data)
print(bytes_recd)
print(data)
if data == b'':
raise RuntimeError("socket connection broken")
except ValueError:
pass
print("FINISHED==========")
```
This question already has an answer here:
sending multiple files in python
(1 answer)
Closed 3 years ago.
I am using python 3.6 and ubuntu 18.04.
I am able to send single music file using socket-python (in binary mode) and i want to send multiple music files from server to client.
Problem is, at the receiver end (that is client), all the music files (approx 120 files sent from server) gets collected in one single file making it a 9 hour long single music file.
I have tried using time.sleep method (does not work), tried sending bogus element (error was shown) and tried sending some random character to end the file writing at the client side and initiate new file write (but random character requires encoding and decoding, so again error as binary data was unable to decode).
SERVER CODE
import socket
import os
import send_file
import time
s = socket.socket()
host = ""
port = 9997
s.bind((host, port))
s.listen(5)
print("Binding Done\n")
socket_object, address = s.accept()
print("Connection Established\n")
print("Sending file...")
file_class = send_file.send_files() #ignore
file_names = file_class.files #ignore - contains list of path of music file
socket_object.sendall( str(len(file_names)).encode() )
for i in file_names:
f = open(i, 'rb')
buf = f.read(1024)
while buf:
socket_object.sendall(buf)
buf = f.read(1024)
f.close()
print("Files Send")
socket_object.close()
s.close()
CLIENT CODE
import socket
import os
import time
def recv_file(i):
f = open("/home/ravi/PycharmProjects/File_Transfer/B/"+"M"+str(i)+".mp3", 'wb')
buf = s.recv(1024)
while buf:
f.write(buf)
buf = s.recv(1024)
f.close()
s = socket.socket()
host = "127.0.0.1"
port = 9997
s.connect((host, port))
print("Receiving data...")
l = s.recv(1024).decode() #ignore - length of total number of files i.e., 120 approx
for i in range(int(l)):
recv_file(i+1)
print("Files Collected")
s.close()
Any suggestions would be appreciated. Thank You.
TCP is a stream-oriented protocol. It is designed for sending streams of data. At TCP level there are no files. It doesn't split streams in any file-oriented chunks.
Look at your code:
for i in file_names:
f = open(i, 'rb')
buf = f.read(1024)
while buf:
socket_object.sendall(buf)
buf = f.read(1024)
f.close()
You just glue all files in a single stream, and your client has no idea when one file ends and the next file starts.
The task of sending multiple files over TCP could be solved in many ways:
Develop your own protocol. E.g.: first send the number of files, then send an array of 8-byte-encoded file lengths, and then the stream of file contents. The receiving end reads number of files, then parses file lengths. Knowing the lengths the receiver correctly splits the stream into files.
Use existing multi-file packaging formats: tar, cpio, zip, etc. Pack files before sending, then send the resulting package. On the receiving end unpack the package after receiving.
Recommended way Use existing protocols for sending files over TCP: TFTP or FTP.
I'd recommend using TFTP. It is very simple and reasonably efficient. There are several implementations in Python, such as tftpy
On the remote machine where you want to upload your files to, start TFTP server:
import tftpy
server = tftpy.TftpServer('<destination-folder>')
server.listen('0.0.0.0', 54321)
On the machine with files start the client:
import tftpy
client = tftpy.TftpClient('your.server.address', 54321)
for name in ("fileA", "fileB", "fileC"):
# the first argument is the name on the remote machine
# the second argument is the name on the local machine
client.upload(name, name)
I'm facing strange error right now, I have python script, that is sending/receiving data using TCP socket, everything works fine, but when I'm trying to download image with this script, it will download it, but there is a missing one-pixel row. Any ideas on how to fix it?
Server download script:
def download(self, cmd):
try:
self.c.send(str.encode(cmd))
command,filename=cmd.split(' ')
nFile = open(filename, 'wb')
i = self.c.recv(1024)
while not ('complete' in str(i)):
nFile.write(i)
i = self.c.recv(1024)
nFile.close()
self.reset = True
print('\nGot that file')
except Exception as e:
print(e)
Client upload script:
def upload(self, filename):
try:
fd = open(filename, 'rb')
data = fd.read(1024)
while (data):
self.s.sendall(data)
data = fd.read(1024)
self.s.send(str.encode('complete'))
fd.close()
except Exception as e:
print(e)
EXAMPLE - You can see, that last row of pixels is missing:
SOLUTION(1): It's not a solution, just workaround, use the other one!
What happens if you remove the complete part of the payload before
writing the last chunk of data to nFile? – mtrw
The problem was with sending 'complete' string to the server, because the script had not enough time to get all bytes from the image. So one way to fix this is to add sleep(0.2) to the script.
Client upload script:
def upload(self, filename):
try:
fd = open(filename, 'rb')
data = fd.read(1024)
while (data):
self.s.sendall(data)
data = fd.read(1024)
sleep(0.2)
self.s.send(str.encode('complete'))
fd.close()
except Exception as e:
print(e)
SOLUTION(2):
TCP is a stream protocol with no message boundaries. This means that
multiple sends can be received in one recv call, or one send can be
received in multiple recv calls.
The delay work-around may not work reliably. You need to delimit
messages in the stream.
– Maxim Egorushkin
Server download script:
try:
msg_header = self.c.recv(4)
while len(msg_header) != 4:
msg_header += self.c.recv(4 - len(msg_header))
file_len = struct.unpack('<I', msg_header)[0]
nFile = open(filename, 'wb')
data = self.c.recv(file_len)
while len(data) != file_len:
data += self.c.recv(file_len - len(data))
nFile.write(data)
nFile.close()
print('\nGot that file')
except Exception as e:
print(e)
Client upload script:
try:
file_len = os.stat(filename).st_size
msg_header = struct.pack('<I', file_len)
self.s.sendall(msg_header)
fd = open(filename, 'rb')
data = fd.read(file_len)
while (data):
self.s.sendall(data)
data = fd.read(file_len)
fd.close()
except Exception as e:
print(e)
The problem was with sending 'complete' string to the server, because the script had not enough time to get all bytes from the image. So one way to fix this is to add sleep(0.2) to the script.
TCP is a stream protocol with no message boundaries. This means that multiple sends can be received in one recv call, or one send can be received in multiple recv calls.
The delay work-around may not work reliably. You need to delimit messages in the stream.
There are 2 common ways of delimiting messages in a stream:
Prefix messages with a header.
End messages with a suffix.
Since you are sending binary data any suffix can naturally be present in the payload. Unless the suffix is longer than the payload, which isn't practical.
Hence, what you may like to do here is prefix a fixed-size header to your payload. In this particular case, a header with a 4-byte binary file length would suffice. E.g.:
file_len = os.stat(filename).st_size
msg_header = struct.pack('<I', file_len)
self.s.sendall(msg_header)
The receiver needs to read the header first:
msg_header = self.s.recv(4)
while len(msg_header) != 4:
msg_header += self.s.recv(4 - len(msg_header))
file_len = struct.unpack('<I', msg_header)
And then read exactly file_len from the socket.
Knowing the size of the file being received also allows you to preallocate the buffer to avoid memory reallocations and/or preallocate the entire file to minimize disk fragmentation or avoid out of disk space error after the file transfer has started.
I tried to create a client server script for sending and receiving some files.
The scenario is that initially Server sends automatically a csv file to client,
client process the file and sends back an answer file.
When I start receiving the file I can see all the contents of the file but is not doing a break to continue to the next function
for sending back the answer file
Is server part problem or client or both?
Client
#GET THE WORK FILE
with open('received_file.csv', 'wb') as f:
print ('file opened')
while True:
data = s.recv(BUFFER_SIZE)
print('receiving data...')
print('data=%s', (data))
#data =''
if not data:
print('Successfully get the file')
break
f.write(data)
f.close()
Only if I place data=' ' it goes next but shouldn't server sends an empty data packet? It wouldn't work if a received file is over buffer size correct?
Server
while True:
print('Starting is ',starting)
l = f.read(BUFFER_SIZE)
while (l):
self.sock.send(l)
print('Sending..')
l = f.read(BUFFER_SIZE)
if not l:
f.close()
starting = 0
del filenames[0]
print('Sending in over..')
self.getfile()
I think the problem is that the break command is exiting the if statement instead of the while loop that you intend to break.
Here is a possible solution
Client
with open('received_file.csv', 'wb') as f:
print ('file opened')
while_loop_status=True
while while_loop_status:
data = s.recv(BUFFER_SIZE)
print('receiving data...')
print('data=%s', (data))
if not data:
print('Successfully get the file')
while_loop_status=False
f.write(data)
f.close()
Here you create a variable that determines whether to continue the while loop called "while_loop_status" then you change it from True to False when you need to break the loop.
I have written two client and server python scripts were server is listening for client to send files following is my server code for listening files. The problems with the code i cannot separate filename from the file data. Following output is received data.txtMyName
def sendFileName(self):
self.clientsocket.send("name:" + self.filename)
print 'filename', self.filename
def sendFile(self):
f=open(self.filename,"rb")
data= f.read(1024)
while (data):
if(self.clientsocket.send(data)):
print "sending data"
data = f.read(1024)
#readByte = open(self.filename, "rb")
#data = readByte.read()
#readByte.close()
#self.gateway.send(data)
self.clientsocket.close()
f.close
def receiveFile(self,sock):
data = sock.recv(1024)
print 'filename', data.strip()
f = open(data.strip(), "wb")
data = sock.recv(1024)
while (data):
f.write(data)
data=sock.recv(1024)
f.close()
self.server_socket.close()
You have to create your own simple protocol on top of TCP/IP to get it work. The simplest I would imagine is to add one special character (for example 0x00) between filename and file content. Receiving site could detect this char and split filename and file content.
More complicated protocol could also send file size so the receiving site could detect when transmission is finally over and if all bytes were send.
It will also be good to send acknowledge message to sender that receiver got whole file.
There is lot of transfer file protocols (TFTP, FTP) and they exist for a reason, because pure TCP/IP without any protocol on top is useless.