First I am sending the requested file from the server to the client and after that I want to send the computed sha of the file from the server to the client, so that the client can check if both the sha from the sent and the received files are the same.
I manage to send the file but when I try to also send the sha (which is a variable) I receive a error ( i believe that the sha is also added to the file content)
How can i send them separately?
if (reqCommand == 'get'):
with open (reqFile, 'rb') as in_file, open(encFile, "wb") as out_file:
encrypt(in_file, out_file, "abc")
f = open(encFile,'rb')
for data in f:
# print 'here3'
conn.sendall(data)
f.close()
file_sh = hashfile(reqFile)
print 'the sha1 function from the server: ', file_sh
conn.send(file_sh)
and the client:
while True:
data = sock.recv(1024)
if not data:
break
#print data
file_to_write.write(data)
You should redesign a bit how your app works:
First the servers sends to the client the file size
The client reads the file size (converts it to a number) and notifies the server (sends "OK" to the server for example)
The server reads the "OK" from the client and starts to send the file contents (preferably in smaller chunks)
The client keeps reading data until either it reads exactly "file size" bytes or error occurs
If no error occurred the client computes the hash of the file that it just received and sends it to the server
The server reads the hash from client and compares with the one of its local file - if they match it sends "OK" to the client "ERROR" otherwise
The client reads the response from server: if "ERROR" is received the file is deleted
A TCP stream needs to be formatted in both ends, since it is a stream not packets. I suppose you could look for a nullbyte ('\x00') which should signal end of file.
Related
My server is sending serial of files with name like file_1, file_2, and so on.
The sending socket works well and I've checked those files all correct.
Server - sending files
f = open(new_filename, 'rb')
start_ts = ts
seconds += 1
try:
print('Sending %s' % new_filename)
conn.sendall(f.read(99999999))
f.flush()
f.close()
except socket.error:
if errno == errno.ECONNREFUSED:
print(os.strerror(socket.error.errno))
else:
raise
print('Send failed')
sys.exit()
Here on the client side, I want to receive those files and save them as the way it was on the server (save_1, save_2, ...) on my computer.
Client- receive files and try to save them
try:
client.send(b'Receiving Data...\n')
while True:
save_filename = 'savefolder/save_%i.pcap' % file_index
f = open(save_filename, 'wb')
data = client.recv(99999999)
f.write(data)
reply = b'Message Received.\n'
if not data:
break
client.sendall(reply)
f.close()
file_index += 1
except socket.timeout:
print('Done receiving.', end=' ')
client.close()
print('Client socket is closed')
But the saved files look weird. When the server sends one 3,755 KB file, the client writes two files with 192KB and 3,563KB. Worse, the bigger one crashes. Is there a fix for this? I don't know why it happens here with my code.
Is it the only way to input 99999999 value to get the entire file without cutting?
Why is my client not sending message to the server even though I wrote some code in there?
TCP is not a message based protocol but a byte stream protocol. There is no fixed relation between how much was send or sendall and how much gets read with recv.
In order to send multiple messages (files) over the same connection you have to define some application protocol which clearly defines where messages start and end. And alternative would be to use a new TCP connection for each file and read until recv returns '', i.e. indicates that the other site has closed the connection.
I'm currently writing a small client-server application for transferring an arbitrary file from a server to a client, via sockets.
The server will only handle one client at a time, but when a client is served it shall be ready to handle a new client connection.
The client will request a file, if the file exist, the client will receive the file, write it to disk and close the connection.
Server code:
PORT = 9000
BUFSIZE = 1000
def main(argv):
print('The server is ready to receive')
server_socket = socket(AF_INET, SOCK_STREAM)
server_socket.bind(('', PORT))
server_socket.listen(1)
while True:
connection_socket, addr = server_socket.accept()
try:
requested_filepath = connection_socket.recv(BUFSIZE).decode()
print("Client requested the file: " + requested_filepath)
capital_sentence = requested_filepath.upper()
if(os.path.isfile(requested_filepath)):
filesize = str(os.path.getsize(requested_filepath))
connection_socket.send(filesize.encode())
with open(requested_filepath, 'rb') as f:
while(True):
content = f.read(BUFSIZE)
if not content:
break
connection_socket.send(content)
print('File has been send')
else:
error = "error"
connection_socket.send(error.encode())
finally:
connection_socket.close()
Client code:
PORT = 9000
BUFSIZE = 1000
def main(argv):
servername = argv[0]
filepath = argv[1]
client_socket = socket(AF_INET, SOCK_STREAM)
client_socket.connect((servername, PORT))
try:
client_socket.send(filepath.encode())
response = client_socket.recv(BUFSIZE).decode()
if(response != "error"):
filesize = int(response)
print("Requested filesize: " + str(filesize))
filename = filepath.split('/')[-1]
with open(filename, 'wb') as f:
while(True):
content = client_socket.recv(BUFSIZE)
if not content:
break
f.write(content)
print('File recived')
else:
print("The requested file did not exist")
finally:
client_socket.close()
I can run the server and get the client to request and get a file, but when i run the client for a second or third time, the server and client seems to get out of sync. Both programs breaks and returns the following error message:
Client error:
Traceback (most recent call last):
File "client.py", line 37, in <module>
main(sys.argv[1:])
File "client.py", line 16, in main
response = client_socket.recv(BUFSIZE).decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 6: invalid start byte
Server error:
The server is ready to receive
Client requested the file: /pepe.jpeg
File has been send
Client requested the file: /pepe.jpeg
File has been send
Client requested the file: /pepe.jpeg
Traceback (most recent call last):
File "server.py", line 44, in <module>
main(sys.argv[1:])
File "server.py", line 30, in main
connection_socket.send(content)
ConnectionResetError: [Errno 104] Connection reset by peer
Am I not closing the socket connection in a proper way?
You have fallen into one of the most common TCP socket programming traps. You assumed your socket would send messages, while it sends and receives only data and is completely agnostic to your messaging structure. Even if you send data using several send calls, your recv calls do not receive this exact structure but whatever happens to be in the buffer. If you sent one byte a thousand times, your recv(1000) would receive a thousand bytes and this is what is going on here.
Your issue is caused by your server being a bit faster than your client. I had to tweak your code to be able to reproduce the code reliably but this does it:
client_socket.send(filepath.encode())
sleep(1)
response = client_socket.recv(BUFSIZE).decode()
This emulates your server being faster than the client, which eventually will happen anyway. By adding sleep we can make it happen every time.
When you call recv on a TCP socket, one of the following five things can happen:
There is no data and the call blocks
You received data and the data you received is exactly one "message", whatever that is in your context
Your server had sent more than one message before you read from the socket and you received them all on one go
Your client was too eager to read and it decided to read when only a part of your first message was available
Combination of 3 and 4: You receive several full messages plus one partial
What happens with your code is that your server has managed to send the encoded file size and some of your data as well. On your client you now assume your first recv receives only the file size, but this is no way guaranteed. There can be already some file data (as you will read BUFSIZE - there can be almost a full buffer of data there) and when you try to decode that as an integer, weird things happen as the data is not what you expected it to be.
The only reliable way to handle TCP sockets is to read from the socket, append to a temporary processing buffer, then parse that buffer and see what is in there. If there is a "message", process it and delete it from the buffer. Whatever remains in the buffer must stay there and your next recv result gets appended to this.
The simplest way to quickfix this is if your server makes the initial message of a fixed length. Then you can safely read exactly this amount of characters from the socket and process this as the size/error message, and the rest will be data. This is a horrible fix in many, many ways and you should aim for something better. The "proper" way is to devise a protocol, where the server puts delimiters in place so that your client can detect which message means what. Your protocol could be for example
SIZE: <decimal>\n
DATA: <data>
or even as simple as assuming everything before a newline is filesize and everything that follows is data.
But this works better even with sleep(1) added as it will now pad the initial message to exactly 100 bytes. This could still go wrong because of (4), so actually you will need to check that you received 100 characters initially and keep reading until you do, but I will leave this for you to implement.
if(os.path.isfile(requested_filepath)):
filesize = str(os.path.getsize(requested_filepath))
connection_socket.send(("%s" % filesize).encode().ljust(100))
with open(requested_filepath, 'rb') as f:
while(True):
content = f.read(BUFSIZE)
if not content:
break
connection_socket.send(content)
print('File has been send')
else:
error = "error"
connection_socket.send(error.encode().ljust(100))
Client:
try:
client_socket.send(filepath.encode())
sleep(1)
response_raw = client_socket.recv(100)
response = response_raw.strip().decode()
PS your server should catch the "connection reset by peer" error. It is something that can happen if there is a network problem or the client application crashes. The server can safely ignore this error and just stop sending to that particular client socket.
I’m trying to send and receive file through TCP socket
There are a lot of problems
1. When the client connects to the server. The server does not “print Client connected ..” but it prints after using the command.
2. When I use the ‘put’ command at the server occur an error socket.error: [Errno 107] Transport endpoint is not connected but the file image is already uploaded to the server.
3. When I use the ‘get’ command at the client. I can’t continue to use another command.
4. The last problem is the client can’t quit and list file from the server. It shows AttributeError: 'module' object has no attribute 'send'
Server
import socket
import sys
import os
HOST = 'localhost'
PORT = 3820
socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket.bind((HOST, PORT))
socket.listen(1)
while (1):
conn, addr = socket.accept()
print 'Client connected ..'
reqCommand = conn.recv(2048)
print 'Client> %s' %(reqCommand)
if (reqCommand == 'quit'):
break
#list file on server
elif (reqCommand == 'lls'):
start_path = os.listdir('.') # server directory
for path,dirs,files in os.walk(start_path):
for filename in files:
print os.path.join(filename)
else:
string = reqCommand.split(' ', 1) #in case of 'put' and 'get' method
reqFile = string[1]
if (string[0] == 'put'):
with open(reqFile, 'wb') as file_to_write:
while True:
data = socket.recv(1024)
# print data
if not data:
break
# print data
file_to_write.write(data)
file_to_write.close()
break
print 'Receive Successful'
elif (string[0] == 'get'):
with open(reqFile, 'rb') as file_to_send:
for data in file_to_send:
conn.sendall(data)
print 'Send Successful'
conn.close()
socket.close()
Client
import socket
import sys
import os
HOST = 'localhost'
PORT = 3820
def put(commandName):
socket1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket1.connect((HOST, PORT))
socket1.send(commandName)
string = commandName.split(' ', 1)
inputFile = string[1]
with open(inputFile, 'rb') as file_to_send:
for data in file_to_send:
socket1.sendall(data)
print 'PUT Successful'
socket1.close()
return
def get(commandName):
socket1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket1.connect((HOST, PORT))
socket1.send(commandName)
string = commandName.split(' ', 1)
inputFile = string[1]
with open(inputFile, 'wb') as file_to_write:
while True:
data = socket1.recv(2048)
# print data
if not data:
break
# print data
file_to_write.write(data)
file_to_write.close()
print 'GET Successful'
socket1.close()
return
msg = raw_input('Enter your name: ')
while(1):
print 'Instructions'
print '"put [filename]" to upload the file to the server '
print '"get [filename]" to download the file from the server '
print '"ls" to list all files in the client'
print '"lls" to list all files in the server'
print '"quit" to exit'
sys.stdout.write('%s> ' % msg)
inputCommand = sys.stdin.readline().strip()
if (inputCommand == 'quit'):
socket.send('quit')
break
#list file on client
elif (inputCommand == 'ls'):
start_path = os.listdir('.') # client directory
print start_path
#list file on server
elif (inputCommand == 'lls'):
socket.send('lls')
else:
string = inputCommand.split(' ', 1)
if (string[0] == 'put'):
put(inputCommand)
elif (string[0] == 'get'):
get(inputCommand)
socket.close()
Among other things, you need to add "framing" to your transfer protocol. When you do a send on a stream socket, the data gets added to a buffer that will eventually be delivered to the other side. However, the size of that buffer is not transmitted to the other side. In other words, say you send an initial segment with the command "put myfile.txt". Then you send the data from myfile.txt. Now because you are using the file object iterator (for data in file_to_send), you are actually sending it a line at a time (arguably, for a file transfer protocol, it would make more sense to read and send fixed chunks but this would work too). Let's assume the first line of myfile.txt is "The quick brown fox\n"
When the server does its first receive, it could receive "put " or "put myfile.txt" or "put myfile.txtThe quick brown fox\n" or the put command plus the entire file contents. That's because the stream protocol (TCP) does not maintain message boundaries for you.
Now in practice, you may be receiving only the "put myfile.txt" in the first receive, but it's very unwise to count on that because it's dependent on the timing of all sorts of factors on both sending and receiving systems that are outside your control.
So, there are two common ways of handling this:
Add a length at the beginning that delineates the size of the command and any command argument (so that you know where in the stream the actual file data to be transferred begins). (Most binary file transfer protocols work this way.)
Add some known delimiter at the end of your command -- for example, '\n'. HTTP, for example, works this way.
And likewise your receiving side needs to ensure that it is reading exactly the amount needed at each point in order to preserve the full content of the file being transferred.
That means you either (a) are careful to recv exactly the number of bytes you need for the command data, then separately process the file content, or (b) recv an initial chunk of data into a buffer, then carve off exactly what you need for the "command", and ensure the rest will be processed later as file data. Option (b) can often be accomplished by building a file object on top of the socket (see socket.makefile), then using the file object's readline method to obtain only the first line.
Other issues with your code:
The line socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) is a very bad idea. You have just hidden the entire socket module by creating a variable with the same name. For example, if you attempted to refer to socket.AF_INET again on the subsequent line, you'd get the exception AttributeError: '_socketobject' object has no attribute 'AF_INET'. You should name the variable something else, say socket1 as you did on the client side.
On the client side, you have the opposite problem. You're attempting to use a socket object method, but providing the socket module object instead. (That's why you get AttributeError: 'module' object has no attribute 'send'.) You need to reorganize your code so that you call the send method on a connected socket object as you are doing in the put and get functions.
The error socket.error: [Errno 107] Transport endpoint is not connected occurs because you are attempting to recv on the listening socket, not the connected one (conn -- which is returned by socket.accept). The only thing you can do with a listening socket, is accept new connections (or close).
You should be using sendall instead of send to ensure that every byte gets sent. Generally, all the data will get sent with send too, but there are corner cases where that doesn't happen.
Your file-receiving loop in the server begins with while True: but then always breaks. Hence it will only receive the first chunk of data (up to 1024 bytes). If the file is larger than that, you will certainly end up truncating it.
Your server file listing function (lls command) is not sending its output back to the client, it's only printing it to the server side's console. (And don't forget, you will need to provide a delimiter of some kind between the file names you send back or they will end up all concatenated into a single big string.)
I have written two client and server python scripts were server is listening for client to send files following is my server code for listening files. The problems with the code i cannot separate filename from the file data. Following output is received data.txtMyName
def sendFileName(self):
self.clientsocket.send("name:" + self.filename)
print 'filename', self.filename
def sendFile(self):
f=open(self.filename,"rb")
data= f.read(1024)
while (data):
if(self.clientsocket.send(data)):
print "sending data"
data = f.read(1024)
#readByte = open(self.filename, "rb")
#data = readByte.read()
#readByte.close()
#self.gateway.send(data)
self.clientsocket.close()
f.close
def receiveFile(self,sock):
data = sock.recv(1024)
print 'filename', data.strip()
f = open(data.strip(), "wb")
data = sock.recv(1024)
while (data):
f.write(data)
data=sock.recv(1024)
f.close()
self.server_socket.close()
You have to create your own simple protocol on top of TCP/IP to get it work. The simplest I would imagine is to add one special character (for example 0x00) between filename and file content. Receiving site could detect this char and split filename and file content.
More complicated protocol could also send file size so the receiving site could detect when transmission is finally over and if all bytes were send.
It will also be good to send acknowledge message to sender that receiver got whole file.
There is lot of transfer file protocols (TFTP, FTP) and they exist for a reason, because pure TCP/IP without any protocol on top is useless.
I am trying to implement a RFC1350 on top of a UDP. So far all was smooth sending a file
from server to client worked like a charm i gave the method for receiving data to server and sending data to client but this direction is a no go.
Key Server code:
def listen(self):
while True:
packet, address = self.serverSocket.recvfrom(512)
mode = str(packet)[2:5]
self.file = str(str(packet)[6:]).replace("'", "")
if(mode == "RRQ"):
self.sendResponse(address)
else:
self.receiveData()
def receiveData(self):
data = open("new1.jpg", "wb")
while True:
packet, server = self.serverSocket.recvfrom(512)
if packet.__len__() == 512:
data.write(packet)
else:
data.write(packet)
break;
Key Client code:
def sendWRQ(self):
request = 'WRQ-' + self.file
self.clientSocket.sendto(str(request).encode(), (self.serverAddress, self.serverPort))
self.sendData()
def sendData(self):
with open(self.file, "rb") as data:
while True:
packet = data.read(512)
if packet != b"":
self.clientSocket.sendto(packet, (self.serverAddress, self.serverPort))
else:
self.clientSocket.sendto(packet, (self.serverAddress, self.serverPort))
break
time.sleep(0.0005)
Client sends WRQ packet whit a name of a file that will be the key of transfer
Server recog. the transfer type RRQ or WRQ in this instance and starts listening for
transfer via receiveData().
Client terminates after sending sendWRQ() now a problem occurs on either Server or Client side in sendData or receiveData i get a file whit 0kB
All of the code:
Server Class: http://www.copypastecode.com/181330/
Client Class: http://www.copypastecode.com/181326/
The method that the client informs the server the file is completed, is not correct.
In your code, when the file is completed, at the client side, you will call "sendto" to send an empty string, but this will actually do nothing; at the server side, you use condition "packet.len() == 512" to judge whether the file is completed, however, during the transfer process, if the server cpu is running faster than the transfer speed, you will get zero packet length frequently, but this does not indicate that the transfer is completed, maybe the next packet is just on the way.
My suggestion is to use a special command to indicate the end of transfer, and the server will only break the loop when that command is received.