I'm trying to use ftplib to transfer a set of files from my computer (running 64-bit Windows 7) to a Linux server. The process is similar to this snippet of test code (server address, username, and password changed, obviously):
import ftplib
import os.path
import os
host = "some.ftp.server.com"
username = "username"
password = "password"
outDir = "/some/output/directory"
def transfer_files():
ftp = ftplib.FTP(host, username, password)
ftp.cwd(outDir)
names = ftp.nlst()
if "transferred" not in names:
ftp.mkd("transferred")
ftp.cwd("transferred")
names = ftp.nlst()
# Transfrer arbitrary files to the server
filesToTransfer = os.listdir('.')
for fName in filesToTransfer:
if not os.path.isfile(fName):
continue
if fName in names:
ftp.delete(fName)
with open(fName, 'r') as f:
ftp.storbinary("STOR %s" % fName, f)
print fName
ftp.quit()
print "Done"
if __name__ == "__main__":
transfer_files()
The behavior I'm seeing is that most files transfer quickly and successfully, but randomly a file will instead time out and raise the following exception:
Traceback (most recent call last):
File "Test.py", line 37, in <module>
transfer_files()
File "Test.py", line 29, in transfer_files
ftp.storbinary("STOR %s" % base, f)
File "C:\Python27\Lib\ftplib.py", line 471, in storbinary
conn = self.transfercmd(cmd, rest)
File "C:\Python27\Lib\ftplib.py", line 376, in transfercmd
return self.ntransfercmd(cmd, rest)[0]
File "C:\Python27\Lib\ftplib.py", line 335, in ntransfercmd
conn = socket.create_connection((host, port), self.timeout)
File "C:\Python27\Lib\socket.py", line 575, in create_connection
raise err
socket.error: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
The file that times out is different each time I run the program, but it always seems to happen to one file or another. Why could the transfer be randomly timing out, and what can I do to prevent this?
When working with network requests in bulk, it's completely normal to get a certain number of timeouts - it's even expected. Rather than try to prevent timeouts from happening (because they always will), you should add logic to your application to handle cases where timeouts occur. Since you're uploading files in bulk, that probably looks like re-attempting the upload when that exception gets raised.
Related
I am trying to send a file from the master node to minion nodes using a python script but a single error OSError: Failure keeps on coming up.
I tried to code this file to send this file from one local machine to another local machine.
My code:
#! /usr/bin/python
#! /usr/bin/python3
import paramiko
import os
#Defining working connect
def workon(host):
#Making a connection
ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy()) #To add the missing host key and auto add policy
ssh_client.connect(hostname = host, username = 'username', password = 'password')
ftp_client = ssh_client.open_sftp()
ftp_client.put("/home/TrialFolder/HelloPython", "/home/")
ftp_client.close()
#stdin, stdout, stderr = ssh_client.exec_command("ls")
#lines = stdout.readlines()
#print(lines)
def main():
hosts = ['192.16.15.32', '192.16.15.33', '192.16.15.34']
threads = []
for h in hosts:
workon(h)
main()
Error:
Traceback (most recent call last):
File "PythonMultipleConnectionUsinhSSH.py", line 28, in <module>
main()
File "PythonMultipleConnectionUsinhSSH.py", line 26, in main
workon(h)
File "PythonMultipleConnectionUsinhSSH.py", line 15, in workon
ftp_client.put("/home/Sahil/HelloPython", "/home/")
File "/usr/local/lib/python3.6/site-packages/paramiko/sftp_client.py", line 759, in put
return self.putfo(fl, remotepath, file_size, callback, confirm)
File "/usr/local/lib/python3.6/site-packages/paramiko/sftp_client.py", line 714, in putfo
with self.file(remotepath, "wb") as fr:
File "/usr/local/lib/python3.6/site-packages/paramiko/sftp_client.py", line 372, in open
t, msg = self._request(CMD_OPEN, filename, imode, attrblock)
File "/usr/local/lib/python3.6/site-packages/paramiko/sftp_client.py", line 813, in _request
return self._read_response(num)
File "/usr/local/lib/python3.6/site-packages/paramiko/sftp_client.py", line 865, in _read_response
self._convert_status(msg)
File "/usr/local/lib/python3.6/site-packages/paramiko/sftp_client.py", line 898, in _convert_status
raise IOError(text)
OSError: Failure
First, you should make sure the target directory /home/ is writable for you. Then you should review documentation for the put method. It says this about the second argument (remotepath):
The destination path on the SFTP server. Note that the filename should be included. Only specifying a directory may result in an error.
Try including the filename in the path, like:
...
ftp_client.put("/home/TrialFolder/HelloPython", "/home/HelloPython")
...
I need to create simple code that will upload a .csv file to an FTP server. The code is below.
import ftplib
import os
import sys
sourceFilePath = '/home/user/dir/'
filename = 'testing.csv'
destinationDirectory = 'anotherDirectory'
server = 'ftp://12.123.12.234'
username = 'aUser'
password = 'passwd1234'
myFTP = ftplib.FTP(server, username, password)
myFTP.cwd('/anotherDirectory/')
myFTP.storbinary('STOR '+filename, open(filename,'rb'))
myFTP.quit()
However, when I run the code, I get the following error:
Traceback (most recent call last):
File "./UploadToFTP.py", line 20, in <module>
File "./UploadToFTP.py", line 13, in uploadFileFTP
myFTP = ftplib.FTP(server, username, password)
File "/usr/lib64/python2.6/ftplib.py", line 119, in __init__
self.connect(host)
File "/usr/lib64/python2.6/ftplib.py", line 134, in connect
self.sock = socket.create_connection((self.host, self.port),
self.timeout)
File "/usr/lib64/python2.6/socket.py", line 553, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno -3] Temporary failure in name resolution
Has anyone seen this before? It seems rather generic to me and doesn't tell me much. So far as other code I've seen performing this same task, I don't have any currently visible errors. Any help would be appreciated.
The host argument of ftplib.FTP constructor is a hostname/IP address, not a URL.
So this is wrong:
server = 'ftp://12.123.12.234'
It should be:
server = '12.123.12.234'
If your URL contains a custom port, see Python ftplib - specify port.
I tried to find some topic related, found some, tried some suggested solution, but seems not to work.
I'm trying to figure out where I did it wrong, I'm basically listing a directory and putting the output into a text file that I want to send via smtp to dest address.
#!/usr/bin/python
import os, sys
import smtplib
from email.mime.text import MIMEText
# GRABE LISTED DIRECTORY AND CREATE THE FILE #
#open folder and list it
Winpath = "C:\"
dirs = os.listdir(Winpath)
#Put it into file
fo = open("activitygraber.txt", "w+")
for file in dirs:
fo.write(file + "\n")
# EMAIL IT TO GIVEN ADDRESS #
msg = MIMEText(fo.read())
fo.close()
msg["Subject"] = "Test ActivityGrab"
msg["From"] = "test#test.me"
msg["To"] = "dest#dest.com"
s = smtplib.SMTP("localhost")
s.sendmail("test#test.me", "dest#dest.com", msg.as_string())
s.quit()
That's simple code mostly from Python documentation (I'm in the learning curve).
The file is being created and ready to send I guess, but somehow the connection is not being created. I guess it's because my smtp port in not open ? should I tried to connect to a mail service ? then send it from there ?
Here the message I get when I try to send it:
Traceback (most recent call last):
File "C:\Users\PC\Desktop\testfile.py", line 33, in <module>
s = smtplib.SMTP("localhost")
File "C:\Python27\lib\smtplib.py", line 250, in __init__
(code, msg) = self.connect(host, port)
File "C:\Python27\lib\smtplib.py", line 310, in connect
self.sock = self._get_socket(host, port, self.timeout)
File "C:\Python27\lib\smtplib.py", line 285, in _get_socket
return socket.create_connection((host, port), timeout)
File "C:\Python27\lib\socket.py", line 571, in create_connection
raise err
socket.error: [Errno 10061] No connection could be made because the target machine actively refused it
Any help of hint would really be appreciated thanks !
Meanwhile I'm gonna continue to search the cause.
Cheers,
s = smtplib.SMTP("localhost") isn't going to work unless you have a running SMTP server on your local machine (which you probably don't...).
If you do, make sure it's listening on port 25.
If you don't, you will have to use a remote SMTP server. You can use google's if you have a gmail account, or find a free one that is set up as a mail relay (that means you can use it anonymously), but these are very rare these days.
I created a python FTP client and a server. I used python pftpdlib for the server and ftplib for client. I was able to start the server successfully and connect to the server through the python client. But, when I need to store a file in the server, I get following error.
File "/usr/lib/python2.7/ftplib.py", line 471, in storbinary
conn = self.transfercmd(cmd, rest)
File "/usr/lib/python2.7/ftplib.py", line 376, in transfercmd
return self.ntransfercmd(cmd, rest)[0]
File "/usr/lib/python2.7/ftplib.py", line 339, in ntransfercmd
resp = self.sendcmd(cmd)
File "/usr/lib/python2.7/ftplib.py", line 249, in sendcmd
return self.getresp()
File "/usr/lib/python2.7/ftplib.py", line 224, in getresp
raise error_perm, resp
ftplib.error_perm: 550 Not enough privileges.
Following is the source of my server
from pyftpdlib.authorizers import DummyAuthorizer
from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
authorizer = DummyAuthorizer()
authorizer.add_anonymous("/home/deepal/anonymous")
handler = FTPHandler
handler.authorizer = authorizer
handler.banner = "Server Ready.."
address = ("",21)
server = FTPServer(address, handler)
server.max_cons = 10
server.serve_forever()
And following is the client.
from ftplib import FTP
hostname = sys.argv[2]
ftp = FTP(hostname)
ftp.login()
filepath = "/home/deepal/Desktop/infile"
localfile = open(filepath,"rb")
ftp.storbinary("STOR "+str(os.path.basename),localfile)
I need to transfer a file to the server anonymously. Can anybody help me pointing what is wrong with my code which gives above error? Isn't it possible to store files anonymously in this way?
Thank you.
I solved it myself. I need to give appropriate permissions to the anonymous user. Changing the line,
authorizer.add_anonymous("/home/deepal/anonymous")
to
authorizer.add_anonymous("/home/deepal/anonymous", perm='elradfmwM')
worked.
I am currently using the one-line Twisted FTP server to transfer files back and forth between machines:
twistd -n ftp
Which works fine for downloading files from the server. However when I try to write to the server using:
with open('testFile.bmp', 'rb') as f:
ftp.storbinary('STOR ' + 'testFile.bmp', f)
with open('surrogate.py', 'rb') as f:
ftp.storbinary('STOR ' + 'surrogateCode.py', f)
I get errors:
Traceback (most recent call last):
File "client.py", line 13, in <module>
ftp.storbinary('STOR ' + 'testFile.bmp', f)
File "/usr/lib/python2.7/ftplib.py", line 461, in storbinary
conn = self.transfercmd(cmd, rest)
File "/usr/lib/python2.7/ftplib.py", line 368, in transfercmd
return self.ntransfercmd(cmd, rest)[0]
File "/usr/lib/python2.7/ftplib.py", line 331, in ntransfercmd
resp = self.sendcmd(cmd)
File "/usr/lib/python2.7/ftplib.py", line 244, in sendcmd
return self.getresp()
File "/usr/lib/python2.7/ftplib.py", line 219, in getresp
raise error_perm, resp
ftplib.error_perm: 550 Requested action not taken: internal server error
I tried it with the WinSCP FTP client and receive this error:
Copying files to remote side failed.
Requested action not taken: internal server error
I'm not sure if I am writing incorrectly or calling the server incorrectly.
Your code looks okay and from your description of the problem (encountering it in both WinSCP and the twisted library) I would hazard a guess that the issue is on the server side.
Using http://en.wikipedia.org/wiki/List_of_FTP_server_return_codes as a reference
Error 550 and the error ftplib.error_perm would suggest that perhaps the user you au don't have write permissions to that location