I am currently using the one-line Twisted FTP server to transfer files back and forth between machines:
twistd -n ftp
Which works fine for downloading files from the server. However when I try to write to the server using:
with open('testFile.bmp', 'rb') as f:
ftp.storbinary('STOR ' + 'testFile.bmp', f)
with open('surrogate.py', 'rb') as f:
ftp.storbinary('STOR ' + 'surrogateCode.py', f)
I get errors:
Traceback (most recent call last):
File "client.py", line 13, in <module>
ftp.storbinary('STOR ' + 'testFile.bmp', f)
File "/usr/lib/python2.7/ftplib.py", line 461, in storbinary
conn = self.transfercmd(cmd, rest)
File "/usr/lib/python2.7/ftplib.py", line 368, in transfercmd
return self.ntransfercmd(cmd, rest)[0]
File "/usr/lib/python2.7/ftplib.py", line 331, in ntransfercmd
resp = self.sendcmd(cmd)
File "/usr/lib/python2.7/ftplib.py", line 244, in sendcmd
return self.getresp()
File "/usr/lib/python2.7/ftplib.py", line 219, in getresp
raise error_perm, resp
ftplib.error_perm: 550 Requested action not taken: internal server error
I tried it with the WinSCP FTP client and receive this error:
Copying files to remote side failed.
Requested action not taken: internal server error
I'm not sure if I am writing incorrectly or calling the server incorrectly.
Your code looks okay and from your description of the problem (encountering it in both WinSCP and the twisted library) I would hazard a guess that the issue is on the server side.
Using http://en.wikipedia.org/wiki/List_of_FTP_server_return_codes as a reference
Error 550 and the error ftplib.error_perm would suggest that perhaps the user you au don't have write permissions to that location
Related
I am trying to send a file from the master node to minion nodes using a python script but a single error OSError: Failure keeps on coming up.
I tried to code this file to send this file from one local machine to another local machine.
My code:
#! /usr/bin/python
#! /usr/bin/python3
import paramiko
import os
#Defining working connect
def workon(host):
#Making a connection
ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy()) #To add the missing host key and auto add policy
ssh_client.connect(hostname = host, username = 'username', password = 'password')
ftp_client = ssh_client.open_sftp()
ftp_client.put("/home/TrialFolder/HelloPython", "/home/")
ftp_client.close()
#stdin, stdout, stderr = ssh_client.exec_command("ls")
#lines = stdout.readlines()
#print(lines)
def main():
hosts = ['192.16.15.32', '192.16.15.33', '192.16.15.34']
threads = []
for h in hosts:
workon(h)
main()
Error:
Traceback (most recent call last):
File "PythonMultipleConnectionUsinhSSH.py", line 28, in <module>
main()
File "PythonMultipleConnectionUsinhSSH.py", line 26, in main
workon(h)
File "PythonMultipleConnectionUsinhSSH.py", line 15, in workon
ftp_client.put("/home/Sahil/HelloPython", "/home/")
File "/usr/local/lib/python3.6/site-packages/paramiko/sftp_client.py", line 759, in put
return self.putfo(fl, remotepath, file_size, callback, confirm)
File "/usr/local/lib/python3.6/site-packages/paramiko/sftp_client.py", line 714, in putfo
with self.file(remotepath, "wb") as fr:
File "/usr/local/lib/python3.6/site-packages/paramiko/sftp_client.py", line 372, in open
t, msg = self._request(CMD_OPEN, filename, imode, attrblock)
File "/usr/local/lib/python3.6/site-packages/paramiko/sftp_client.py", line 813, in _request
return self._read_response(num)
File "/usr/local/lib/python3.6/site-packages/paramiko/sftp_client.py", line 865, in _read_response
self._convert_status(msg)
File "/usr/local/lib/python3.6/site-packages/paramiko/sftp_client.py", line 898, in _convert_status
raise IOError(text)
OSError: Failure
First, you should make sure the target directory /home/ is writable for you. Then you should review documentation for the put method. It says this about the second argument (remotepath):
The destination path on the SFTP server. Note that the filename should be included. Only specifying a directory may result in an error.
Try including the filename in the path, like:
...
ftp_client.put("/home/TrialFolder/HelloPython", "/home/HelloPython")
...
I want to connect to an FTPS server containing some not trusted certificate. When I use simple:
lftp -u user hostname
then after dir command there's an error:
ls: Fatal error: Certificate verification: Not trusted
The problem can be solved in lftp by executing the following command:
lftp -e "set ssl:verify-certificate false" -u user hostname
I'm trying to make the same connection in Python, using for example ftplib module:
import ftplib
ftp = ftplib.FTP_TLS()
ftp.connect(hostname, port)
ftp.login(username, password)
ftp.prot_p()
ftp.dir()
But it raises OSError exception:
Traceback (most recent call last):
File "/usr/lib/python3.8/code.py", line 90, in runcode
exec(code, self.locals)
File "<console>", line 1, in <module>
File "/usr/lib/python3.8/ftplib.py", line 558, in dir
self.retrlines(cmd, func)
File "/usr/lib/python3.8/ftplib.py", line 451, in retrlines
with self.transfercmd(cmd) as conn, \
File "/usr/lib/python3.8/ftplib.py", line 382, in transfercmd
return self.ntransfercmd(cmd, rest)[0]
File "/usr/lib/python3.8/ftplib.py", line 783, in ntransfercmd
conn = self.context.wrap_socket(conn,
File "/usr/lib/python3.8/ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "/usr/lib/python3.8/ssl.py", line 1040, in _create
self.do_handshake()
File "/usr/lib/python3.8/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
OSError: [Errno 0] Error
The problem seems to be similar to te OSError during authenticating to an ftps server with ftplib.FTP_TLS so I also tried to use some other context, like:
import ssl
ctx = ssl._create_stdlib_context(ssl.PROTOCOL_TLSv1_2)
ftp = FTP_TLS(context=ctx)
or
ctx = ssl.ssl._create_unverified_context(ssl.PROTOCOL_TLSv1_2)
ftp = FTP_TLS(context=ctx)
But the error is still the same. Any ideas how to disable certificate verification?
It cannot be a certificate problem, as you are getting error only at dir. The connect succeeds.
You get a TLS error when opening FTP data connection. It quite possible that the root cause is that the server require TLS session resumption.
See FTPS with Python ftplib - Session reuse required.
I'm creating an csv file and uploading it to a ftp server.
So the upload includes paths of multiple companies on that ftp server.
like :
COMANY1/FOO/BAR
But for some companies at the end I get this traceback:
Traceback (most recent call last):
File "exporter.py", line 117, in <module>
Exporter().run()
File "exporter.py", line 115, in run
self.upload(file, vendor['ftp_path'], filename)
File "exporter.py", line 78, in upload
sftp.chdir(dir)
File "/home/johndoe/exports/daily/venv/local/lib/python2.7/site-packages/paramiko/sftp_client.py", line 580, in chdir
if not stat.S_ISDIR(self.stat(path).st_mode):
File "/home/johndoe/exports/daily/venv/local/lib/python2.7/site-packages/paramiko/sftp_client.py", line 413, in stat
t, msg = self._request(CMD_STAT, path)
File "/home/johndoe/exports/daily/venv/local/lib/python2.7/site-packages/paramiko/sftp_client.py", line 730, in _request
return self._read_response(num)
File "/home/johndoe/exports/daily/venv/local/lib/python2.7/site-packages/paramiko/sftp_client.py", line 781, in _read_response
self._convert_status(msg)
File "/home/johndoe/exports/daily/venv/local/lib/python2.7/site-packages/paramiko/sftp_client.py", line 807, in _convert_status
raise IOError(errno.ENOENT, text)
IOError: [Errno 2] The requested file does not exist
I think the stack trace refers to the ftp path, that it doesn't exists (but It does exist).
But when I try to run that script on those exact same paths alone (that are causing the problem) it passes?
So, its not a logical error, and those ftp paths do exist - but can it be due to some timeout occurring?
Thanks,
Tom
Update:
I call the method like this:
self.upload(file, vendor['ftp_path'], filename)
And the actual method that does the uploading:
def upload(self, buffer, ftp_dir, filename):
for dir in ftp_dir.split('/'):
if dir == '':
continue
sftp.chdir(dir)
with sftp.open(filename, 'w') as f:
f.write(buffer)
f.close()
sftp.close()
I am writing to the home directory of the user I'm FTPing into, so permissions shouldn't be an issue. FTP works in FileZilla.
I checked the vsftp.conf and made the local_enable=YES change
On a Debian4 system with Python 2.4.4 (I can't upgrade it), I am using this code with ftplib
>>> f = ftplib.FTP('address', 'user', 'password')
>>> f.cwd('/home/user/some/dir/')
'250 Directory successfully changed.'
>>> myfile = '/full/path/of/file.txt'
>>> o = open(myfile, 'rb')
>>> f.storbinary('STOR ' + myfile, o)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/lib/python2.4/ftplib.py", line 415, in storbinary
conn = self.transfercmd(cmd)
File "/usr/lib/python2.4/ftplib.py", line 345, in transfercmd
return self.ntransfercmd(cmd, rest)[0]
File "/usr/lib/python2.4/ftplib.py", line 327, in ntransfercmd
resp = self.sendcmd(cmd)
File "/usr/lib/python2.4/ftplib.py", line 241, in sendcmd
return self.getresp()
File "/usr/lib/python2.4/ftplib.py", line 216, in getresp
raise error_perm, resp
ftplib.error_perm: 553 Could not create file.
Any ideas why it fails?
You are not writing to a home directory, you are writing to /full/path/of/file.txt:
myfile = '/full/path/of/file.txt'
...
f.storbinary('STOR ' + myfile, o)
You have to use a file name only with the STOR command (once the "cwd" is already the correct target path):
f.cwd('/home/user/some/dir/')
f.storbinary('STOR file.txt', o)
or a correct absolute path for the remote host:
f.storbinary('STOR /home/user/some/dir/file.txt', o)
I have a Python script that downloads product feeds from multiple affiliates in different ways. This didn't give me any problems until last Wednesday, when it started throwing all kinds of timeout exceptions from different locations.
Examples: Here I connect with a FTP service:
ftp = FTP(host=self.host)
threw:
Exception in thread Thread-7:
Traceback (most recent call last):
File "C:\Python27\Lib\threading.py", line 810, in __bootstrap_inner
self.run()
File "C:\Python27\Lib\threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "C:\Users\Administrator\Documents\Crawler\src\Crawlers\LDLC.py", line 23, in main
ftp = FTP(host=self.host)
File "C:\Python27\Lib\ftplib.py", line 120, in __init__
self.connect(host)
File "C:\Python27\Lib\ftplib.py", line 138, in connect
self.welcome = self.getresp()
File "C:\Python27\Lib\ftplib.py", line 215, in getresp
resp = self.getmultiline()
File "C:\Python27\Lib\ftplib.py", line 201, in getmultiline
line = self.getline()
File "C:\Python27\Lib\ftplib.py", line 186, in getline
line = self.file.readline(self.maxline + 1)
File "C:\Python27\Lib\socket.py", line 476, in readline
data = self._sock.recv(self._rbufsize)
timeout: timed out
Or downloading an XML File :
xmlFile = urllib.URLopener()
xmlFile.retrieve(url, self.feedPath + affiliate + "/" + website + '.' + fileType)
xmlFile.close()
throws:
File "C:\Users\Administrator\Documents\Crawler\src\Crawlers\FeedCrawler.py", line 106, in save
xmlFile.retrieve(url, self.feedPath + affiliate + "/" + website + '.' + fileType)
File "C:\Python27\Lib\urllib.py", line 240, in retrieve
fp = self.open(url, data)
File "C:\Python27\Lib\urllib.py", line 208, in open
return getattr(self, name)(url)
File "C:\Python27\Lib\urllib.py", line 346, in open_http
errcode, errmsg, headers = h.getreply()
File "C:\Python27\Lib\httplib.py", line 1139, in getreply
response = self._conn.getresponse()
File "C:\Python27\Lib\httplib.py", line 1067, in getresponse
response.begin()
File "C:\Python27\Lib\httplib.py", line 409, in begin
version, status, reason = self._read_status()
File "C:\Python27\Lib\httplib.py", line 365, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "C:\Python27\Lib\socket.py", line 476, in readline
data = self._sock.recv(self._rbufsize)
IOError: [Errno socket error] timed out
These are just two examples but there are other methods, like authenticate or other API specific methods where my script throws these timeout errors. It never showed this behavior until Wednesday. Also, it starts throwing them at random times. Sometimes at the beginning of the crawl, sometimes later on. My script has this behavior on both my server and my local machine. I've been struggling with it for two days now but can't seem to figure it out.
This is what I know might have caused this:
On Wednesday one affiliate script broke down with the following error:
URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)>
I didn't change anything to my script but suddenly it stopped crawling that affiliate and threw that error all the time where I tried to authenticate. I looked it up and found that is was due to an OpenSSL error (where did that come from). I fixed it by adding the following before the authenticate method:
if hasattr(ssl, '_create_unverified_context'):
ssl._create_default_https_context = ssl._create_unverified_context
Little did I know, this was just the start of my problems... At that same time, I changed from Python 2.7.8 to Python 2.7.9. It seems that this is the moment that everything broke down and started throwing timeouts.
I tried changing my script in endless ways but nothing worked and like I said, it's not just one method that throws it. Also I switched back to Python 2.7.8, but this didn't do the trick either. Basically everything that makes a request to an external source can throw an error.
Final note: My script is multi threaded. It downloads product feeds from different affiliates at the same time. It used to run 10 threads per affiliate without a problem. Now I tried lowering it to 3 per affiliate, but it still throws these errors. Setting it to 1 is no option because that will take ages. I don't think that's the problem anyway because it used to work fine.
What could be wrong?