I’m having a hard time with this simple script. It’s giving me an error of file or directory not found but the file is there. Script below I’ve masked user and pass plus FTP site
Here is my script
from ftplib import FTP
ftp = FTP('ftp.domain.ca')
pas = str('PASSWORD')
ftp.login(user = 'user', passwd=pas)
ftp.cwd('/public_html/')
filepaths = open('errorstest.csv', 'rb')
for j in filepaths:
print(j)
ftp.delete(str(j))
ftp.quit()
The funny thing tho is if I slight change the script to have ftp.delete() it finds the file and deletes it. So modified to be like this:
from ftplib import FTP
ftp = FTP('ftp.domain.ca')
pas = str('PASSWORD')
ftp.login(user = 'user', passwd=pas)
ftp.cwd('/public_html/')
ftp.delete(<file path>)
ftp.quit()
I’m trying to read this from a csv file. What am I doing wrong?
Whatever you have showed seems to be fine. But could you try this?
from ftplib import FTP
ftp = FTP(host)
ftp.login(username, password)
ftp.cwd('/public_html/')
print(ftp.pwd())
print(ftp.nlst())
with open('errorstest.csv') as file:
for line in file:
if line.strip():
ftp.delete(line.strip())
print(ftp.nlst())
Related
I am trying to figure out how to retrieve metadata from an FTP folder and all sub folders. I want to get the file name, file size, and date/time (of when the file was modified). I found the sample code (below) online. I entered in my credentials, ran the code, and received this error: No hostkey for host ftp.barra.com found.
Is there a quick fix for this?
from __future__ import print_function
import os
import time
import pysftp
ftp_username='xxx'
ftp_password='xxx'
ftp_host='xxx'
year = time.strftime("%Y")
month = time.strftime("%m")
day = time.strftime("%d")
ftp_dir = 'data/'+year+'/'+month
filename = time.strftime('ftp_file_lists.txt')
fout = open(filename, 'w')
wtcb = pysftp.WTCallbacks()
with pysftp.Connection(ftp_host, username=ftp_username, password=ftp_password) as sftp:
sftp.walktree(ftp_dir, fcallback=wtcb.file_cb, dcallback=wtcb.dir_cb, ucallback=wtcb.unk_cb)
print(len(wtcb.flist))
for fpath in wtcb.flist:
print(fpath, file=fout)
sftp.close()
Code from here.
http://alvincjin.blogspot.com/2014/09/recursively-fetch-file-paths-from-ftp.html
I have a very simple script that uses urllib to retrieve a zip file and place it on my desktop. The zip file is only a couple MB in size and doesn't take long to download. However, the script doesn't seem to finish, it just hangs. Is there a way to forcibly close the urlretrieve?...or a better solution?
The URL is to a public ftp size. Is the ftp perhaps the cause?
I'm using python 2.7.8.
url = r'ftp://ftp.ngs.noaa.gov/pub/DS_ARCHIVE/ShapeFiles/IA.ZIP'
zip_path = r'C:\Users\***\Desktop\ngs.zip'
urllib.urlretrieve(url, zip_path)
Thanks in advance!
---Edit---
Was able to use ftplib to accomplish the task...
import os
from ftplib import FTP
import zipfile
ftp_site = 'ftp.ngs.noaa.gov'
ftp_file = 'IA.ZIP'
download_folder = '//folder to place file'
download_file = 'name of file'
download_path = os.path.join(download_folder, download_file)
# Download file from ftp
ftp = FTP(ftp_site)
ftp.login()
ftp.cwd('pub/DS_ARCHIVE/ShapeFiles') #change directory
ftp.retrlines('LIST') #show me the files located in directory
download = open(download_path, 'wb')
ftp.retrbinary('RETR ' + ftp_file, download.write)
ftp.quit()
download.close()
# Unzip if .zip file is downloaded
with zipfile.ZipFile(download_path, "r") as z:
z.extractall(download_folder)
urllib has a very bad support for error catching and debugging. urllib2 is a much better choice. The urlretrieve equivalent in urllib2 is:
resp = urllib2.urlopen(im_url)
with open(sav_name, 'wb') as f:
f.write(resp.read())
And the errors to catch are:
urllib2.URLError, urllib2.HTTPError, httplib.HTTPException
And you can also catch socket.error in case that the network is down.
You can use python requests library with requests-ftp module. It provides easier API and better processes exceptions. See: https://pypi.python.org/pypi/requests-ftp and http://docs.python-requests.org/en/latest/
I'm trying to retrieve a zip folder(s) from an ftp site and save them to my local machine, using python (ideally I'd like to specify where they are saved on my C:).
The code below connects to the FTP site and then *something happens in the PyScripter window that looks like random characters for about 1000 lines... but nothing actually gets downloaded to my hard drive.
Any tips?
import ftplib
import sys
def gettext(ftp, filename, outfile=None):
# fetch a text file
if outfile is None:
outfile = sys.stdout
# use a lambda to add newlines to the lines read from the server
ftp.retrlines("RETR " + filename, lambda s, w=outfile.write: w(s+"\n"))
def getbinary(ftp, filename, outfile=None):
# fetch a binary file
if outfile is None:
outfile = sys.stdout
ftp.retrbinary("RETR " + filename, outfile.write)
ftp = ftplib.FTP("FTP IP Address")
ftp.login("username", "password")
ftp.cwd("/MCPA")
#gettext(ftp, "subbdy.zip")
getbinary(ftp, "subbdy.zip")
Well, it seems that you simply forgot to open the file you want to write into.
Something like:
getbinary(ftp, "subbdy.zip", open(r'C:\Path\to\subbdy.zip', 'wb'))
quick and simple:
I have the following function, works well if i specify the file name.
import os
import ftplib
def ftpcon(self, host, port, login, pwd, path):
ftp = ftplib.FTP()
ftp.connect(host, port, 20)
try:
ftp.login(login, pwd)
ftp.cwd(path)
for files in ftp.nlst():
if files.endswith('.doc') or files.endswith('.DOC'):
ftp.retrbinary('RETR ' + files, open(file, 'wb').write)
print files
But when i use the for loop with ftp.nlst() to try to match an specific type of file, i receive the error:
coercing to Unicode: need string or buffer, type found
Since im not sure if this is the best way to do it, what could the "correct" way to download a file ?
Maybe try:
from ftplib import FTP
server = FTP("ip/serveradress")
server.login("user", "password")
server.retrlines("LIST") # Will show a FTP content list.
server.cwd("Name_Of_Folder_in_FTP_to_browse_to") # browse to folder containing your file for DL
then:
server.sendcmd("TYPE i") # ready for file transfer
server.retrbinary("RETR %s"%("FILENAME+EXT to DL"), open("DESTINATIONPATH+EXT", "wb").write) # this will transfer the selected file...to selected path/file
believe this is as correct as serves..
u can set server.set_debuglevel(0) to (1) or (2) for more detailed description while logged in to server.
I am able to download files from the FTP using the ftplib in Python, but this is like I hard code the name the name of the file(R.pdf) and this downloads only (R.pdf), is there a way to download all files in the FTP with the extension .PDF to my local system using Python. I am able to do this in Shell by just using *.pdf
Replace host, user and password with your credentials,
and 'public_html/soleil' with the address of the directory in which are the PDF files you want to be downloaded,
in the following code and it should be OK I think.
from ftplib import *
from os import listdir
from os.path import getsize
ftp_dt = FTP(host,user,password)
ftp_pi = FTP(host,user,password)
print '\n- Ouverture de connection et logging : OK'
ftp_dt.cwd('public_html/soleil')
ftp_pi.cwd('public_html/soleil')
def func(content, li = [0], la = [], si = [0], memname = ['']):
if name!=memname[0]:
memname[0],li[0:1],la[:],si[0:1] = name,[0],[],[0]
li[0] = li[0] + 1
si[0] = si[0] + len(content)
la.append(str(len(content)))
if li[0]%8==0:
print ' '.join(la) +\
' total: '+str(li[0])+' chunks, '+str(si[0])+' bytes'
la[:] = []
f.write(content)
li_files = []
for name in ftp_dt.nlst():
try:
ftp_dt.size(name)
if name not in ('.','..') and name[-4:]=='.pdf':
li_files.append(name)
except:
pass
if li_files:
for name in li_files:
print '\n- Downloading '+name
with open('E:\\PDF\\DOWNS\\'+name,'wb') as f:
ftp_pi.retrbinary('RETR '+name,func)
if getsize('E:\\PDF\\DOWNS\\'+name)==ftp_dt.size(name):
print ' OK ! Download of complete '+repr(name)+' SUCCEEDED'
else:
print ' FAILURE !! : '+name+' only partially downloaded'
else:
print '\nThere is no PDF file in this FTP directory'
ftp_dt.quit()
ftp_pi.quit()
Two connexions ftp_dt and ftp_pi are defined for “Data Transfers“ and “Protocol Interpretation“ because FTP protocol is based on two channels, one for the commands and the other for..... guess what ?
The func() function is used as callback in the fonction retrbinary()
It could be just
def func(content):
f.write()
but I played a bit with the possibilities of default variables of a function.
One thing I don’t understand well: how can this code work while the reference f in func() is only defined in the text of code after the definition of func() . But I tested it and it works !
I don't have access to an FTP server I can try this but a cursory look at the documentation indicates that this is not possible.
You can, however, obtain a list of files on the remote end with the dir or nlst commands and then fetch each file in a loop.
use two python modules glob and wget. Your snippet could look like this
import glob
import wget
list_to_download = glob.glob(url+'*.pdf')
for file in list_to_download:
wget.download(file)