I am trying to download files from an ftp server to a local folder. Below I have included my code, the script is able to access the ftp succesfully and list the names of the files on the ftp server however where my issue lies is I am not sure how to download them to a local directory.
from ftplib import FTP
import os, sys, os.path
def handleDownload(block):
file.write(block)
ddir='U:/Test Folder'
os.chdir(ddir)
ftp = FTP('sidads.colorado.edu')
ftp.login()
print ('Logging in.')
directory = '/pub/DATASETS/NOAA/G02158/unmasked/2012/04_Apr/'
print ('Changing to ' + directory)
ftp.cwd(directory)
ftp.retrlines('LIST')
print ('Accessing files')
filenames = ftp.nlst() # get filenames within the directory
print (filenames)
I tried using the following code below however it gives me the error in the photo below.
for filename in filenames:
if filename != '.':
local_filename = os.path.join('U:/Test Folder/', filename)
file = open(local_filename, 'wb')
ftp.retrbinary('RETR '+ filename, file.write)
file.close()
ftp.quit()
Try the following:
for filename in filenames:
if filename not in ['.', '..']:
local_filename = os.path.join(ddir, filename)
with open(local_filename, 'wb') as f_output:
ftp.retrbinary('RETR '+ filename, f_output.write)
ftp.quit()
Your code was still attempting to download . filenames. You need to indent the writing of the valid filenames. Also by using with it will close the file automatically.
Related
I need to take PDF files from a clients ftp, move them onto my pc, move their uploaded files to a folder called 'transferred' and delete them from the previous directory.
my code is
ftp.cwd('/folder1/')
ftp.retrlines('LIST')
os.chdir('path')
datestr = time.strftime("%d-%m")
os.chdir(datestr)
os.chdir('Folder1')
filenames = ftp.nlst()
print(filenames)
for filename in filenames:
if filename.endswith(".pdf"):
local_filename = os.path.join(newdir, filename)
file = open(local_filename, 'wb')
ftp.retrbinary('RETR '+ filename, file.write)
remotefile = open(filename, 'rb')
ftp.storbinary('STOR ' + 'transferred/' + filename, remotefile)
file.close()
ftp.delete(filename)
when i do this, it copies the files over to my pc, moves them to the transferred folder. all fine. however, the PDFs are then corrupt on my pc and on the FTP server... why is this? is my code wrong?
Ok, so I have to upload a directory, with subdirectories and files inside, on a FTP server. But I can't seem to get it right. I want to upload the directory as it is, with it's subdirectories and files where they were.
ftp = FTP()
ftp.connect('host',port)
ftp.login('user','pass')
filenameCV = "directorypath"
def placeFiles():
for root,dirnames,filenames in os.walk(filenameCV):
for files in filenames:
print(files)
ftp.storbinary('STOR ' + files, open(files,'rb'))
ftp.quit()
placeFiles()
There are multiple problems with your code: First, the filenames array will only contain the actual filenames, not the entire path, so you need to join it with fullpath = os.path.join(root, files) and then use open(fullpath). Secondly, you quit the FTP connection inside the loop, move that ftp.quit() down on the level of the placeFiles() function.
To recursively upload your directory, you have to walk through your root directories and at the same time through your remote directory, uploading files on the go.
Full example code:
import os.path, os
from ftplib import FTP, error_perm
host = 'localhost'
port = 21
ftp = FTP()
ftp.connect(host,port)
ftp.login('user','pass')
filenameCV = "directorypath"
def placeFiles(ftp, path):
for name in os.listdir(path):
localpath = os.path.join(path, name)
if os.path.isfile(localpath):
print("STOR", name, localpath)
ftp.storbinary('STOR ' + name, open(localpath,'rb'))
elif os.path.isdir(localpath):
print("MKD", name)
try:
ftp.mkd(name)
# ignore "directory already exists"
except error_perm as e:
if not e.args[0].startswith('550'):
raise
print("CWD", name)
ftp.cwd(name)
placeFiles(ftp, localpath)
print("CWD", "..")
ftp.cwd("..")
placeFiles(ftp, filenameCV)
ftp.quit()
I am newbie to python and learning to delete the file which are downloaded from ftp site.
this is my error :OSError: [Errno 2] No such file or directory: 'RingGoData-2014-07-02.csv'
this is my code:
ftp = ftplib.FTP('192.198.0.20', 'bingo', 'Password')
files = ftp.dir('/')
ftp.cwd("/")
#ftp.retrlines('LIST')
filematch = '*.csv'
target_dir = '/home/toor/ringolist'
import os
for filename in ftp.nlst(filematch):
target_file_name = os.path.join(target_dir,os.path.basename(filename))
with open(target_file_name,'wb') as fhandle:
ftp.retrbinary('RETR %s' % filename, fhandle.write)
if os.path.isdir(filename)== True:
shutil.rmtree(filename)
else:
os.remove(filename)
Shutil and os.remove work on your current filesystem, NOT on the FTP server. Your trying to delete a local file which isn't there.
You should use FTP.delete(filename)
I have a unique situation. I have a local zip file (C:\Temp\abc.zip). I want to extract this file in python to a remote drive (\Crprvau01n1\Cdv_prd$\DataDrop\Quartz\IMM\DevRuns). Please note that there is no drive letter. I have write permissions to this folder and I can access via windows explorer. I have this following code,
import zipfile, os
def main():
zfile = zipfile.ZipFile("\\Crprvau01n1\Cdv_prd$\DataDrop\Quartz\IMM\DevRuns\Zinc.zip", 'r')
for name in zfile.namelist():
(dirname, filename) = os.path.split(name)
print "Decompressing " + filename + " on " + dirname
filename = "C:/Temp/" + filename
fd = open(filename,"w")
fd.write(zfile.read(name))
fd.close()
I get the below error:
IOError: [Errno 2] No such file or directory: '\\Crprvau01n1\\Cdv_prd$\\DataDrop\\Quartz\\IMM\\DevRuns\\Zinc.zip'
Any suggestions on how to read the remote zip file is appreciated.
Thanks
Use a raw string r'...' (or double every backslash):
zipfile.ZipFile(r"\\Crprvau01n1\Cdv_prd$\DataDrop\Quartz\IMM\DevRuns\Zinc.zip", 'r')
Backslash is an escape character in normal strings.
I want to move a large number of files from a windows system to a unix ftp server using python. I have a csv which has the current full path and filename and the new base bath to send it to (see here for an example dataset).
I have got a script using os.renames to do the transfer and directory creation in windows but can figure out a way to easily do it via ftp.
import os, glob, arcpy, csv, sys, shutil, datetime
top=os.getcwd()
RootOutput = top
startpath=top
FileList = csv.reader(open('FileList.csv'))
filecount=0
successcount=0
errorcount=0
# Copy/Move to FTP when required
ftp = ftplib.FTP('xxxxxx')
ftp.login('xxxx', 'xxxx')
directory = '/TransferredData'
ftp.cwd(directory)
##f = open(RootOutput+'\\Success_LOG.txt', 'a')
##f.write("Log of files Succesfully processed. RESULT of process run #:"+str(datetime.datetime.now())+"\n")
##f.close()
##
for File in FileList:
infile=File[0]
# local network ver
#outfile=RootOutput+File[4]
#os.renames(infile, outfile)
# ftp netowrk ver
# outfile=RootOutput+File[4]
# ftp.mkd(directory)
print infile, outfile
I tried the process in http://forums.arcgis.com/threads/17047-Upload-file-to-FTP-using-Python-ftplib but this is for moving all files in a directory, I have the old and new full file names and just need it to create the intermediate directories.
Thanks,
The following might work (untested):
def mkpath(ftp, path):
path = path.rsplit('/', 1)[0] # parent directory
if not path:
return
try:
ftp.cwd(path)
except ftplib.error_perm:
mkpath(ftp, path)
ftp.mkd(path)
ftp = FTP(...)
directory = '/TransferredData/'
for File in FileList:
infile = File[0]
outfile = File[4].split('\\') # need forward slashes in FTP
outfile = directory + '/'.join(outfile)
mkpath(ftp, outfile)
ftp.storbinary('STOR '+outfile, open(infile, 'rb'))