Develop Reference

Python : Downloaded tar.gz files over FTP cannot be opened

I'd like to download tar.gz files over FTP. It uses FTP_TLS.
Files are downloaded but when I try to open them on Windows (I use 7zip), it's not working.
Error is :
"File [...] cannot be opened as an archive"
This is my code (It needs improvment I know I'm quite newbee :) ):
def get_ftp(ip, login, passwd, path):
""" Connexion FTP """
try:
with ftplib.FTP_TLS(ip, login, passwd) as ftps:
ftps.prot_p()
# ftp.login(login, passwd)
files = ftps.nlst('/home/user/dir/' + path)
# ftp.retrlines('LIST')
if files:
for file in files:
if file.endswith('.tar.gz'):
if file + '.md5' in files:
localfile = join(path_recu, basename(file))
with open(localfile, 'wb') as binary_file:
response = ftps.retrbinary('RETR %s' % file, binary_file.write, blocksize=8192, rest=None)
if response.startswith('226'):
with open(localfile, 'w') as text_file:
ftps.retrlines('RETR %s' % file + '.md5', text_file.write)
except ftplib.error_perm as resp:
if str(resp):
logger.critical('ERREUR : ' + repr(resp))
raise
else:
return files
I've tried with "blocksize=4096": same error.
Any ideas ?

Python ftplib add new line to txt file during upload file to FTP server

I use following code to upload txt files in ASCII mode to FTP server
import glob
import os
import hashlib
from ftplib import FTP
server = '1.1.1.1'
login = 'user'
password = 'password'
path = './test_files/'
file_mask = '*.txt'
def upload_to_ftp(srv, uname, pwd, file_name):
ftp = FTP(srv, uname, pwd)
ftp.cwd('Pava')
file = open(path+file_name, 'rb')
ftp.storlines('STOR '+file_name, file)
size = ftp.size(file_name)
ftp.close()
file.close()
print (size)
def local_size_check(file_name):
file_size = os.stat(path+file_name)
print (file_size.st_size)
file_to_upload = glob.glob1(path, file_mask)
for i in file_to_upload:
try:
os.rename(path+i, path+i)
except OSError as e:
print ('Access-error on file ' + i + ' ! \n' + str(e))
else:
upload_to_ftp(server, login, password, i)
local_size_check(i)
The output of this two functions is:
78
76
Then i have dowloaded file from ftp and found that during transfering by FTP was added new line at the end of file.
local and remote file screens
Please help to solve this problem.
BTW if use binary mode new line do not add

You should upload your file in binary mode so that it will not be subject to the server's text interpretation.
Change:
ftp.storlines('STOR '+file_name, file)
to:
ftp.storbinary('STOR '+file_name, file)

Problems downloading files from ftp server to directory

I am trying to download files from an ftp server to a local folder. Below I have included my code, the script is able to access the ftp succesfully and list the names of the files on the ftp server however where my issue lies is I am not sure how to download them to a local directory.
from ftplib import FTP
import os, sys, os.path
def handleDownload(block):
file.write(block)
ddir='U:/Test Folder'
os.chdir(ddir)
ftp = FTP('sidads.colorado.edu')
ftp.login()
print ('Logging in.')
directory = '/pub/DATASETS/NOAA/G02158/unmasked/2012/04_Apr/'
print ('Changing to ' + directory)
ftp.cwd(directory)
ftp.retrlines('LIST')
print ('Accessing files')
filenames = ftp.nlst() # get filenames within the directory
print (filenames)
I tried using the following code below however it gives me the error in the photo below.
for filename in filenames:
if filename != '.':
local_filename = os.path.join('U:/Test Folder/', filename)
file = open(local_filename, 'wb')
ftp.retrbinary('RETR '+ filename, file.write)
file.close()
ftp.quit()

Try the following:
for filename in filenames:
if filename not in ['.', '..']:
local_filename = os.path.join(ddir, filename)
with open(local_filename, 'wb') as f_output:
ftp.retrbinary('RETR '+ filename, f_output.write)
ftp.quit()
Your code was still attempting to download . filenames. You need to indent the writing of the valid filenames. Also by using with it will close the file automatically.

trying to transfer a directory using paramiko

when trying to transfer the file like .txt .xml etc. But when transferring whole directory to another it is giving error.
let me know what is wrong with the code. why it is not transferring the directories
import paramiko
import os
paramiko.util.log_to_file('/tmp/paramiko.log')
host = '10.2.216.195'
t = paramiko.Transport((host))
t.connect(username = 'stack',password = 'flow')
sftp = paramiko.SFTPClient.from_transport(t)
filepath = '/home/proteek'
localpath = '/home/proteek/newfolder/movie'
sftp.get(localpath,filepath)
sftp.close()
t.close()
it is giving an error which is
File "/usr/lib/python2.6/site-packages/paramiko/sftp_client.py", line 599, in get
fl = file(localpath, 'wb')
IOError: [Errno 21] Is a directory: '/home/proteek/movie'

os.renames for ftp in python

I want to move a large number of files from a windows system to a unix ftp server using python. I have a csv which has the current full path and filename and the new base bath to send it to (see here for an example dataset).
I have got a script using os.renames to do the transfer and directory creation in windows but can figure out a way to easily do it via ftp.
import os, glob, arcpy, csv, sys, shutil, datetime
top=os.getcwd()
RootOutput = top
startpath=top
FileList = csv.reader(open('FileList.csv'))
filecount=0
successcount=0
errorcount=0
# Copy/Move to FTP when required
ftp = ftplib.FTP('xxxxxx')
ftp.login('xxxx', 'xxxx')
directory = '/TransferredData'
ftp.cwd(directory)
##f = open(RootOutput+'\\Success_LOG.txt', 'a')
##f.write("Log of files Succesfully processed. RESULT of process run #:"+str(datetime.datetime.now())+"\n")
##f.close()
##
for File in FileList:
infile=File[0]
# local network ver
#outfile=RootOutput+File[4]
#os.renames(infile, outfile)
# ftp netowrk ver
# outfile=RootOutput+File[4]
# ftp.mkd(directory)
print infile, outfile
I tried the process in http://forums.arcgis.com/threads/17047-Upload-file-to-FTP-using-Python-ftplib but this is for moving all files in a directory, I have the old and new full file names and just need it to create the intermediate directories.
Thanks,

The following might work (untested):
def mkpath(ftp, path):
path = path.rsplit('/', 1)[0] # parent directory
if not path:
return
try:
ftp.cwd(path)
except ftplib.error_perm:
mkpath(ftp, path)
ftp.mkd(path)
ftp = FTP(...)
directory = '/TransferredData/'
for File in FileList:
infile = File[0]
outfile = File[4].split('\\') # need forward slashes in FTP
outfile = directory + '/'.join(outfile)
mkpath(ftp, outfile)
ftp.storbinary('STOR '+outfile, open(infile, 'rb'))