Develop Reference

Python ftplib add new line to txt file during upload file to FTP server

I use following code to upload txt files in ASCII mode to FTP server
import glob
import os
import hashlib
from ftplib import FTP
server = '1.1.1.1'
login = 'user'
password = 'password'
path = './test_files/'
file_mask = '*.txt'
def upload_to_ftp(srv, uname, pwd, file_name):
ftp = FTP(srv, uname, pwd)
ftp.cwd('Pava')
file = open(path+file_name, 'rb')
ftp.storlines('STOR '+file_name, file)
size = ftp.size(file_name)
ftp.close()
file.close()
print (size)
def local_size_check(file_name):
file_size = os.stat(path+file_name)
print (file_size.st_size)
file_to_upload = glob.glob1(path, file_mask)
for i in file_to_upload:
try:
os.rename(path+i, path+i)
except OSError as e:
print ('Access-error on file ' + i + ' ! \n' + str(e))
else:
upload_to_ftp(server, login, password, i)
local_size_check(i)
The output of this two functions is:
78
76
Then i have dowloaded file from ftp and found that during transfering by FTP was added new line at the end of file.
local and remote file screens
Please help to solve this problem.
BTW if use binary mode new line do not add

You should upload your file in binary mode so that it will not be subject to the server's text interpretation.
Change:
ftp.storlines('STOR '+file_name, file)
to:
ftp.storbinary('STOR '+file_name, file)

Downloading files from SFTP server listed by remote command with Paramiko in Python

I am using Paramiko to connect to the SFTP server from my local machine and download txt files from remote path. I am able to make successful connection and can also print the remote path and the files but i cannot get the files locally. I can print the file_path and file_name but not able to download all the files. Below is the code I am using:
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname=hostname, username=username, password=password, port=port)
remotepath = '/home/blahblah'
pattern = '"*.txt"'
stdin,stdout,stderr = ssh.exec_command("find {remotepath} -name {pattern}".format(remotepath=remotepath, pattern=pattern))
ftp = ssh.open_sftp()
for file_path in stdout.readlines():
file_name = file_path.split('/')[-1]
print(file_path)
print(file_name)
ftp.get(file_path, "/home/mylocalpath/{file_name}".format(file_name=file_name))
I can see the file_path and file_name like below from print statement but get error while using ftp.get for multiple files. I can copy a single file by hardcoding the name on source and destination.
file_path = '/home/blahblah/abc.txt'
file_name = 'abc.txt'
file_path = '/home/blahblah/def.txt'
file_name = 'def.txt'
I see one file is downloaded and then i get the following error:
FileNotFoundErrorTraceback (most recent call last)
Error trace:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "...anaconda3/lib/python3.6/site-packages/paramiko/sftp_client.py", line 769, in get
with open(localpath, 'wb') as fl:
FileNotFoundError: [Errno 2] No such file or directory: 'localpath/abc.txt\n'

readlines does not remove newline from the line. So as you can see in the traceback, you are trying to create a file named abc.txt\n, what is not possible on many file systems, and mainly, it's not what you want.
Trim the trailing new lines from file_path:
for file_path in stdout.readlines():
file_path = file_path.rstrip()
file_name = file_path.split('/')[-1]
# ...
Though you would have saved yourself lot of troubles, had you used a pure SFTP solution, instead of hacking it by executing a remote find command (what is a very fragile solution, as hinted in comments by #CharlesDuffy).
See List files on SFTP server matching wildcard in Python using Paramiko.
Side note: Do not use AutoAddPolicy. You
lose security by doing so. See Paramiko "Unknown Server".

Python FTP filename invalid error

I'm trying to upload a file using ftp in python, but I get an error saying:
ftplib.error_perm: 550 Filename invalid
when I run the following code:
ftp = FTP('xxx.xxx.x.xxx', 'MY_FTP', '')
ftp.cwd("/incoming")
file = open('c:\Automation\FTP_Files\MaxErrors1.json', 'rb')
ftp.storbinary('STOR c:\Automation\FTP_Files\MaxErrors1.json', file)
ftp.close()
I've checked that the file exists in the location I specified, does anyone know what might be causing the issue?

The problem is that on the server, the path c:\Automation\FTP_Files\MaxErrors1.json is not valid. Instead try just doing:
ftp.storbinary('STOR MaxErrors1.json', file)

The argument to STOR needs to be the destination file name, not the source path. You should just do ftp.storbinary('STOR MaxErrors1.json', file).

you should upload file without absolute path in ftp server
for example :
import ftplib
session = ftplib.FTP('server.address.com','USERNAME','PASSWORD')
file = open('kitten.jpg','rb') # file to send
session.storbinary('STOR kitten.jpg', file) # send the file
file.close() # close file and FTP
session.quit()

Delete files from ftp after download

I am newbie to python and learning to delete the file which are downloaded from ftp site.
this is my error :OSError: [Errno 2] No such file or directory: 'RingGoData-2014-07-02.csv'
this is my code:
ftp = ftplib.FTP('192.198.0.20', 'bingo', 'Password')
files = ftp.dir('/')
ftp.cwd("/")
#ftp.retrlines('LIST')
filematch = '*.csv'
target_dir = '/home/toor/ringolist'
import os
for filename in ftp.nlst(filematch):
target_file_name = os.path.join(target_dir,os.path.basename(filename))
with open(target_file_name,'wb') as fhandle:
ftp.retrbinary('RETR %s' % filename, fhandle.write)
if os.path.isdir(filename)== True:
shutil.rmtree(filename)
else:
os.remove(filename)

Shutil and os.remove work on your current filesystem, NOT on the FTP server. Your trying to delete a local file which isn't there.
You should use FTP.delete(filename)

os.renames for ftp in python

I want to move a large number of files from a windows system to a unix ftp server using python. I have a csv which has the current full path and filename and the new base bath to send it to (see here for an example dataset).
I have got a script using os.renames to do the transfer and directory creation in windows but can figure out a way to easily do it via ftp.
import os, glob, arcpy, csv, sys, shutil, datetime
top=os.getcwd()
RootOutput = top
startpath=top
FileList = csv.reader(open('FileList.csv'))
filecount=0
successcount=0
errorcount=0
# Copy/Move to FTP when required
ftp = ftplib.FTP('xxxxxx')
ftp.login('xxxx', 'xxxx')
directory = '/TransferredData'
ftp.cwd(directory)
##f = open(RootOutput+'\\Success_LOG.txt', 'a')
##f.write("Log of files Succesfully processed. RESULT of process run #:"+str(datetime.datetime.now())+"\n")
##f.close()
##
for File in FileList:
infile=File[0]
# local network ver
#outfile=RootOutput+File[4]
#os.renames(infile, outfile)
# ftp netowrk ver
# outfile=RootOutput+File[4]
# ftp.mkd(directory)
print infile, outfile
I tried the process in http://forums.arcgis.com/threads/17047-Upload-file-to-FTP-using-Python-ftplib but this is for moving all files in a directory, I have the old and new full file names and just need it to create the intermediate directories.
Thanks,

The following might work (untested):
def mkpath(ftp, path):
path = path.rsplit('/', 1)[0] # parent directory
if not path:
return
try:
ftp.cwd(path)
except ftplib.error_perm:
mkpath(ftp, path)
ftp.mkd(path)
ftp = FTP(...)
directory = '/TransferredData/'
for File in FileList:
infile = File[0]
outfile = File[4].split('\\') # need forward slashes in FTP
outfile = directory + '/'.join(outfile)
mkpath(ftp, outfile)
ftp.storbinary('STOR '+outfile, open(infile, 'rb'))