Writing path for sftp server python - python

I am using pysftp on Python and am trying to run a loop for a certain directory in the sftp server.
I don't know how to write the directory paths on sftp servers. I thought that connecting to the server and just writing the directory as below would work, but it doesn't. Please let me know how to write sftp paths so that python can read them.
sftp = pysftp.Connection('128.59.164.112', username = '', password = '');
source = sftp.u\'weatherForecast\'/dataRAW/2004/grib/tmax/

Try this:
import pysftp
sftp = pysftp.Connection('hostname', username='me', password='secret')
sftp.chdir('/home/user/development/stackoverflow')
ls = sftp.listdir()
for filename in ls:
print filename
You should read this: http://pysftp.readthedocs.org/en/latest/index.html
PS1: ; is optional in Python, but not Pythonic.

After enough trial and error I figured it out.
source = 'weatherForecast/dataRAW/2004/grib/tmax/'
destination= 'sftp.u\'weatherForecast\'/csv/2004/tmax'
This works.

Related

Download files from SFTP server to specific local directory using Python pysftp

I'm attempting to download a set of files from an SFTP server. I can download the files successfully, but now I want to change the script so that as the files download, they go to a specified directory on my local server.
I attempted doing this with the sftp.getfo() command, but am getting an error:
getfo() got an unexpected keyword argument 'fl'
#SET LOCAL DIRECTORY
directory1 = r'C:\Users\Me\Documents\test sftp'
#CONNECT TO SFTP
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
sftp = pysftp.Connection(host = hostname, username = usereu, password = passeu, cnopts = cnopts)
print("Connection successfully established ... ")
#DOWNLOAD ALL FILES IN FOLDER
files = sftp.listdir()
for file in files:
sftp.getfo(remotepath = file, fl = directory1)
print(file,' downloaded successfully ')
Am I using the right command here or is there a better way to do this?
Thanks!
To download a file to local path, use Connection.get (not getfo):
sftp.get(remotepath=file, localpath=os.path.join(directory1, file))
Some notes:
remotepath takes path to the remote file, not just filename. Your code (which passes filename only) works only because you are downloading from the home folder.
The pysftp is dead. Do not use it. Use Paramiko. See pysftp vs. Paramiko
For an example, see Download file from SFTP server in Python Paramiko
Do not set cnopts.hostkeys = None, unless you do not care about security. For the correct solution see Verify host key with pysftp.

Download the latest file according to timestamp in file name from SFTP server

I'm trying to get latest new file in a directory of remote Linux server. The file in SFTP server is created every 4 hours and the file have specific name start with filegen_date_hour.json as per example below. In this case latest file 'filegen_20200101_0800.json' need to be transferred to my local directory.
filegen_20200101_0000.json
filegen_20200101_0400.json
filegen_20200101_0800.json
I use Python 3 code below, but got error
latestFile = max(listFile, key=os.path.getctime)
ValueError: max() arg is an empty sequence
SFTP code below
myHostname = "192.168.100.10"
myUsername = "user"
myPassword = "password"
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
with pysftp.Connection(host=myHostname, username=myUsername, password=myPassword, cnopts=cnopts) as sftp:
with sftp.cd('/home/operation/genfiles/'):
fileDir = '/home/operation/genfiles/filegen_*.json'
**#file have specific pattern with filegen_*.json**
listFile = glob.glob(fileDir)
latestFile = max(listFile, key=os.path.getctime)
sftp.get(latestFile)
Appreciate help on this matter. Thank you for your response and help.
First, you cannot use glob to list files on an SFTP server. The glob won't magically start querying SFTP server only because you have opened an SFTP connection before. It will still query local file system.
Use pysftp Connection.listdir. Though it does not support wildcards, so you will have to filter the files you want locally. Like here:
List files on SFTP server matching wildcard in Python using Paramiko
Only then you can try finding the latest file.
In general, you may use file modification time, as here:
How to download only the latest file from SFTP server with Paramiko?
The code is for Paramiko SFTPClient.listdir_attr, but it's the same with pysftp Connection.listdir_attr.
But in your case, I'm not sure if you can rely on the modification timestamp. It seems that you actually want to use the timestamp in the filename. With your file name format, you can simply pick the last file lexicographically.
import fnmatch
...
with sftp.cd('/home/operation/genfiles'):
files = []
for filename in sftp.listdir():
if fnmatch.fnmatch(filename, "filegen_*.json"):
files.append(filename)
latestFile = max(files)
Obligatory warning: Do not set cnopts.hostkeys = None, unless you do not care about security. For the correct solution see Verify host key with pysftp.

unable to download files from sftp server using python

i am writing a python script that helps me download files from sftp server to my local folder.
when i run the script it just downloads the blank document
i keep on trying but i am failing
i made a sftp server to test the code and the path specified in remote path is server root directory
i giving code below
import pysftp
myHostname = "192.168.56.1"
myUsername = "new45"
myPassword = "146515"
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
with pysftp.Connection(host=myHostname, username=myUsername, password=myPassword,cnopts=cnopts) as sftp:
# Define the file that you want to download from the remote directory
remoteFilePath = 'C:\\Users\\Simon\\Desktop\\ftp\\testfile.txt'
# Define the local path where the file will be saved
# or absolute "C:\Users\sdkca\Desktop\TUTORIAL.txt"
localFilePath = 'C:\\Users\\Simon\\Desktop\\ftp2\\textfile.txt'
sftp.get(remoteFilePath, localFilePath)
# connection closed automatically at the end of the with-block'''
please tell me whats the error why is blank file is being downloaded
Login to that server using any GUI SFTP client. And observe what path syntax your server is using. It's definitely not C:\Users\Simon\Desktop\ftp\testfile.txt. SFTP uses forward slashes. So it can be like /C:/Users/Simon/Desktop/ftp/testfile.txt. But even something completely different, particularly if the server or your account are chrooted.
You are probably trying to use the path syntax, which you would have used, were you directly on that target Windows system. But your SFTP server presents the files using a different path syntax. You need to adhere to that syntax.

How can I download file from SFTP using pysftp?

I have a SFTP server that contains a set of folders. I would like to download a file from one of these folders.
Can someone help me with the python code?
Where should I start?
Have you tried the example in pysftp's documentation?
import pysftp
with pysftp.Connection('hostname', username='me', password='secret') as sftp:
with sftp.cd('public'): # temporarily chdir to public
sftp.put('/my/local/filename') # upload file to public/ on remote
sftp.get('remote_file') # get a remote file
An addition from Michael Powers.
If your server is one account and your personal account is another one.
If your server account has root permission, then follow Michael's method.
If no, you may use io buffer to download this file
import io
server_sftp = pysftp.Connection(host=test_host, username=server_account,
password=server_pwd, cnopts=cn_opts)
your_user_sftp = pysftp.Connection(host=host_name, username = user_name,
password = user_pwd, cnopts=cn_opts)
try:
file_like=io.BytesIO()
server_account.getfo(server_file_location,file_like)
your_user_sftp.putfo(file_like, target_file_location)
finally:
flie_like.close()
server_sftp.close()
your_user_sftp.close()

Python Script Uploading files via FTP

I would like to make a script to upload a file to FTP.
How would the login system work? I'm looking for something like this:
ftp.login=(mylogin)
ftp.pass=(mypass)
And any other sign in credentials.
Use ftplib, you can write it like this:
import ftplib
session = ftplib.FTP('server.address.com','USERNAME','PASSWORD')
file = open('kitten.jpg','rb') # file to send
session.storbinary('STOR kitten.jpg', file) # send the file
file.close() # close file and FTP
session.quit()
Use ftplib.FTP_TLS instead if you FTP host requires TLS.
To retrieve it, you can use urllib.retrieve:
import urllib
urllib.urlretrieve('ftp://server/path/to/file', 'file')
EDIT:
To find out the current directory, use FTP.pwd():
FTP.pwd(): Return the pathname of the current directory on the server.
To change the directory, use FTP.cwd(pathname):
FTP.cwd(pathname): Set the current directory on the server.
ftplib now supports context managers so I guess it can be made even easier
from ftplib import FTP
from pathlib import Path
file_path = Path('kitten.jpg')
with FTP('server.address.com', 'USER', 'PWD') as ftp, open(file_path, 'rb') as file:
ftp.storbinary(f'STOR {file_path.name}', file)
No need to close the file or the session
You will most likely want to use the ftplib module for python
import ftplib
ftp = ftplib.FTP()
host = "ftp.site.uk"
port = 21
ftp.connect(host, port)
print (ftp.getwelcome())
try:
print ("Logging in...")
ftp.login("yourusername", "yourpassword")
except:
"failed to login"
This logs you into an FTP server. What you do from there is up to you. Your question doesnt indicate any other operations that really need doing.
Try this:
#!/usr/bin/env python
import os
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('hostname', username="username", password="password")
sftp = ssh.open_sftp()
localpath = '/home/e100075/python/ss.txt'
remotepath = '/home/developers/screenshots/ss.txt'
sftp.put(localpath, remotepath)
sftp.close()
ssh.close()
To avoid getting the encryption error you can also try out below commands
ftp = ftplib.FTP_TLS("ftps.dummy.com")
ftp.login("username", "password")
ftp.prot_p()
file = open("filename", "rb")
ftp.storbinary("STOR filename", file)
file.close()
ftp.close()
ftp.prot_p() ensure that your connections are encrypted
I just answered a similar question here
IMHO, if your FTP server is able to communicate with Fabric please us Fabric. It is far better than doing raw ftp.
I have an FTP account from dotgeek.com so I am not sure if this will work for other FTP accounts.
#!/usr/bin/python
from fabric.api import run, env, sudo, put
env.user = 'username'
env.hosts = ['ftp_host_name',] # such as ftp.google.com
def copy():
# assuming i have wong_8066.zip in the same directory as this script
put('wong_8066.zip', '/www/public/wong_8066.zip')
save the file as fabfile.py and run fab copy locally.
yeukhon#yeukhon-P5E-VM-DO:~$ fab copy2
[1.ai] Executing task 'copy2'
[1.ai] Login password:
[1.ai] put: wong_8066.zip -> /www/public/wong_8066.zip
Done.
Disconnecting from 1.ai... done.
Once again, if you don't want to input password all the time, just add
env.password = 'my_password'
You can use the below function. I haven't tested it yet, but it should work fine. Remember the destination is a directory path where as source is complete file path.
import ftplib
import os
def uploadFileFTP(sourceFilePath, destinationDirectory, server, username, password):
myFTP = ftplib.FTP(server, username, password)
if destinationDirectory in [name for name, data in list(remote.mlsd())]:
print "Destination Directory does not exist. Creating it first"
myFTP.mkd(destinationDirectory)
# Changing Working Directory
myFTP.cwd(destinationDirectory)
if os.path.isfile(sourceFilePath):
fh = open(sourceFilePath, 'rb')
myFTP.storbinary('STOR %s' % f, fh)
fh.close()
else:
print "Source File does not exist"

Categories

Resources