Aim: I am trying to use SFTP through Paramiko in Python to upload files on server pc.
What I've done: To test that functionality, I am using my localhost (127.0.0.1) IP. To achieve that I created the following code with the help of Stack Overflow suggestions.
Problem: The moment I run this code and enter the file name, I get the "IOError : Failure", despite handling that error. Here's a snapshot of the error:
import paramiko as pk
import os
userName = "sk"
ip = "127.0.0.1"
pwd = "1234"
client=""
try:
client = pk.SSHClient()
client.set_missing_host_key_policy(pk.AutoAddPolicy())
client.connect(hostname=ip, port=22, username=userName, password=pwd)
print '\nConnection Successful!'
# This exception takes care of Authentication error& exceptions
except pk.AuthenticationException:
print 'ERROR : Authentication failed because of irrelevant details!'
# This exception will take care of the rest of the error& exceptions
except:
print 'ERROR : Could not connect to %s.'%ip
local_path = '/home/sk'
remote_path = '/home/%s/Desktop'%userName
#File Upload
file_name = raw_input('Enter the name of the file to upload :')
local_path = os.path.join(local_path, file_name)
ftp_client = client.open_sftp()
try:
ftp_client.chdir(remote_path) #Test if remote path exists
except IOError:
ftp_client.mkdir(remote_path) #Create remote path
ftp_client.chdir(remote_path)
ftp_client.put(local_path, '.') #At this point, you are in remote_path in either case
ftp_client.close()
client.close()
Can you point out where's the problem and the method to resolve it?
Thanks in advance!
The second argument of SFTPClient.put (remotepath) is path to a file, not a folder.
So use file_name instead of '.':
ftp_client.put(local_path, file_name)
... assuming you are already in remote_path, as you call .chdir earlier.
To avoid a need for .chdir, you can use an absolute path:
ftp_client.put(local_path, remote_path + '/' + file_name)
I'm working on a simple tool that transfers files to a hard-coded location with the password also hard-coded. I'm a python novice, but thanks to ftplib, it was easy:
import ftplib
info= ('someuser', 'password') #hard-coded
def putfile(file, site, dir, user=(), verbose=True):
"""
upload a file by ftp to a site/directory
login hard-coded, binary transfer
"""
if verbose: print 'Uploading', file
local = open(file, 'rb')
remote = ftplib.FTP(site)
remote.login(*user)
remote.cwd(dir)
remote.storbinary('STOR ' + file, local, 1024)
remote.quit()
local.close()
if verbose: print 'Upload done.'
if __name__ == '__main__':
site = 'somewhere.com' #hard-coded
dir = './uploads/' #hard-coded
import sys, getpass
putfile(sys.argv[1], site, dir, user=info)
The problem is that I can't find any library that supports sFTP. What's the normal way to do something like this securely?
Edit: Thanks to the answers here, I've gotten it working with Paramiko and this was the syntax.
import paramiko
host = "THEHOST.com" #hard-coded
port = 22
transport = paramiko.Transport((host, port))
password = "THEPASSWORD" #hard-coded
username = "THEUSERNAME" #hard-coded
transport.connect(username = username, password = password)
sftp = paramiko.SFTPClient.from_transport(transport)
import sys
path = './THETARGETDIRECTORY/' + sys.argv[1] #hard-coded
localpath = sys.argv[1]
sftp.put(localpath, path)
sftp.close()
transport.close()
print 'Upload done.'
Thanks again!
Paramiko supports SFTP. I've used it, and I've used Twisted. Both have their place, but you might find it easier to start with Paramiko.
You should check out pysftp https://pypi.python.org/pypi/pysftp it depends on paramiko, but wraps most common use cases to just a few lines of code.
import pysftp
import sys
path = './THETARGETDIRECTORY/' + sys.argv[1] #hard-coded
localpath = sys.argv[1]
host = "THEHOST.com" #hard-coded
password = "THEPASSWORD" #hard-coded
username = "THEUSERNAME" #hard-coded
with pysftp.Connection(host, username=username, password=password) as sftp:
sftp.put(localpath, path)
print 'Upload done.'
Here is a sample using pysftp and a private key.
import pysftp
def upload_file(file_path):
private_key = "~/.ssh/your-key.pem" # can use password keyword in Connection instead
srv = pysftp.Connection(host="your-host", username="user-name", private_key=private_key)
srv.chdir('/var/web/public_files/media/uploads') # change directory on remote server
srv.put(file_path) # To download a file, replace put with get
srv.close() # Close connection
pysftp is an easy to use sftp module that utilizes paramiko and pycrypto. It provides a simple interface to sftp.. Other things that you can do with pysftp which are quite useful:
data = srv.listdir() # Get the directory and file listing in a list
srv.get(file_path) # Download a file from remote server
srv.execute('pwd') # Execute a command on the server
More commands and about PySFTP here.
If you want easy and simple, you might also want to look at Fabric. It's an automated deployment tool like Ruby's Capistrano, but simpler and of course for Python. It's build on top of Paramiko.
You might not want to do 'automated deployment' but Fabric would suit your use case perfectly none the less. To show you how simple Fabric is: the fab file and command for your script would look like this (not tested, but 99% sure it will work):
fab_putfile.py:
from fabric.api import *
env.hosts = ['THEHOST.com']
env.user = 'THEUSER'
env.password = 'THEPASSWORD'
def put_file(file):
put(file, './THETARGETDIRECTORY/') # it's copied into the target directory
Then run the file with the fab command:
fab -f fab_putfile.py put_file:file=./path/to/my/file
And you're done! :)
fsspec is a great option for this, it offers a filesystem like implementation of sftp.
from fsspec.implementations.sftp import SFTPFileSystem
fs = SFTPFileSystem(host=host, username=username, password=password)
# list a directory
fs.ls("/")
# open a file
with fs.open(file_name) as file:
content = file.read()
Also worth noting that fsspec uses paramiko in the implementation.
With RSA Key then refer here
Snippet:
import pysftp
import paramiko
from base64 import decodebytes
keydata = b"""AAAAB3NzaC1yc2EAAAADAQABAAABAQDl"""
key = paramiko.RSAKey(data=decodebytes(keydata))
cnopts = pysftp.CnOpts()
cnopts.hostkeys.add(host, 'ssh-rsa', key)
with pysftp.Connection(host=host, username=username, password=password, cnopts=cnopts) as sftp:
with sftp.cd(directory):
sftp.put(file_to_sent_to_ftp)
Twisted can help you with what you are doing, check out their documentation, there are plenty of examples. Also it is a mature product with a big developer/user community behind it.
There are a bunch of answers that mention pysftp, so in the event that you want a context manager wrapper around pysftp, here is a solution that is even less code that ends up looking like the following when used
path = "sftp://user:p#ssw0rd#test.com/path/to/file.txt"
# Read a file
with open_sftp(path) as f:
s = f.read()
print s
# Write to a file
with open_sftp(path, mode='w') as f:
f.write("Some content.")
The (fuller) example: http://www.prschmid.com/2016/09/simple-opensftp-context-manager-for.html
This context manager happens to have auto-retry logic baked in in the event you can't connect the first time around (which surprisingly happens more often than you'd expect in a production environment...)
The context manager gist for open_sftp: https://gist.github.com/prschmid/80a19c22012e42d4d6e791c1e4eb8515
Paramiko is so slow. Use subprocess and shell, here is an example:
remote_file_name = "filename"
remotedir = "/remote/dir"
localpath = "/local/file/dir"
ftp_cmd_p = """
#!/bin/sh
lftp -u username,password sftp://ip:port <<EOF
cd {remotedir}
lcd {localpath}
get {filename}
EOF
"""
subprocess.call(ftp_cmd_p.format(remotedir=remotedir,
localpath=localpath,
filename=remote_file_name
),
shell=True, stdout=sys.stdout, stderr=sys.stderr)
PyFilesystem with its sshfs is one option. It uses Paramiko under the hood and provides a nicer paltform independent interface on top.
import fs
sf = fs.open_fs("sftp://[user[:password]#]host[:port]/[directory]")
sf.makedir('my_dir')
or
from fs.sshfs import SSHFS
sf = SSHFS(...
Here's a generic function that will download any given sftp url to a specified path
from urllib.parse import urlparse
import paramiko
url = 'sftp://username:password#hostname/filepath.txt'
def sftp_download(url, dest):
url = urlparse(url)
with paramiko.Transport((url.hostname, 22)) as transport:
transport.connect(None,url.username,url.password)
with paramiko.SFTPClient.from_transport(transport) as sftp:
sftp.get(url.path, dest)
Call it with
sftp_download(url, "/tmp/filepath.txt")
I am trying to download a file from my FTP server to a specific folder, without a GUI. This is what I have so far, but it does nothing,
import urllib
urllib.urlretrieve('ftp://USERNAME:PASSWORD#ftp.SERVERNAME/File path/', 'FILENAME')
I edited my answer to be more simpler ..now we will need to use FtpLib
the code below is straightforward and it's elegant :D
import ftplib
path = 'pub/Health_Statistics/NCHS/nhanes/2001-2002/'
filename = 'L28POC_B.xpt'
ftp = ftplib.FTP("Server IP")
ftp.login("UserName", "Password")
ftp.cwd(path)
ftp.retrbinary("RETR " + filename ,open(filename, 'wb').write)
ftp.quit()
Just in case you need some explanation:
path is obviously the location of the file in the ftp server
filename is the name + extension of the file you want to download form server
ftp.login is where you'll put your credentials(username, password)
ftp.cwd will change the current working directory to where the file is located in order to download it :)
retrbinary simply will get the file from the server and store in your local machine using the same name it had on the server :)
Do not forget to change Server IP argument to your server's ip
and Voila that's it.
I am trying to make a script that downloads ( or upload ) files over ssh, as ftp port is disabled from firewall. This is my script :
import os
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('10.170.21.93', username="abhishek", password="#bhishek$")
sftp = ssh.open_sftp()
localpath = 'abc.txt'
remotepath = '/opt/crestelsetup/patchzip'
sftp.put(localpath, remotepath)
sftp.close()
ssh.close()
This is giving me "IOError: Failure", can any one help?
You need to explicitly specify the remote path:
import os
import paramiko
ssh = paramiko.SSHClient()
ssh.connect('10.170.21.93', username="abhishek", password="#bhishek$")
sftp = ssh.open_sftp()
localpath = 'abc.txt'
remotepath = '/opt/crestelsetup/patchzip/abc.txt'
sftp.put(localpath, remotepath)
sftp.close()
ssh.close()
As per Martin Prikryl's comment, the following code line is highly discouraged as it opens you up against man in the middle attack, however, it can be a temporary fix for missing host keys
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
Just modified the destination path to include the file name as well.Try to change.
remotepath = '/opt/crestelsetup/patchzip'
to
remotepath = '/opt/crestelsetup/patchzip/abc.txt'
You need to modify remotepath. Since, your remote path is /opt/crestelsetup/patchzip. Now need to upload file join with remote path. It can be done using following way.
fname = os.path.basename(localpath)
sftp.put(localpath, os.path.join(remotepath, fname))
I am sure this has been resolved before but I cannot seem to find a similar Q&A (newbie)
Using Windows XP and Python 2.5, I m trying to use a script to connect to an FTP server and dowload files. It should be simple but following the instructions of similar scripts I get the errors:
ftp.login('USERNAME')
File "C:\Python25\lib\ftplib.py", line 373, in login
if resp[0] == '3': resp = self.sendcmd('PASS ' + passwd)
File "C:\Python25\lib\ftplib.py", line 241, in sendcmd
return self.getresp()
File "C:\Python25\lib\ftplib.py", line 216, in getresp
raise error_perm, resp
error_perm: 530 User USERNAME cannot log in.
The script I use is:
def handleDownload(block):
file.write(block)
print ".",
# Create an instance of the FTP object
# FTP('hostname', 'username', 'password')
ftp = FTP('servername')
print 'ftplib example'
# Log in to the server
print 'Logging in.'
# You can specify username and password here if you like:
ftp.login('USERNAME', 'password')
#print ftp.login()
# This is the directory
directory = '/GIS/test/data'
# Change to that directory.
print 'Changing to ' + directory
ftp.cwd(directory)
# Print the contents of the directory
ftp.retrlines('LIST')
I appreciate this might be a trivial question, but if anyone can provide some insights it would be very helpful!
Thanks, S
I can't understand which library are you using. Python standard urllib2 is sufficient:
import urllib2, shutil
ftpfile = urllib2.urlopen("ftp://host.example.com/path/to/file")
localfile = open("/tmp/downloaded", "wb")
shutil.copyfileobj(ftpfile, localfile)
If you need to login (anonymous login isn't sufficient), then specify the credentials inside the url:
urllib2.urlopen("ftp://user:password#host.example.com/rest/of/the/url")
ftp.login('USERNAME', 'password')
Replace this with real data. According to the error you are trying to login as "USERNAME" with the password "password" which obviously won't work.
Also, replace servername in ftp = FTP('servername')
with the hostname of the server you want to connect to.
the first trivial check would be to open an interactive session (i.e. ftp yourself to this server with the same credentials), to be sure that this is not a permission issue..
Another source of failure, you might need to give your username as domain\username when connecting to a MS ftp server.
Maybe that helps ?