I am using pysftp to upload a local file that is processed remotely, then returned to the SFTP, where I can download it. At the moment, I cannot get the code to recognize that the processed remote file has been uploaded. The code just waits forever, despite the remotely processed file successfully being uploaded.
While the code is running, if I refresh the connection on a service such as FileZilla, the code instantly recognizes the file is in and works perfectly. However, I need it to work without manually refreshing on FileZilla. I am not sure why the code cannot recognize itself that the file has been uploaded and is ready to be downloaded.
I've already tried disconnecting then re-connecting to the SFTP using a while statement, which was not successful.
import pysftp
import stat
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
srv = pysftp.Connection(host="host", username="username", password="passowrd", cnopts=cnopts)
print("Connection Made!")
# Put File In
srv.chdir(r'/path_to_remote_directory')
srv.put(r'C:\Path_to_local_file.csv')
print("File Uploaded")
while not srv.isfile(r'/Path_to_newly_uploaded_remote_file.csv'):
time.sleep(3)
print("Still Waiting")
if srv.isfile(r'/Path_to_newly_uploaded_remote_file.csv'):
print("File is Ready!")
srv.get('/Path_to_newly_uploaded_remote_file.csv', r'C:\Local_path_save_to/file.csv')
# Closes the connection
srv.close()
Just remove the /:
srv.isfile(r'/Path_to_newly_uploaded_remote_file.csv'):
->
srv.isfile(r'Path_to_newly_uploaded_remote_file.csv'):
NOTE:
Do not set cnopts.hostkeys = None,
unless you do not care about security. You lose a protection against
Man-in-the-middle attacks by doing so.
I've implemented auto_add_key in my pysftp github fork.
auto_add_key will add the key to known_hosts if auto_add_key=True
Once a key is present for a host in known_hosts this key will be checked.
Please refer to Martin Prikryl -> answer about security concerns.
Why using context manager is a good approach:
import pysftp
with pysftp.Connection(host, username="whatever", password="whatever", auto_add_key=True) as sftp:
#do your stuff here
#connection closed
EDIT:
Checked with my code and / was the problem ... please check spelling of your file and that you are in the correct working dir getcwd()_
import pysftp
import stat
import time
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None #do not do this !!!
srv = pysftp.Connection("host", username="user", password="pass", cnopts=cnopts)
print("Connection Made!")
# Put File In
srv.chdir(r'/upload')
srv.put(r'C:\Users\Fabian\Desktop\SFTP\testcsv.csv')
print("File Uploaded")
print(srv.getcwd()) #get the current folder ! <- you should looking here
while not srv.isfile(r'testcsv.csv'):
time.sleep(3)
print("Still Waiting")
if srv.isfile(r'testcsv.csv'):
print("File is Ready!")
srv.get('testcsv.csv', r'C:\Users\Fabian\Desktop\SFTP\testcsv_new.csv')
# Closes the connection
srv.close()
Found a working solution. I think it was caused by the old directory being cached. By pointing to the new directory every time for an update, it appears to work. Obvious solution now but took me a while. Posting in case anyone else runs into this.
while not srv.isfile(r'/Path_to_newly_uploaded_remote_file.csv'):
time.sleep(3)
srv.chdir(r'/Path_to_newly_uploaded_remote_file')
print("Still Waiting")
Related
I'm trying to connect to SFTP from a notebook with databricks in python. My notebook looks like this:
import pysftp
import paramiko
# set parameters
host_name = 'xx.xxx.xxx.xxx'
username = 'FTP_USERNAME'
file_path_to_rsa_key = "/path/key_rsa"
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
# connect to SFTP
sftp = pysftp.Connection(host_name, username=username, private_key=file_path_to_rsa_key, cnopts=cnopts)
data = sftp.listdir()
sftp.close()
# Prints out the directories and files, line by line
for i in data:
print(i)
I have the following error:
Oops, unhandled type 3 ('unimplemented')
when running the following block:
try:
conn = pysftp.Connection(host_name, username=username, private_key=file_path_to_rsa_key, cnopts=cnopts)
print("connection established successfully")
except:
print('failed to establish connection to targeted server')
It print connection established successfully
What does it mean? What should I do? Is the issue with listdir()?
It seems that the issue is mostly because pysftp. I end up switching entirely to paramiko and everything is working fine. The SFTP server modified the login response and pysftp was too strict in the acceptable responses to take it without trouble. I noticed something similar when I tried to use a non-RSA private key that worked fine for ssh / sftp directly but not pysftp.
There is also some interesting information here pysftp vs. Paramiko
I am trying to upload a file via SFTP to my server. But instead of just uploading it, i have to explicitly tell my skript what file to overwrite on the server. I don't know how to change that.
#!/usr/bin/python3
import paramiko
k = paramiko.RSAKey.from_private_key_file("/home/abdulkarim/.ssh/id_rsa")
c = paramiko.SSHClient()
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
print("connecting")
c.connect( hostname = "do-test", username = "abdulkarim", pkey = k )
print("connected")
sftp = c.open_sftp()
sftp.put('/home/abdulkarim/Skripte/data/test.txt', '/home/abdulkarim/test/test1.txt')
c.close()
In the below call, the second (remotepath) parameter refers to the path, where the file will be stored on the server. There is not requirement for the remote file to actually exist. It will be created.
sftp.put('/home/abdulkarim/Skripte/data/test.txt', '/home/abdulkarim/test/test1.txt')
Obligatory warning: Do not use AutoAddPolicy – You are losing a protection against MITM attacks by doing so. For a correct solution, see Paramiko "Unknown Server".
Python 3.7.4
Idle 3.7 32-bit
I am attempting to connect to an SFTP and download a .CSV file to a local network drive. When I run my code, I get no error message, but nothing prints in the shell and nothing is being downloaded to my local directory.
import pysftp
import time
def SFTP_Get_File():
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
host='ServerName'
user='FakeUser'
pw='FakePassword'
with pysftp.Connection(host,
username=user,
password=pw,
cnopts = cnopts
) as sftp:
print("Connection succesfully stablished ... ")
sftp.get('/upload/File_Name.csv', r'Local_File_Name.csv')
time.sleep(2)
sftp.close()
Expected Results is I get a print of connection success and the file in the SFTP downloads to my local directory.
Actual Results. Nothing happens.
It looks like you are defining the function SFTP_Get_File but you are not calling it.
Try adding
SFTP_Get_File()
to the bottom of your script and running it.
I have a SFTP server that contains a set of folders. I would like to download a file from one of these folders.
Can someone help me with the python code?
Where should I start?
Have you tried the example in pysftp's documentation?
import pysftp
with pysftp.Connection('hostname', username='me', password='secret') as sftp:
with sftp.cd('public'): # temporarily chdir to public
sftp.put('/my/local/filename') # upload file to public/ on remote
sftp.get('remote_file') # get a remote file
An addition from Michael Powers.
If your server is one account and your personal account is another one.
If your server account has root permission, then follow Michael's method.
If no, you may use io buffer to download this file
import io
server_sftp = pysftp.Connection(host=test_host, username=server_account,
password=server_pwd, cnopts=cn_opts)
your_user_sftp = pysftp.Connection(host=host_name, username = user_name,
password = user_pwd, cnopts=cn_opts)
try:
file_like=io.BytesIO()
server_account.getfo(server_file_location,file_like)
your_user_sftp.putfo(file_like, target_file_location)
finally:
flie_like.close()
server_sftp.close()
your_user_sftp.close()
I need some help I'm getting below message when i try to connect. The script Im executing is below. Any suggestions as to how I can fix this issue?
Also, once I get this issue fixed Im looking to add to this script the ability to listen until the result file is available and to download it from a different remote directory.
Thank you, in advance
No hostkey for host mysftpserver.com found.
Exception AttributeError: "'Connection' object has no attribute
'_sftp_live'" in <bound method Connection.__del__ of <pysftp.Connection
object at 0x101d8cd50>> ignored
Process finished with exit code 0
Script
import pysftp as sftp
def sftpExample():
try:
s = sftp.Connection(host='mysftpserver', port=1211, username='myusername', password='mypassword')
remotepath='/Orders/incoming/myfile.csv'
localpath='/Users/myusername/projects/order88.csv'
s.put(localpath,remotepath)
s.close()
except Exception, e:
print str(e)
sftpExample()
The pysftp documentation and other answers on Stack Overflow also seem to suggest that the error can be resolved by setting cnopts.hostkeys = None.
So the full code would be something like:
import pysftp
def sftpExample():
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
with pysftp.Connection('hostname', username='me', password='secret', cnopts=cnopts) as sftp:
sftp.put('/Users/myusername/projects/order88.csv','/Orders/incoming/myfile.csv') # upload file to public/ on remote
sftp.close()
sftpExample()
Hope it helps!
Per the pysftp documentation you need to have the remote host key in your ~/.ssh/known_hosts. You can do this by connecting to the server via the ssh command (it will ask you if you want to add a new key to your hosts file; simply type yes if you trust the remote host's fingerprint).
The official documentation also points out how to disable host checking entirely, but this is NOT recommended as it would defeat critical security features for SSH identification.