Develop Reference

Show Python FTP file upload messages

I have a remote FTP server where I want to upload new firmware images. When using the Linux ftp client I can do this using put <somefile> the server then responds with status messages like:
ftp> put TS252P005.bin flash
local: TS252P005.bin remote: flash
200 PORT command successful
150 Connecting to port 40929
226-File Transfer Complete. Starting Operation:
Checking file integrity...
Updating firmware...
Result: Success
Rebooting...
421 Service not available, remote server has closed connection
38563840 bytes sent in 6.71 secs (5.4779 MB/s)
ftp>
Now I can upload the file using Python as well using ftplib:
fw_path = ....
ip = ....
user = ...
pw = ....
with open(fw_path, "rb") as f:
with ftplib.FTP(ip) as ftp:
ftp.login(user=user, passwd=pw)
ftp.storbinary("stor flash", f)
But I can't see a way for me to get the status messages that I can see using the ftp utility. This is important for me because I need to check that the update actually succeeded.
How can I get this output in my Python program? I'm also willing to use a different library if ftplib can't do it.
Any help is appreciated!

If you want to check the response programatically, check the result of FTP.storbinary:
print(ftp.storbinary("STOR flash", f))
Though as your server actually closes the connection before even sending a complete response, the FTP.storbinary throws an exception.
If you want to read the partial response, you will have to re-implement what the FTP.storbinary does. Like
ftp.voidcmd('TYPE I')
with ftp.transfercmd("STOR flash") as conn:
while 1:
buf = f.read(8192)
if not buf:
break
conn.sendall(buf)
line = ftp.getline()
print(line)
if line[3:4] == '-':
code = line[:3]
while 1:
nextline = self.getline()
print(nextline)
if nextline[:3] == code and nextline[3:4] != '-':
break
If you want to check the response manually, enable logging using FTP.set_debuglevel.

Uploading file using Paramiko in Python seemingly works, but the file cannot be found on the server

I am new to Python and sorry for my bad english.
I'm trying to save a file "toto.txt" from my HDD "d:" to my Synology NAS.
So I'll use paramiko for this, here is the code :
import paramiko
import os
ip_address = "my nas ip"
username = "my username"
password = "mypass"
utilisateur = os.getenv("USERNAME") // to get Windows username
remote_path = f"{utilisateur}/test.txt" // the file downloaded
local_path = "d:/toto.txt" //the file stored on my pc
ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect(hostname=ip_address,username=username,password=password)
print("Connexion OK >", ip_address)
stdin, stdout, stderr = ssh_client.exec_command(f'mkdir {utilisateur}') //creating folder for the
user
sftp = ssh_client.open_sftp()
sftp.put(local_path,remote_path) // trying to send the file
sftp.close()
ssh_client.close()
i am not getting error but nothing is happening.
The folder is successful created but no file is sending in it.
Have someone an idea?
thanks a lot

If Paramiko does not throw any error, the upload was successful. The file just possibly ended in different location than you wanted/than you look to.
At least for a test, if not permanently, try an absolute absolute path. Make no guesses, use the exact path you see in your GUI SFTP client.
Another possibility is that the server automatically processes the uploaded file somehow and moves it away.

Connect from a cisco device to http server on debian

I'm trying to communicate with a http server which is running on debian strech from a brand new out of the box cisco device. Now, the so called zero touch configuration is no problem:
The Switch gets an IP address and such via DHCP and a link to where to fetch it's initial configuration.
The switch gets its basic configuration such as user credentials etc.
The problem rises when I try to search through a database on the server from the switch. In this database some variables are stored. Depending the serialnumber of the switch, it should receive a specific hostname, Mgmt address etc.
On those new switches there is a python module integrated so I ran some tests. I tried to fetch the serial number and got them whithout any problems. The moment I tried to write the serial number on a txt file on the server I got this error
Traceback (most recent call last): File "", line 1, in
IOError: [Errno 2] No such file or directory:
'http://10.232.152.19:80/temp.txt'
Code so far:
from cli import cli
def get_serial():
serial = cli("show version | include System Serial\n")
serial = (serial.split()[-1])
f = open ("http://10.232.152.19:80/temp.txt", "a")
f.write(serial)
f.close
get_serial()

The problem you are facing is because you are trying to open a file from the network. You need to download the file first in you system and then open it. You should use urllib to fetch the file and then open it. then save it and again push it back.
import urllib
txt = urllib.urlopen(target_url).read()

utf-8 issue when using socket in server PC

I want to send some file from a client PC to a server PC using Python.
But when I run the client code, there is a problem.
Here is the client code:
with socket.socket(socket.AF_INET,socket.SOCK_STREAM) as sock:
sock.connect((HOST,PORT))
sock.sendall(filename.encode('utf-8'))
print(filename.encode('utf-8'))
and this is the server code:
filename = self.request.recv(1024)
print(filename)
filename = filename.decode('utf-8')
When I run it within my PC (for example, server=client localhost), it runs perfectly.
But in my server PC, print result has some added text like:
b'filename\~~~~~~~~~~'
filename result
error message
So I think if the added text is gone when I use my server PC, the problem will be solved.
But I dont know how to get there.
Please let me know...

Open a remote file using paramiko in python slow [duplicate]

This question already has an answer here:
Reading file opened with Python Paramiko SFTPClient.open method is slow
(1 answer)
Closed 7 months ago.
I am using paramiko to open a remote sftp file in python. With the file object returned by paramiko, I am reading the file line by line and processing the information. This seems really slow compared to using the python in-built method 'open' from the os. Following is the code I am using to get the file object.
Using paramiko (slower by 2 times) -
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(myHost,myPort,myUser,myPassword)
sftp = client.open_sftp()
fileObject = sftp.file(fullFilePath,'rb')
Using os -
import os
fileObject = open(fullFilePath,'rb')
Am I missing anything? Is there a way to make the paramiko fileobject read method as fast as the one using the os fileobject?
Thanks!!

Your problem is likely to be caused by the file being a remote object. You've opened it on the server and are requesting one line at a time - because it's not local, each request takes much longer than if the file was sitting on your hard drive. The best alternative is probably to copy the file down to a local location first, using Paramiko's SFTP get.
Once you've done that, you can open the file from the local location using os.open.

I was having the same issue and I could not afford to copy the file locally because of security reasons, I solved it by using a combination of prefetching and bytesIO:
def fetch_file_as_bytesIO(sftp, path):
"""
Using the sftp client it retrieves the file on the given path by using pre fetching.
:param sftp: the sftp client
:param path: path of the file to retrieve
:return: bytesIO with the file content
"""
with sftp.file(path, mode='rb') as file:
file_size = file.stat().st_size
file.prefetch(file_size)
file.set_pipelined()
return io.BytesIO(file.read(file_size))

Here is a way that works using scraping the command line (cat) in paramiko, and reading all lines at once. Works well for me:
import paramiko
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.WarningPolicy())
client.connect(hostname=host, port=port, username=user, key_filename=ssh_file)
stdin, stdout, stderr = client.exec_command('cat /proc/net/dev')
net_dump = stdout.readlines()
#your entire file is now in net_dump .. do as you wish with it below ...
client.close()
The files I open are quite small so it all depends on your file size. Worth a try :)