Any way to increase speed of closing a file? - python

I'm trying to write a html-file and then upload it to my website using the following code:
webpage = open('testfile.html',"w")
webpage.write(contents)
webpage.close
server = 'ftp.xxx.be'
username = 'userxxx'
password = 'topsecret'
ftp_connection = ftplib.FTP(server, username, password)
remote_path = "/"
ftp_connection.cwd(remote_path)
fh = open("testfile.html", 'rb')
ftp_connection.storbinary('STOR testfile.html', fh)
fh.close()
The problem is the .close command seems to be slower than the ftp connection and the file that is sent over ftp is empty. A few seconds after the ftp is executed I see the file correctly locally on my PC.
Any hints to be certain the .close is finished before the ftp starts (apart from using time.sleep())?
Running Python 3.xx on W7pro

Try blocking on the close call:
Blocking until a file is closed in python
By the way, are the parentheses missing on your close call?

Related

Show Python FTP file upload messages

I have a remote FTP server where I want to upload new firmware images. When using the Linux ftp client I can do this using put <somefile> the server then responds with status messages like:
ftp> put TS252P005.bin flash
local: TS252P005.bin remote: flash
200 PORT command successful
150 Connecting to port 40929
226-File Transfer Complete. Starting Operation:
Checking file integrity...
Updating firmware...
Result: Success
Rebooting...
421 Service not available, remote server has closed connection
38563840 bytes sent in 6.71 secs (5.4779 MB/s)
ftp>
Now I can upload the file using Python as well using ftplib:
fw_path = ....
ip = ....
user = ...
pw = ....
with open(fw_path, "rb") as f:
with ftplib.FTP(ip) as ftp:
ftp.login(user=user, passwd=pw)
ftp.storbinary("stor flash", f)
But I can't see a way for me to get the status messages that I can see using the ftp utility. This is important for me because I need to check that the update actually succeeded.
How can I get this output in my Python program? I'm also willing to use a different library if ftplib can't do it.
Any help is appreciated!
If you want to check the response programatically, check the result of FTP.storbinary:
print(ftp.storbinary("STOR flash", f))
Though as your server actually closes the connection before even sending a complete response, the FTP.storbinary throws an exception.
If you want to read the partial response, you will have to re-implement what the FTP.storbinary does. Like
ftp.voidcmd('TYPE I')
with ftp.transfercmd("STOR flash") as conn:
while 1:
buf = f.read(8192)
if not buf:
break
conn.sendall(buf)
line = ftp.getline()
print(line)
if line[3:4] == '-':
code = line[:3]
while 1:
nextline = self.getline()
print(nextline)
if nextline[:3] == code and nextline[3:4] != '-':
break
If you want to check the response manually, enable logging using FTP.set_debuglevel.

Uploading file using Paramiko in Python seemingly works, but the file cannot be found on the server

I am new to Python and sorry for my bad english.
I'm trying to save a file "toto.txt" from my HDD "d:" to my Synology NAS.
So I'll use paramiko for this, here is the code :
import paramiko
import os
ip_address = "my nas ip"
username = "my username"
password = "mypass"
utilisateur = os.getenv("USERNAME") // to get Windows username
remote_path = f"{utilisateur}/test.txt" // the file downloaded
local_path = "d:/toto.txt" //the file stored on my pc
ssh_client = paramiko.SSHClient()
ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect(hostname=ip_address,username=username,password=password)
print("Connexion OK >", ip_address)
stdin, stdout, stderr = ssh_client.exec_command(f'mkdir {utilisateur}') //creating folder for the
user
sftp = ssh_client.open_sftp()
sftp.put(local_path,remote_path) // trying to send the file
sftp.close()
ssh_client.close()
i am not getting error but nothing is happening.
The folder is successful created but no file is sending in it.
Have someone an idea?
thanks a lot
If Paramiko does not throw any error, the upload was successful. The file just possibly ended in different location than you wanted/than you look to.
At least for a test, if not permanently, try an absolute absolute path. Make no guesses, use the exact path you see in your GUI SFTP client.
Another possibility is that the server automatically processes the uploaded file somehow and moves it away.

How do I execute ftp "put C:\path\to\file" in Python

I am trying to upload a file to an FTP server, I wanted to write a Python script to simplify the process. Currently I ftp to the server and execute these commands:
quote allo file_size_in_bytes
put c:\path\to\file
This is what I have so far, I am not able to get the file to transfer using the put command.
from ftplib import FTP
import os
import time
from subprocess import call
ip = raw_input('Enter ip address: ') # User input for host
print ip # Prints host
filelocation = raw_input('Drag file here: ') # File to upload
print ('This is the local file '+filelocation) # Verify file location
filename = os.path.basename(filelocation) # If a File name is needed
filesize = os.path.getsize(filelocation) # Need file size in bytes for quote allo command
print ('This is the file size in bytes '+str(filesize)) # Verify correct size
time.sleep(2) # Pause to look at screen
ftp = FTP(ip) # Open ftp connection
ftp.login('user','pass') # Login
print ftp.getwelcome() # Verify proper connection
remotepath = ftp.pwd()+filename # If a remote path to ftp server is needed
print ('This is the file path on the processor '+remotepath) # Verify remote path
"\"quote\"" # Need this, not sure why
ftp.sendcmd('allo '+str(filesize)) # quote allo filesize, seems to work
#"\"put\"" # Experimenting, don't know if this is needed
call(['echo "put C:\filelocation" | ftp']) # This doesn't appear to work
time.sleep(5)
ftp.quit()
You are logging in with ftplib, yet you are trying to run an external ftp process for the upload itself. That cannot work as the external ftp process does not know about your ftplib FTP session.
To upload a file using the ftplib, use storbinary or storlines method.
See for example Python Script Uploading files via FTP.

Python: changing directory name at FTP server for downloading the file

I am trying to download a file from FTP server. I am able to connect to the server. But not able to change the directory.
#! /user/bin/python33
import os
import ftplib
ftp = ftplib.FTP("ftp.sec.gov")
ftp.login("anonymous", "abcd#yahoo.com")
data = []
ftp.dir(data.append)
ftp.quit()
for line in data:
print( "-", line)
print(os.getcwd())
path= "/edgar/full-index/2013/"
print(path)
ftp.cwd(path)
it fails in the last line. can some one suggest what needs to be done"
thanks a lot in advance
Your cwd call fails because you previously called ftp.quit().
The docs for that method say:
Send a QUIT command to the server and close the connection. This is the “polite” way to close a connection, but it may raise an exception if the server responds with an error to the QUIT command. This implies a call to the close() method which renders the FTP instance useless for subsequent calls (see below).
(The "below" reference is to the next part of the documentation which says you can't do any operations on a closed FTP object.)

How to stop ftp from downloading in python?

Here are some bits of code I use to download through ftp. I was trying to stop the download then continue or redownload it afterwards. I've tried ftp.abort() but it only hangs and returns timeout.
ftplib.error_proto: 421 Data timeout. Reconnect. Sorry.
SCENARIO:
The scenario is the user will choose the file to download, while downloading, the user can stop the current download and download a new file. The code 'if os.path.getsize(self.file_path) >117625:' is just my example if the user stops the download. Its not the full size of the file.
thanks.
from ftplib import FTP
class ftpness:
def __init__(self):
self.connect(myhost, myusername, mypassword)
def handleDownload(self,block):
self.f.write(block)
if os.path.getsize(self.file_path) >117625:
self.ftp.abort()
def connect(self,host, username, password):
self.ftp = FTP(host)
self.ftp.login(username, password)
self.get(self.file_path)
def get(self,filename):
self.f = open(filename, 'wb')
self.ftp.retrbinary('RETR ' + filename, self.handleDownload)
self.f.close()
self.ftp.close
a = ftpness()
error 421 is the std timeout error. so need to have the connection until the file has been downloaded.
def handleDownload(self,block):
self.f.write(block)
if os.path.getsize(self.file_path) >117625:
self.ftp.abort()
else:
self.ftp.sendcmd('NOOP')
#try to add this line just to keep the connection alive.
hope this will help you. :)
Here's a way to do it with a watchdog timer. This involves creating a separate thread, which depending on the design of your application may not be acceptable.
To kill a download with a user event, it's the same idea. If the GUI works in a separate thread, then that thread can just reach inside the FTP instance and close its socket directly.
from threading import Timer
class ftpness:
...
def connect(self,host, username, password):
self.ftp = FTP(host)
self.ftp.login(username, password)
watchdog = Timer(self.timeout, self.ftp.sock.close)
watchdog.start()
self.get(self.file_path)
watchdog.cancel() # if file transfer succeeds cancel timer
This way, if the file transfer runs longer than your preset timeout, the timer thread will close the socket underneath the transfer, forcing the get call to raise an exception. Only when the transfer succeeds is the watchdog timer cancelled.
And though this has nothing to do with your question, normally a connect call should not transfer payload data.
This is Your session idle time too long.You can file after the President into instantiate ftplib. Otherwise. Modify ftp software configuration.
For example, you use vsftpd, you can add the following configuration to vsftpd.conf:
idle_session_timeout=60000 # The default is 600 seconds

Categories

Resources