Hi I am trying to create an FTP server and to aid the development I'm using pyftpdlib. What I wanted to do is to do some file operations if a user downloads a specific file but sometimes it raises an exception and I don't really know why.
I wrote my own handler in pyftpdlib after this tutorial: http://code.google.com/p/pyftpdlib/wiki/Tutorial#3.8_-_Event_callbacks
But something goes terribly wrong sometimes when the user downloads the log file (which I intend to do some file operations on) and I don't really understand why. I have another class which basically reads from a configuration file and the error message said it couldn't find FTP Section. But it's strange because I clearly have it in my configuration file and it is working sometimes perfectly.
May this error appear because I have two "Connection" objects? That's the only guess I have, so I would be very glad if someone could explain what's going wrong. Here is my code that's troubled (nevermind the file.name check because that was very recently added):
class ArchiveHandler(ftpserver.FTPHandler):
def on_login(self, username):
# do something when user login
pass
def on_logout(self, username):
# do something when user logs out
pass
def on_file_sent(self, file):
"What to do when retrieved the file the class is watching over"
attr = Connection()
if attr.getarchive() == 'true':
t = datetime.now()
if file.name == "log.log":
try:
shutil.copy2(file, attr.getdir() + ".archive/" + str(t.strftime("%Y-%m-%d_%H:%M:%S") + '.log'))
except OSError:
print 'Could not copy file'
raise
if attr.getremain() == 'false':
try:
os.remove(file)
except OSError:
print 'Could not remove file'
raise
The full source:
http://pastie.org/3552079
Source of the config-file:
http://pastie.org/3552085
EDIT-> (and of course the error):
[root]#85.230.122.159:40659 unhandled exception in instance <pyftpdlib.ftpserver.DTPHandler object at 0xb75f49ec>
Traceback (most recent call last):
File "/usr/lib/python2.6/asyncore.py", line 84, in write
obj.handle_write_event()
File "/usr/lib/python2.6/asyncore.py", line 435, in handle_write_event
self.handle_write()
File "/usr/lib/python2.6/asynchat.py", line 174, in handle_write
self.initiate_send()
File "/usr/lib/python2.6/asynchat.py", line 215, in initiate_send
self.handle_close()
File "/usr/local/lib/python2.6/dist-packages/pyftpdlib/ftpserver.py", line 1232, in handle_close
self.close()
File "/usr/local/lib/python2.6/dist-packages/pyftpdlib/ftpserver.py", line 1261, in close
self.cmd_channel.on_file_sent(filename)
File "ftp.py", line 87, in on_file_sent
File "ftp.py", line 12, in __init__
File "/usr/lib/python2.6/ConfigParser.py", line 311, in get
raise NoSectionError(section)
NoSectionError: No section: 'FTP Section'
The problem is in a section you didn't include. It says
File "ftp.py", line 12, in __init__
File "/usr/lib/python2.6/ConfigParser.py", line 311, in get
raise NoSectionError(section)
NoSectionError: No section: 'FTP Section'
So from the first line, we know that whatever is on line 12 of ftp.py is the problem (since everything below that isn't our code, so we assume that it's correct).
Line 12 is this:
self.ip = config.get('FTP Section', 'hostname')
And the error message says "No section: 'FTP Section'".
From this we can assume there's an error in the config file (that it doesn't have a "FTP Section").
Are you pointing at the correct config file? Is it in the same directory that you're running the script from? Being in the same folder as the script will not work, it must be the folder that you run the script from.
I think this is the problem you're having, since according to the documentation:
If none of the named files exist, the ConfigParser instance will contain an empty dataset.
You can confirm this by trying to open the file.
The problem in this case was that by reading the file it left the file open.
I changed to this and it's working much better:
config = ConfigParser.RawConfigParser()
fp = open('config.cfg')
config.readfp(fp)
And then when I'm finished reading in the constructor I add:
#Close the file
fp.close()
And voila, you can open how many objects of the class you want and it won't show any errors. :)
Related
I would appreciate a hand with this.
It has previously popped corrupt file errors when opening the word file, but if I change .doc to .docx and remove some hyperlinks (I understand from another post somewhere that hyperlinks, footnotes, comments all cause errors), this time IDLE pops out the following error:
Traceback (most recent call last):
File "C:\Users\User\AppData\Local\Programs\Python\Python38-32\Files_tempfiller_\tempfiller.py", line 43, in
document.write('TBCO.docx')
File "C:\Users\User\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\mailmerge.py", line 129, in write
output.writestr(zi.filename, self.zip.read(zi))
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.1776.0_x64__qbz5n2kfra8p0\lib\zipfile.py", line 1475, in read
with self.open(name, "r", pwd) as fp:
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.1776.0_x64__qbz5n2kfra8p0\lib\zipfile.py", line 1532, in open
raise BadZipFile("Truncated file header")
zipfile.BadZipFile: Truncated file header
Because this is in the mailmerge.py file I can't really understand what this is.
My code is as follows:
from __future__ import print_function
from mailmerge import MailMerge
from datetime import date
template = "TBCO.docx" #add .docx suffix if failing
print('1')
document = MailMerge(template)
print(document.get_merge_fields())
print('2')
document.merge(
date = '1.1.1',
name = 'Bob',
nhs = '2223')
print('3')
document.write('TBCO.docx')
print('4')
The prints were for me to see what was happening when it was giving set() repeatedly, but that's fixed. The sense I get from the error message is that it is struggling with the file type for some reason, but I can't make head nor tail of the error. Any help would be appreciated.
Thank you
I am working on html parser, it uses Python multiprocessing Pool, because it runs through huge number of pages. The output from every page is saved to a separate CSV file. The problem is sometimes I get unexpected error and whole program crashes and I have errors handling almost everywhere - reading pages, parsing pages, even writing files. Moreover it looks like the script crashes after it finishes writing a batch of files, so it shouldn't be anything to crush on. Thus after whole day of debugging I am left clueless.
Error:
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "D:\Programy\Python36-32\lib\multiprocessing\pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "D:\Programy\Python36-32\lib\multiprocessing\pool.py", line 44, in mapstar
return list(map(*args))
File "D:\ppp\Python\parser\run.py", line 244, in media_process
save_media_product(DIRECTORY, category, media_data)
File "D:\ppp\Python\parser\manage_output.py", line 180, in save_media_product
_file_manager(target_file, temp, temp2)
File "D:\ppp\Python\store_parser\manage_output.py", line 214, in _file_manager
file_to_write.close()
UnboundLocalError: local variable 'file_to_write' referenced before assignment
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "D:\ppp\Python\store_parser\run.py", line 356, in <module>
main()
File "D:\Rzeczy Mariusza\Python\store_parser\run.py", line 318, in main
process.map(media_process, batch)
File "D:\Programy\Python36-32\lib\multiprocessing\pool.py", line 266, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "D:\Programy\Python36-32\lib\multiprocessing\pool.py", line 644, in get
raise self._value
UnboundLocalError: local variable 'file_to_write' referenced before assignment
It look like, there is an error with variable assignment, but it is not:
try:
file_to_write = open(target_file, 'w')
except OSError:
message = 'OSError while writing file name - {}'.format(target_file)
log_error(message)
except UnboundLocalError:
message = 'UnboundLocalError while writing file name - {}'.format(target_file)
log_error(message)
except Exception as e:
message = 'Total failure "{}" while writing file name - {}'.format(e, target_file)
log_error(message)
else:
file_to_write.write(temp)
file_to_write.write(temp2)
finally:
file_to_write.close()
Line - except Exception as e:, does not help with anything, the whole thing still crashes. So far i have excluded only Out Of Memory scenario, because this script is designed to be handled on low spec VPS, but in testing stage I run it in environment with 8 GB of ram. So if You have any theories please share.
The exception really says what is happening.
This part is telling you obvious issue:
UnboundLocalError: local variable 'file_to_write' referenced before assignment
Even you have try/except blocks that catches various exceptions, else/finally doesn't.
More specifically in finally block you reference variable that might not exist since exception with doing: file_to_write = open(target_file, 'w') is being handled by at least last except Exception as e block, but then finally is run too.
Since exception happened as a result of not being able to open target file, you do not have anything assigned to file_to_write and that variable doesn't exist after exception is handled. That is why finally block crashes.
I wrote a python script to automate some tasks on my mail account. So, I can login to my yahoo mail account, read, delete (via imap) and send emails (via smtp). After that I want to logout. However, I am getting this error, which I do not know how to repair:
This is what happens:
Exception in close_imap
Traceback (most recent call last):
File "/Users/Tom/MeineDaten/Programmieren/Sportwetten/Tipico/Report-Gambling-Apps/emailing/Mailer.py", line 55, in close_imap
self.imap.close()
File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/imaplib.py", line 445, in close
typ, dat = self._simple_command('CLOSE')
File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/imaplib.py", line 1180, in _simple_command
return self._command_complete(name, self._command(name, *args))
File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/imaplib.py", line 928, in _command
', '.join(Commands[name])))
imaplib.IMAP4.error: command CLOSE illegal in state LOGOUT, only allowed in states SELECTED
And this is the corresponding code:
import imaplib
...other code here...
def close_imap(self):
if self.imap is None:
print("close_imap: self.imap is None. No further action taken. Returning.")
return
try:
self.imap.close()
self.imap.logout()
except Exception:
print("Exception in close_imap")
print(str(traceback.format_exc()))
pass
What am I doing wrong here?
You are trying to close a folder but you have not selected any.
This may be due to two thinks:
You never selected a folder in this session
You already closed the folder but you are trying to close it a second time.
You should only close a folder one time.
Note: This is not necessary in the IMAP protocol to close a folder before logging out.
Given the following code:
# Edit build number in test report
print(path) # TODO remove
html_report = fileinput.input(path, inplace=True)
for line in html_report:
print(line.replace('$BUILD_NUMBER',
args.number).rstrip())
html_report.close()
I get the following output:
/home/jenkins/workspace/reports/report201610261053.html
Traceback (most recent call last):
File "report_generator.py", line 58, in <module>
for line in html_report:
File "/usr/lib/python2.7/fileinput.py", line 252, in next
line = self.readline()
File "/usr/lib/python2.7/fileinput.py", line 321, in readline
os.rename(self._filename, self._backupfilename)
OSError: [Errno 2] No such file or directory
If I just use the command:
gedit /home/jenkins/workspace/reports/report201610261053.html
I can check that the file exists. In fact, if it didn't I would expect this error to be raised in the fileinput.input() line, not in the line loop.
Any idea of what's wrong?
What is your "path" value?
I think you should try to use absolute path
You can also check the user permissions to file.
I don't see anything wrong in the code that you have shown.
I can check that the file exists. In fact, if it didn't I would expect
this error to be raised in the fileinput.input() line, not in the line
loop.
The error is reported only when an attempt is made to open file and it happens in the for loop.
Is it possible that your code is running under a different user and doesn't see the file on that path as compared to you manually verifying file existence?
I'm trying to write a quick python script to grab some logs with sftp. My first inclination was to use Pysftp, since it seemed like it made it very simple. It worked great, until it got to a larger file. I got an error while getting any file over about 13 MB. I then decided to try writing what I needed directly in Paramiko, rather than relying on the extra layer of Pysftp. After figuring out how to do that, I ended up getting the exact same error. Here's the Paramiko code, as well as the trace from the error I get. Does anyone have any idea why this would have an issue pulling any largish files? Thanks.
# Create tranport and connect
transport = paramiko.Transport((host, 22))
transport.connect(username=username, password=password)
sftp = paramiko.SFTPClient.from_transport(transport)
# List of the log files in c:
files = sftp.listdir('c:/logs')
# Now pull them, logging as you go
for f in files:
if f[0].lower() == 't' or f[:3].lower() == 'std':
logger.info('Pulling {0}'.format(f))
sftp.get('c:/logs/{0}'.format(f), output_dir +'/{0}'.format(f))
# Close the connection
sftp.close()
transport.close()
And here's the error:
No handlers could be found for logger "paramiko.transport"
Traceback (most recent call last):
File "pull_logs.py", line 420, in <module> main()
File "pull_logs.py", line 410, in main
pull_logs(username, host, password, location)
File "pull_logs.py", line 142, in pull_logs
sftp.get('c:/logs/{0}'.format(f), output_dir +'/{0}'.format(f))
File "/Users/me/my_site/site_packages/paramiko/sftp_client.py", line 676, in get
size = self.getfo(remotepath, fl, callback)
File "/Users/me/my_site/site_packages/paramiko/sftp_client.py", line 645, in getfo
data = fr.read(32768)
File "/Users/me/my_site/site_packages/paramiko/file.py", line 153, in read
new_data = self._read(read_size)
File "/Users/me/my_site/site_packages/paramiko/sftp_file.py", line 152, in _read
data = self._read_prefetch(size)
File "/Users/me/my_site/site_packages/paramiko/sftp_file.py", line 132, in _read_prefetch
self.sftp._read_response()
File "/Users/me/my_site/site_packages/paramiko/sftp_client.py", line 721, in _read_response
raise SSHException('Server connection dropped: %s' % (str(e),))
paramiko.SSHException: Server connection dropped: