I encountered a small annoyance with this code:
try:
return await asyncio.wait_for(tcp_command(cmd), timeout=timeout)
except (OSError, asyncio.TimeoutError) as err:
print(f"Network problem: {err}")
When the timeout occurs, it prints just "Network problem: ". It is caused by an empty value attached to the raised asyncio.TimeoutError:
# inside wait_for():
raise futures.TimeoutError()
It is easy to hadle the TimeoutError separately, but I find the original construct quite idiomatic and now a core library breaks it. Is there a good reason for it? Is my assumption - that printing an exception should give us a clue what went wrong - correct?
Is there a good reason for it?
Yes, what kind of message you expect from TimeoutError? "Timeout occured"? The exception itself is self-explanatory, no need for such redundancy.
Is my assumption - that printing an exception should give us a clue what went wrong - correct?
Yes and no. Clue? Yes. Full information? No. The exception message is not mandatory. And the type of an exception is an important piece of information as well. And in many cases even more then the message itself.
So first of all: using print is wrong to begin with. Python has a very rich logging support. For example logger.exception(str(exc)) solves your problem because it logs entire traceback in addition to the message. At least by default, it can be customized.
But if you still want to use print then consider logging whole traceback:
import traceback
# traceback.print_exc()
print(traceback.format_exc())
If whole traceback is too big then you can always simply print the exception's class name:
# print(f'[{type(exc).__name__}] {exc}')
print(f'[{type(exc)}] {exc}')
or customize by exception:
try:
return await asyncio.wait_for(tcp_command(cmd), timeout=timeout)
except OSError as err:
print(f"Network problem: {err}")
except asyncio.TimeoutError:
print('Timeout occured')
The expectation that an exception will provide a message that explains the issue is not part of the general exception contract in Python. It is true for system exceptions such as OSError where the program must be able to get to the error message provided by the operating system, as the program is not qualified to guess the message based on a code or an exception subtype.
But more basic language exceptions do not work like that. Take, for example, KeyError raised by dict.__getitem__:
>>> try:
... d[123]
... except KeyError as err:
... print(f"Dict problem: {err}")
...
Dict problem: 123
In this sense, TimeoutError is much more like KeyError than like OSError. When you catch TimeoutError, you know exactly what happened - a timeout. You typically want to do something based on the fact that a timeout happened, rather than just display a message to the user. And even if you did want to provide a message, you'd use one that would make sense for your application, not a generic one provided by Python. This is in contrast to OSError where you often cannot do anything other than display the message coming from the OS and where that message can prove invaluable for investigating the underlying issue.
To sum it up, the problem is that you are catching two fundamentally different exceptions in the same except clause, and that set you up for trouble. I would restructure the code like this:
try:
return await asyncio.wait_for(tcp_command(cmd), timeout=timeout)
except OSError as err:
print(f"Network problem: {err}")
except asyncio.TimeoutError:
print("Operation timed out")
Related
I'm currently coding my first larger script which is a console based GUI where the user selects options by entering numbers to start several tasks:
I've recently implemented a lot of error handling to prevent the window from closing if something goes wrong in the background. I'm a little bit confused whether or not my approach is correct.
The basic structure of my code is as following:
There is a function read_excel() which loads some Excel files with pandas:
def read_excel(excel_path):
try:
df = pd.read_excel(excel_path, encoding="utf-8")
except FileNotFoundError:
raise FileNotFoundError('Unable to load assignment file, maybe choose custom')
return
# do stuff....
if not all(len(x) == len(signalnames) for x in [frequencies, cans]):
raise ValueError('Frequency and can number must be given for every signal in assignment file!')
else:
logging.info("Successfully loaded assignment file")
return signalnames, frequencies, group_names, cans, can_paths
This function is then used together with others in the function ft_14() which is called by the GUI:
def ft_14(files, draft assignement_path):
try:
signalnames, frequencies, group_names, cans, can_paths = read_excel(assignement_path)
except (ValueError, FileNotFoundError) as e:
logging.error(e)
return
# do stuff..
try:
wb.save(os.path.join(os.path.dirname(files[0]), "FT.14_results.xlsx"))
wb.close()
except Exception:
logging.error('Unable to save report excel')
So my attempt is to raise exceptions in the backend and then except them in functions which are called by the GUI and use logging to display them for the user. So my question is if this approach is the correct way to use exceptions and logging together or if there is a smarter way, because calling:
try:
# some function()
except Exection as e:
logging.error(e)
doesn't feel right to me.
The code you posted at the end, which makes you uncomfortable, is correct in that it does what it says it will do. If there is an exception, it logs the error.
The concern is that it also handles the exception. In many cases, you don't want to change anything else in the exception handling process -- you just want to log, and let the exception propagate normally.
https://docs.python.org/3/tutorial/errors.html#raising-exceptions
import logging
logger = logging.getLogger(__name__)
def fn(x):
try:
return x / 0
except Exception as e:
logger.error(str(e))
raise e
print ("let's do something risky")
try:
fn(20)
except Exception as e:
pass
print ("it has been done")
Notice how fn(x) can detect and handle exceptions, then re-raise them as if it had not done anything to the exception at all? You do that with "raise" and no arguments.
In your first code example, you catch an exception, and then raise a completely different exception, which happens to be of the same type:
except FileNotFoundError:
raise FileNotFoundError('Unable to load assignment file, maybe choose custom')
This can be fair game if you want to hide info, for example if the exception has internal details you do not want to expose to the outside. But you don't know what info you lost by replacing the exception entirely with a new instance. But none of those is "right" or "wrong," they're just choices you make with how to handle exceptions. (edit: it is good practice to handle them eventually or document what it can throw, but we're off topic)
None of that matters to the logger. You could have logged it right there and re-raised. You could log the original exception, then raise the sanitized version. You could handle it there, log it, and not raise (as the example you posted). The logger doesn't care what you do with the exception. If you want it logged, log it immediately.
I have a Python script running on a schedule and I have set up a try/except block to catch all exceptions and email me the exception message so that I can be made aware that something has gone wrong.
I am observing some odd behavior. For every execution of my script, an Exception is being caught where the message is simply:
93
This content is the same when I print it to the console as well.
When I remove the try/except block, there aren't any exceptions being thrown by my code. Does anyone know why the presence of the try/except blocks are causing exceptions and why it is just returning digits?
try:
#do something
except Exception as ex:
s = smtplib.SMTP('000.000.000.000',25)
s.starttls()
s.sendmail('email#email.com', 'email#email.com', 'General Exception in final_parse method ' + str(ex.message))
print str(ex.message)
s.quit()
It depends on what you're doing in your try. The exception is created and raised in that code. It is possible to set an exception's message to 93, or to anything else.
Try evaluating the entire exception in your debugger, or stepping through the code you're running in try until you get the the exception, and look at the context in which it is created/raised. This will at least give you an idea of what the code/module was doing when it failed.
My guess is that the code running in try isn't well written or maintained. Try a different module, or post more details about it here.
Update:
If your stack trace is giving you KeyError, then the code you're executing is throwing a KeyError exception that isn't handled by an except block deeper down in the code, which would mean it's either poorly written or being poorly used. The issue is most likely within your 'do something'
Rather than use smtplib.SMTP you should use Yagmail. It's easier to use:
import yagmail
yag = yagmail.SMTP(username, password)
yag.send(emailto, subject = "I now can send an attachment", contents = fileToSend)
In Python, is there any (proper) way to change the the default exception handling behaviour so that any uncaught exception will terminate/exit the program?
I don't want to wrap the entire program in a generic try-except block:
try:
// write code here
except Exception:
sys.exit(1)
For those asking for more specificity and/or claiming this is already the case, it's my understanding that not all Python exceptions are system-exiting: docs
Edit: It looks like I have forked processes complicating matters so won't be posting any specific details about my own mess.
If you're looking for an answer to the original question, Dmitry's comment is interesting and useful, references the 2nd answer to this question
You can use Specific exception instead of Exception because Exception is a Base class for all exceptions. For more details refer Exception tutorial
You can write your script like this-
try:
# write code here
except OverflowError:
raise SystemExit
except ArithmeticError:
sys.exit()
except IOError:
quit()
Try this different approaches to find what is exactly you are missing.
Edit 1 - Maintain Program Execution
In order to maintain your program execution try this one-
consider error_handler function is raising SystemExit exception then In your main method you need to add below code so you can maintain your program execution.
try:
error_handler()
except SystemExit:
print "sys.exit was called but I'm proceeding anyway (so there!-)."
For example i have a program with this structure:
Domain logic module -> Settings module -> Settings store backend
Next is a part of Settings module.
def load_from_json(self, json_str):
try:
self.load_from_dict(json.loads(json_str))
except ValueError as e:
raise SettingsLoadDataException('Error loading json')
Need I a custom exception SettingsLoadDataException here, or I could just skip catching json.loads errors?
def load_from_json(self, json_str):
self.load_from_dict(json.loads(json_str))
Update.
Also good variant is:
def load_from_json(self, json_str):
try:
self.load_from_dict(json.loads(json_str))
except ValueError as e:
raise ValueError('Error loading json')
That is a problem only you can answer. You could catch all exceptions, or you could let the program crash if it throws an exception you don't handle. If it is vital that the program doesn't crash, catch the exception. However, you should implement a recovery method then. If the Json doesn't load properly, can your program do anything useful without it ? If it can, I would catch the exception, otherwise you could just display an error and terminate.
You should work with exceptions in such a way, that seeing a stack trace explains the problem to you immediately.
I am no Python expert, but won't you loose the piece of information that it was actually ValueError, that caused program crash? You will see only SettingsLoadDataException in a trace without any real reason of it, right?
Also, if you do not rethrow exceptions, you should catch only those, you know how to deal with. It is always better to have your program crash, than to leave it in an unexpected state.
I'm writing a python program the accesses a database. I want to catch three types of exceptions when I make a http request. Timeouts, network errors, and http errors. I'm looking for the best way to deal with this situation. I need to check for these exceptions multiple times in multiple areas of my code, and it will look something like this each time:
try:
//some request
except timeout:
print '\nException: Timeout Error'
except connection error:
print '\nException: Network Error'
except http error, e:
print 'Exception: %s.' % e
Since I have to do this multiple times, at least probably 8 or more, should I make a module to handle these exceptions or no? Also in which of these cases would it be advisable to shut my system down as opposed to just displaying a message?
Thank you.
If you don't want to use decorators, you can also combine all the except statements, and use some function to handle your exception (assuming that your errors are called TimeoutError, ConnectionError, and HttpError...doesn't really matter for clarity) i.e.
try:
# do stuff
except (TimeoutError, ConnectionError, HttpError) as e:
handle_exception(e)