Is anyone using anything like this in Python:
def die(error_message):
raise Exception(error_message)
...
check_something() or die('Incorrect data')
I think this kind of style is used in PHP and Perl.
Do you find any (dis)advantages in this [style]?
Well, first, sys.exit([arg]) is more common, and if you really wanted something equivalent to die in PHP, you should use that, raise a SystemExit error, or call os._exit.
The major use of the die method in PHP is, "The script has reached some impasse cannot recover from it". It is rarely, if ever, used on production code. You are better off raising an exception in a called function, catching it in the parent, and finding a graceful exit point -- that is the best way in both languages.
Lot's of good answers, but no-one has yet suggested the obvious way to write this in Python:
assert check_something(), "Incorrect data"
Just be aware that it won't do the check if you turn on optimisation, not that anyone ever does.
While that style is common in PHP and Perl, it's very un-Pythonic and I'd encourage you not to write Python that way. You should follow the conventions in the language you're using, and write something like this:
if not check_something():
raise Exception('Incorrect data')
FWIW, doing the "or die(...)" way adds another level to your stack trace, which is another minor disadvantage.
The biggest disadvantage is that all dying is now the same. Better to have check_something() raise a more accurate exception and then catch that up above if appropriate.
If you are dealing with an API that you didn't write that returns truthy values on success and falsy values on failure, that seems like a reasonably readable and compact way to do it. If you have control over the API, I'd encourage you to use exceptions instead of return values to indicate errors.
If you use the function, it probably should not be called die() unless it actually exits the program, however. If it merely raises an exception, there's no guarantee that the program will actually die. Ideally you could name it raise() as a functional version of the raise statement, but of course you can't because raise is a reserved word. Perhaps throw().
It would also be a good idea to require the caller to pass in an exception type, since Exception is rather generic and vague.
It occurs to me that this function would be unnecessary if only Python exceptions were capable of raising themselves, i.e., they had a method for it, like so:
class BaseException(object):
def throw(self):
raise self
Then you could just do:
check_something() or Exception("check failed").throw()
Sadly, Python exceptions can't raise themselves. :-)
It seems like you are just wrapping php lingo in python with a one line function. I would advise against it as you might confuse your audience. Exceptions are also a completely different beast than die in PHP.
Ideally, you would want to die/panic with a helpful stack-trace so that you can easily locate the issue.
Hence, unlike the most popular answer, you should AVOID sys.exit() or raise a SystemExit because they will cause your program to die silently(without a helpful trace).
You should use a general exception like RuntimeError that is most likely not going to be handled/caught later, and it will create a beautiful stack-trace:
def die(message):
raise RuntimeError(message)
print("hello world")
die('A helpful message')
print("hello world again")
We get the following helpful stack trace:
hello world
Traceback (most recent call last):
File "/test.py", line 9, in <module>
die("A helpful message")
File "/test.py", line 5, in die
raise RuntimeError(message)
RuntimeError: A helpful message
If I had used sys.exit instead, I would have gotten the following poor output:
import sys
def die(message):
sys.exit(message) # internally raises SystemExit
print("hello world")
die("A helpful message")
print("hello world again")
hello world
A helpful message
Related
Since there are a bunch of questions here on Stack Overflow that deal with SyntaxError in Python, we might want to know:
How do we tackle a SyntaxError? Are there strategies that can generally be applied?
0. Before the Error appears: Syntax Highlighting and Code Formatting
Even before running into a SyntaxError, there are important measurements to deal with SyntaxErrors, because the best way to deal with SyntaxErrors is to avoid them in the first place. This can be done first and foremost by using an editor or an Integrated Development Environment (IDE) which has syntax highlighting for Python.
Besides that, we can decrease the risk of running into a SyntaxError by good code and formatting style. There is a formal definition of the term "good formatting style", PEP 8 -- Style Guide for Python Code. Proper formatting makes our code much more readable which decreases the risk writing code that leads to a SyntaxError.
A very good way to apply good formatting to our code is to use an automatic code formatting tool. A code formatter has multiple advantages, amongst which are the following: Its code formatting is consistent. It applies best practices you might not even have thought of yet. It is very convenient.
For Python, black is a great code formatting tool.
1. Understand the Error Message
The Syntax Error indicates in which file and in which line the interpreter came across a problem in our code. We should use this information to find the bug.
We should be aware that the Python interpreter sometimes indicates a SyntaxError in the line after the actual problem. This is because the parser expects something in the erroneous line and can recognise that this is missing only when the whole line has been parsed. The prototypic example for that kind of SyntaxError is a missing parenthesis. So for instance, the following code raises a SyntaxError in line 2, even though the bug is in line 1:
bar = foo(
baz()
EOL stands for "End Of Line". This helps understanding the very common SyntaxError: EOL while scanning string literal. This is usually raised when you did not properly close a string definition with closing quotation marks, such as in the following example:
foo = "bar
2. Simplify the Code
Generally, a good strategy of bug fixing is to reduce any code that throws an Error or an Exception (or that does not return the expected output) to a minimal example. (This is a requirement for questions here on Stack Overflow, but much more than this, it is a good technique for pinning down a bug.)
In case of a SyntaxError, producing a minimal example is usually very easy, because a SyntaxError does not depend on any values of a variable, or any state of an object or any other semantics of your code. That's why the source of a SyntaxError is usually one line of code.
So, to identify the bug, we remove all the code besides the line that we think is the source of the Error. If the Error vanishes, it has been in a different line. If the Error persists, we try to simplify this line. For instance, we replace nested parentheses, by defining intermediate variables that hold the values:
Instead of
bar = foo(foo(baz(foo()))
the following (logically equivalent) code:
first = foo()
second = baz(first)
third = foo(second)
bar = foo(third
makes it much easier for us to identify the missing closing parenthesis.
Refer to documentation. Syntax Errors unfortunately cannot be captured in a Try: Except: block, so the only way to deal with them is to read the message returned, and if that doesn't help, following up with the python documentation:
https://docs.python.org/3/
Consider the following code:
try:
if True a = 1 #It's missing a colon So it's a SyntaxError!!!!!!!
except SyntaxError:
print 'hey'
You'd expect it to print hey However It raises a SyntaxError, The same error I'm trying to avoid. So Can all Exceptions be handled using a try-except block? Well If SyntaxError's were an exception why is it included in the built-in exceptions? and finally how can I fix the above piece of code so that it handles the exception properly?
Note: I know what I'm trying to do Is utterly pointless and serves no real purpose
SyntaxError is a perfectly ordinary built-in exception. It is not special in any way. Only the circumstances of when it's (usually) thrown are a bit unusual.
A syntax error means that the code featuring said error cannot be parsed. It doesn't even begin to be a valid program, hence it cannot be executed. Therefore SyntaxError exceptions are raised before the program is run, and hence can't be caught from within the program.
More specifically, this exception is raised by the parser. Because the parser runs fully before the code is executed, rather then interleaved with it, a program can't catch its own syntax errors.
The parser itself is just another program though: Code invoking the parser can catch SyntaxErrors like every other exception (because it is like every other exception). Examples of "invoking the parser" include:
compile, exec, eval
import statements
Several functions in modules like ast, tokenizer, parser, etc.
Of course you need SyntaxError as a built-in exception - what else should be raised if the compiler/parser encounters a syntax error?
You're right that this error usually happens at compile time, which is before you're able to catch it (runtime). (And how would you recover from it?)
I can think of one exception, though:
>>> try:
... eval("hello =")
... except SyntaxError:
... print("Hey! Who's using eval() anyway??")
...
Hey! Who's using eval() anyway??
Is there a neat way to inject failures in a Python script? I'd like to avoid sprinkling the source code with stuff like:
failure_ABC = True
failure_XYZ = True
def inject_failure_ABC():
raise Exception('ha! a fake error')
def inject_failure_XYZ():
# delete some critical file
pass
# some real code
if failure_ABC:
inject_failure_ABC()
# some more real code
if failure_XYZ:
inject_failure_XYZ()
# even more real code
Edit:
I have the following idea: insert "failure points" as specially-crafted comments. The write a simple parser that will be called before the Python interpreter, and will produce the actual instrumented Python script with the actual failure code. E.g:
#!/usr/bin/parser_script_producing_actual_code_and_calls python
# some real code
# FAIL_123
if foo():
# FAIL_ABC
execute_some_real_code()
else:
# FAIL_XYZ
execute_some_other_real_code()
Anything starting with FAIL_ is considered as a failure point by the script, and depending on a configuration file the failure is enabled/disabled. What do you think?
You could use mocking libraries, for example unittest.mock, there also exist many third party ones as well. You can then mock some object used by your code such that it throws your exception or behaves in whatever way you want it to.
When testing error handling, the best approach is to isolate the code that can throw errors in a new method which you can override in a test:
class ToTest:
def foo(...):
try:
self.bar() # We want to test the error handling in foo()
except:
....
def bar(self):
... production code ...
In your test case, you can extend ToTest and override bar() with code that throws the exceptions that you want to test.
EDIT You should really consider splitting large methods into smaller ones. It will make the code easier to test, to understand and to maintain. Have a look at Test Driven Development for some ideas how to change your development process.
Regarding your idea to use "Failure Comments". This looks like a good solution. There is one small problem: You will have to write your own Python parser because Python doesn't keep comments when it produces bytecode.
So you can either spend a couple of weeks to write this or a couple of weeks to make your code easier to test.
There is one difference, though: If you don't go all the way, the parser will be useless. Also, the time spent won't have improved one bit of your code. Most of the effort will go into the parser and tools. So after all that time, you will still have to improve the code, add failure comments and write the tests.
With refactoring the code, you can stop whenever you want but the time spent so far will be meaningful and not wasted. Your code will start to get better with the first change you make and it will keep improving.
Writing a complex tool takes time and it will have it's own bugs which need to fix or work around. None of this will improve your situation in the short term and you don't have a guarantee that it will improve the long term.
If you only want to stop your code at some point, and fall back to interactive interpreter, one can use:
assert 1==0
But this only works if you do not run python with -O
Edit
Actually, my first answer was to quick, without really understanding what you want to do, sorry.
Maybe your code becomes already more readable if you do parameterization through parameters, not through variable/function suffices. Something like
failure = {"ABC": False, "XYZ":False}
#Do something, maybe set failure
def inject_failure(failure):
if not any(failure.values()):
return
if failure["ABC"]:
raise Exception('ha! a fake error')
elif failure["XYZ"]:
# delete some critical file
pass
inject_failure(failure)
The Python standard library and other libraries I use (e.g. PyQt) sometimes use exceptions for non-error conditions. Look at the following except of the function os.get_exec_path(). It uses multiple try statements to catch exceptions that are thrown while trying to find some environment data.
try:
path_list = env.get('PATH')
except TypeError:
path_list = None
if supports_bytes_environ:
try:
path_listb = env[b'PATH']
except (KeyError, TypeError):
pass
else:
if path_list is not None:
raise ValueError(
"env cannot contain 'PATH' and b'PATH' keys")
path_list = path_listb
if path_list is not None and isinstance(path_list, bytes):
path_list = fsdecode(path_list)
These exceptions do not signify an error and are thrown under normal conditions. When using exception breakpoints for one of these exceptions, the debugger will also break in these library functions.
Is there a way in PyCharm or in Python in general to have the debugger not break on exceptions that are thrown and caught inside a library without any involvement of my code?
in PyCharm, go to Run-->View Breakpoints, and check "On raise" and "Ignore library files".
The first option makes the debugger stop whenever an exception is raised, instead of just when the program terminates, and the second option gives PyCharm the policy to ignore library files, thus searching mainly in your code.
The solution was found thanks to CrazyCoder's link to the feature request, which has since been added.
For a while I had a complicated scheme which involved something like the following:
try( Closeable ignore = Debugger.newBreakSuppression() )
{
... library call which may throw ...
} <-- exception looks like it is thrown here
This allowed me to never be bothered by exceptions that were thrown and swallowed within library calls. If an exception was thrown by a library call and was not caught, then it would appear as if it occurred at the closing curly bracket.
The way it worked was as follows:
Closeable is an interface which extends AutoCloseable without declaring any checked exceptions.
ignore is just a name that tells IntelliJ IDEA to not complain about the unused variable, and it is necessary because silly java does not support try( Debugger.newBreakSuppression() ).
Debugger is my own class with debugging-related helper methods.
newBreakSuppression() was a method which would create a thread-local instance of some BreakSuppression class which would take note of the fact that we want break-on-exception to be temporarily suspended.
Then I had an exception breakpoint with a break condition that would invoke my Debugger class to ask whether it is okay to break, and the Debugger class would respond with a "no" if any BreakSuppression objects were instantiated.
That was extremely complicated, because the VM throws exceptions before my code has loaded, so the filter could not be evaluated during program startup, and the debugger would pop up a dialog complaining about that instead of ignoring it. (I am not complaining about that, I hate silent errors.) So, I had to have a terrible, horrible, do-not-try-this-at-home hack where the break condition would look like this: java.lang.System.err.equals( this ) Normally, this would never return
true, because System.err is not equal to a thrown exception, therefore the debugger would never break. However, when my Debugger class would get initialized, it would replace System.err with a class of its own,
which provided an implementation for equals(Object) and returned true if the debugger should break. So, essentially, I was using System.err as an eternal global variable.
Eventually I ditched this whole scheme because it is overly complicated and it performs very bad, because exceptions apparently get thrown very often in the java software ecosystem, so evaluating an expression every time an exception is thrown tremendously slows down everything.
This feature is not implemented yet, you can vote for it:
add ability to break (add breakpoint) on exceptions only for my files
There is another SO answer with a solution:
Debugging with pycharm, how to step into project, without entering django libraries
It is working for me, except I still go into the "_pydev_execfile.py" file, but I haven't stepped into other files after adding them to the exclusion in the linked answer.
What is meant by "using the EAFP principle" in Python? Could you provide any examples?
From the glossary:
Easier to ask for forgiveness than permission. This common Python coding style assumes the existence of valid keys or attributes and catches exceptions if the assumption proves false. This clean and fast style is characterized by the presence of many try and except statements. The technique contrasts with the LBYL style common to many other languages such as C.
An example would be an attempt to access a dictionary key.
EAFP:
try:
x = my_dict["key"]
except KeyError:
# handle missing key
LBYL:
if "key" in my_dict:
x = my_dict["key"]
else:
# handle missing key
The LBYL version has to search the key inside the dictionary twice, and might also be considered slightly less readable.
I'll try to explain it with another example.
Here we're trying to access the file and print the contents in console.
LBYL - Look Before You Leap :
We might want to check if we can access the file and if we can, we'll open it and print the contents. If we can't access the file we'll hit the else part. The reason that this is a race condition is because we first make an access-check. By the time we reach with open(my_file) as f: maybe we can't access it anymore due to some permission issues (for example another process gains an exclusive file lock). This code will likely throw an error and we won't be able to catch that error because we thought that we could access the file.
import os
my_file = "/path/to/my/file.txt"
# Race condition
if os.access(my_file, os.R_OK):
with open(my_file) as f:
print(f.read())
else:
print("File can't be accessed")
EAFP - Easier to Ask for Forgiveness than Permission :
In this example, we're just trying to open the file and if we can't open it, it'll throw an IOError. If we can, we'll open the file and print the contents. So instead of asking something we're trying to do it. If it works, great! If it doesn't we catch the error and handle it.
# # No race condition
try:
f = open(my_file)
except IOError as e:
print("File can't be accessed")
else:
with f:
print(f.read())
I call it "optimistic programming". The idea is that most times people will do the right thing, and errors should be few. So code first for the "right thing" to happen, and then catch the errors if they don't.
My feeling is that if a user is going to be making mistakes, they should be the one to suffer the time consequences. People who use the tool the right way are sped through.