I wrote a fibonacci function that infinitely recurses, and while python couldn't detect it and threw errors upon hitting the max recursion limit, when I used try and assert to see if fib(30) was equal to some value, it immediately told me it was not. How did it do this? It seems like it didn't even need to run fib(30).
note:
I realize this only works if I do
try:
assert infiniteFib(30) == 832040
except:
print "done immediately"
When I do just the assert, it produces many errors about too many recursions, but with the try it stops at the first error.
What I'm curious is how does python manage to produce an error so quickly about infinite recursion? Wouldn't it need to hit the limit (which takes a long time) to tell whether it was infinite?
EDIT:
Some requested code, but just to be clear, I DON'T want a solution to the errors (I know it's wrong because I deliberately excluded the base case), I want to know how python produces errors so quickly when it should take much longer (if you do fib(30), clearly it takes a while to hit the max recursion limit, but somehow python produces errors way before then):
def fib(n):
return fib(n-1) + fib(n-2)
try: assert(fib(30) == 832040)
except: print "done immediately"
The reason the code you've shown runs quickly is because it catches the exception that is raised by fib when it hits the recursion limit and doesn't print the traceback. Running to the recursion limit doesn't take very long at all, but formatting and printing hundreds of lines of traceback does.
If you inspect the exception you get, you'll see it is the same RuntimeError you get when you run fib normally, not an AssertionError. Try this, to see what is going on better:
try:
assert(fib(30) == 832040)
except Exception as e:
print("Got an Exception: %r" % e)
It's not done immediately. Your code runs until python reaches maximum recursion depth and maximum recursion depth is set to 1000 by default in python to avoid stack overflow errors.
So, actually your code runs till it reaches recursion depth of 1000 and errors out RuntimeError: maximum recursion depth exceeded. You can verify this by modifying your code as below:
i=0
def fib(n):
global i
i = i + 1
print i
return fib(n-1) + fib(n-2)
assert(fib(30) == 832040)
print i
print "done immediately"
In my machine, i am getting the last i value as 984 before errors out.
Related
The code
def Recursion():
try:
raise RecursionError()
except RecursionError:
Recursion()
Recursion()
crashes.
I am aware that this is a recursive function, but shouldn't Python just raise an StackError, instead of just crashing? If someone could explain this, I would be very grateful!
Something similar has been reported as a bug: https://bugs.python.org/issue42509, with someone commenting that
Catching a RecursionError might be fine sometimes, but the issue is that Program 1 catches a RecursionError and then keeps recursing more rather than stopping.
I think it might have to be the responsibility of the Python user to make sure that if a RecursionError is to be caught, that the program can recover without making things much worse. It's my understanding that the extra buffer of +50 is to make sure that the programmer has room to stop the overflow and do any necessary cleanup [1].
If no attempt is made to clean up, then it seems reasonable to me that Python should crash, unless there's some idea of what could happen that I'm missing. The interpreter could allow arbitrary recursion during the cleanup until the C stack overflows, but that sort of defeats the point of the recursion checker. It could raise some new ExtraSuperRecurionError, but that doesn't fix anything: what if that error is caught ;) ?
https://bugs.python.org/msg382128
Assuming this is a minimally reproducible toy example for a real issue you're facing, you should do:
def Recursion():
try:
raise Exception()
except RecursionError:
raise
except Exception:
Recursion()
Recursion()
The RecursionError that you see is not the one you manually raise. It does not occur within the try-block and therefore is not caught:
def Recursion():
try:
raise RecursionError() # does not happen here ...
except RecursionError:
Recursion() # ... but here where it goes unhandled
I have a scenario where a function calls itself during error handling if a certain condition is met. Something like:
def test(param):
# code
try:
# attempt
except ValueError as e:
if (x):
test(param+1)
I noticed that if I get stuck in a loop of excepts and try to cancel with my keyboard, I get a giant stacktrace. This doesn't seem right.
Is there a better way to handle this?
Edit:
After running this for a while, I got:
RecursionError: maximum recursion depth exceeded while calling a Python object
I am not sure it is related, but I would imagine a recursion depth issue would arise from too many recursive function calls?
Here's one way to repeat your operation on failure, without using recursion.
def dostuff(param):
while True:
# code
try:
# attempt
except ValueError:
if x:
param += 1
continue
break
This way, if attempt is successful, the loop will break. But if it raises a ValueError, and if your x condition (whatever that is) is true, then the body of the loop will be repeated with param incremented by 1.
I guess I'm not the first asking this question, but I haven't found a solution that I could use/understand yet. And the issue is probably not as simple as i first expected.
I think it can be boiled down to two general questions:
1) Is there a way to avoid Python to stop when an error occur and just jump on to the next line of code in the script?
2) Is there a way to make Python execute a line of code if an error occurs? Like, if error then...
My concrete problem:
I have a very large program with a lot of functions and other stuff, which would take forever to adjust individually by using "try" for example (if i understand it correctly)
My program run as a large loop that gather information and keeps running. This means that it does not really matter to me, that my program fails multiple time as long as it keeps running. I can easily handle that some of the information is with error and would just like my program to take a note of it and keep going.
Is there a solution to this?
As you rightly pointed out, the try/catch block in Python is by far your best ally:
for i in range(N):
try: do_foo() ; except: do_other_foo()
try: do_bar() ; except: do_other_bar()
Alternatively, you could also use, in case you didn't need the Exception:
from contextlib import suppress
for i in range(N):
with suppress(Exception):
do_foo()
with suppress(Exception):
do_bar()
Your only possibility is to rely on the try/except clause. Keep in mind that the try/except may use also finally and else (see documentation:
try:
print("problematic code - error NOT raised")
except:
print("code that gets executed only if an error occurs")
else:
print("code that gets executed only if an error does not occur")
finally:
print("code that gets ALWAYS executed")
# OUTPUT:
# problematic code - error NOT raised
# code that gets executed only if an error does not occur
# code that gets ALWAYS executed
or, when an error is raised:
try:
print("problematic code - error raised!")
raise "Terrible, terrible error"
except:
print("code that gets executed only if an error occurs")
else:
print("code that gets executed only if an error does not occur")
finally:
print("code that gets ALWAYS executed")
# OUTPUT:
# problematic code - error raised!
# code that gets executed only if an error occurs
# code that gets ALWAYS executed
I urge to point out, by the way, that ignoring everything makes me shiver:
you really should (at least, more or less) identify which exception can be raised, catch them (except ArithmeticError: ..., check built-in exceptions) and handle them individually. What you're trying to do will probably snowball into an endless chain of problems, and ignoring them will probably create more problems!
I think that this question helps to understand what a robust software is, meanwhile on this one you can see how SO community thinks python exceptions should be handled
A python internals inquiry. Can you gurus please help me understand why the following code leads to a "Fatal python error" rather than nicely catching over the recursion limit overflow?
def flowfunc(x):
try:
if x < 0:
flowfunc(x + 3)
flowfunc(x + 5)
else:
flowfunc(x - 1)
except:
print("exception at: x = ", x)
Call it with e.g.:
flowfunc(0)
and all breaks loss with
exception at: x = -1
Fatal Python error: Cannot recover from stack overflow.
...
...
[threads data...]
Seems like it all depends on the 2nd recursive call. The following code will behave nicely:
def flowfunc2(x):
try:
if x < 0:
flowfunc2(x + 3)
# flowfunc2(x + 5)
else:
flowfunc2(x - 1)
except:
print("exception at: x = ", x)
with
flowfunc2(0)
returning (value may change depending on available stack depth):
exception at: x = -1
I'm on python 3.6. Your insight will be appreciated.
Similarity between the two snippets:
You have an oscillating recursive function going between positives and negatives, never really terminating. Once you reach the max recursive depth set by Python, you catch the exception, print the current value.
Difference between the two snippets:
In the first version (where you get the fatal error), after printing the max depth reached exception, instead of winding up your call stack, you just go one level-up and call flowfunc(x + 5) piling up even more calls in the call stack than Python can possibly handle. That's where you get the fatal error. In the second version, after printing the exception you just go straight up returning None to the original caller and terminate the execution of your program. In a way, the printing of exception in the second snippet serves as your base case for recursion since there is nothing else to do in the caller and hence your program terminates gracefully.
So yes, your observation is right, it does depend on the second recursive call.
This is the same error as https://bugs.python.org/issue6028.
And the reason is explained there as well:
"This is normal behaviour, actually. The RuntimeError is raised, but
you catch it in the except clause and then recurse again ad infinitum.
The interpreter realizes that it "cannot recover from stack overflow",
as the message says, and then bails out."
Basically, because of the existence of your second recursive call, the program continues after the error is caught in your first one and it keeps branching and generates infinite RuntimeError.
Is there a way to test when a program's recursion limit is reached? For example:
def funct():
print "Hello"
#Some code for testing the recursion limit.
#Some more code to execute
If you are wondering why I want to know this, here is why: I want to hide an Easter egg in a program, but I can't do that until I found out how to do what I am asking. So, how can I test when the recursion limit is reached? Is there even a way?
You can test for the RuntimeError like any other error, with try:
def funct():
...
try:
funct()
except RuntimeError as e:
if "recursion" in e.message:
print("Easter egg!")
Note that I've added an extra check that the error is warning about recursion, so you don't prevent any other RuntimeErrors from going about their business.