I have a flask python server that runs in a docker container. I have found that once the execution reaches a certain point, the container just crashes, and no exceptions or messages are thrown even if you surround your code in any try-except block.
The problem seems to arise when the code tries to access a certain variable. In my case it crashes when accessing the bar variable:
try:
logger.trace(foo)
logger.trace(bar)
except Exception as e:
logger.error(e)
My code is this simple. Reading foo and logging its value works as expected but the bar variable is never logged and the container crashes. The console only shows this message after logging foo:
Disconnected from container.
This happens whenever I make a request from Postman and reach this part of the code.
The variable bar is supposed to be an instance of a class modeled with Swagger.
What could make Python crash when a variable is read that would not show or throw any exception?
After some research I found out that this can happen when there is an infinite recursion problem.
In my case, at some point in the code the object instance gets assigned to the wrong property resulting in this infinite recursion problem:
from swagger_server.model import ObjectModel
def do_something(bar: ObjectModel) -> ObjectModel:
<some code that modifies bar>
return bar
bar = ObjectModel(
id = 'obj1'
)
bar.id = do_something(bar)
# Here is the problem. The ID now points to the sub_object creating
# infinite recursion but it does not throw an error here
# bar = do_something(bar) # This is the correct way to use the do_something() function
# do_something(bar) # As they're objects we could use this too and will modify the original object
<some more code>
logger.trace(bar) # Accessing the data triggers the crash
I also found a way to find this kinds of errors. Swagger creates automatically a test file for the controller. If the endpoint is tested for that specific case with the test file the error is shown this way:
maximum recursion depth exceeded while calling a Python object
When I tested this case with Postman the error is not thrown, it just crashes.
Related
I have a pytest function as such:
def test_zork1_serial_number_error(zork1_unicode_error_serial):
"handles a serial code with a unicode error"
with pytest.raises(UnicodeDecodeError) as execinfo:
serial_code = zork1_unicode_error_serial.serial_code
assert serial_code == "XXXXXX"
The code that this hits is:
#property
def serial_code(self) -> str:
code_bytes = bytes(self.view[0x12:0x18])
try:
if code_bytes.count(b"\x00"):
print("111111111111")
return "XXXXXX"
return code_bytes.decode("ascii")
except UnicodeDecodeError:
print("222222222222")
return "XXXXXX"
The print statements were just there for me to validate that the appropriate path was being hit. When I run the test I get this:
zork1_unicode_error_serial = <zmachine.header.Header object at 0x10e320d60>
def test_zork1_serial_number_error(zork1_unicode_error_serial):
"handles a serial code with a unicode error"
with pytest.raises(UnicodeDecodeError) as execinfo:
serial_code = zork1_unicode_error_serial.serial_code
> assert serial_code == "XXXXXX"
E Failed: DID NOT RAISE <class 'UnicodeDecodeError'>
tests/header_test.py:42: Failed
------------------------------------------------------------------------------ Captured stdout setup ------------------------------------------------------------------------------
/Users/jnyman/AppDev/quendor/tests/../zcode/zork1-r15-sXXXXXX.z2
------------------------------------------------------------------------------ Captured stdout call -------------------------------------------------------------------------------
222222222222
Notice how the "222222222222" is captured in the standard output, thus the appropriate path is being hit and thus the exception is also clearly being generated. Yet Pytest is saying that this exception was not raised. (I have also tested this code manually as well to make sure the exception is being generated.)
I've also tried the path of instead "marking" the test as such, like this:
#pytest.mark.xfail(raises=UnicodeDecodeError)
def test_zork1_serial_number_error(zork1_unicode_error_serial):
...
And that passes. However, it also passes regardless of what exception I put in there. For example, if I do #pytest.mark.xfail(raises=IndexError) that also passes even though an IndexError is never raised.
I can't tell if this has something to do with the fact that what I'm testing is a property. Again, as can seen from the captured standard output, the appropriate code path is being executed and the exception is most definitely being raised. But perhaps the fact that my function is a property is causing an issue?
I have read this Python - test a property throws exception but that isn't using Pytest and it's unclear to me how to retrofit the thinking there. I also aware that perhaps throwing an exception in a property is not a good thing (referencing this: By design should a property getter ever throw an exception in python?) so maybe this test problem is pointing to a code smell. But I don't see an immediate to make this better without adding extra complication. And that still wouldn't explain why Pytest is not seeing the exception generated when it clearly is being generated.
My code has the following
import mod_curves
import mod_dates
import datetime
import modcalinputs
curvesclass = mod_curves.curvecalcs()
datesclass = mod_dates.datecalcs()
calcInputs = modcalinputs.calcinputs()
1. List item
class payment_calculations:
def __init__(self):
self.__accrualdays = 0
self.__accrualrate = 0
self.__coupondates = {}
....
When I run the PyCharm over any of the variables including the self in the init declaration I see the following reported by PyCharm (NameError)name ´self´ is not defined. This error appears on every variable in my code that starts with self.__
This is blocking any debugging I can do.
However, when I execute my code the app functions as expected.
How can I get rid of that error message_
Seems to work for me, using PyCharm 4.5.3. Apologies if this is too obvious of an answer:
When debugging, you can only view contents of variables if the debugger is currently in a place where those variables are in scope.
For example, here I have set a breakpoint in main, where self and self.__x are not in scope.
Here, I have set a breakpoint in the constructor, where self and self.__x ARE in scope.
An alternative, if you want to access it from main, is you just access with the instance instead of with self, like this:
I am not sure what exactly amongst the following worked - but something has
Did an auto code cleanup
Closed down PyCharm and reopened all the files
Methodically went through the entire project and made sure every line of code was very tight without any form of errors.
Now when I set a watch against self.__ and related variables I do not see the error. I think part of the problem was that I wrote the code originally in IDLE and then copied the code across to a PyCharm project. Thus I did not have the PyCharm error checking etc. That meant I had a lot of unnecessary errors etc.
The Python standard library and other libraries I use (e.g. PyQt) sometimes use exceptions for non-error conditions. Look at the following except of the function os.get_exec_path(). It uses multiple try statements to catch exceptions that are thrown while trying to find some environment data.
try:
path_list = env.get('PATH')
except TypeError:
path_list = None
if supports_bytes_environ:
try:
path_listb = env[b'PATH']
except (KeyError, TypeError):
pass
else:
if path_list is not None:
raise ValueError(
"env cannot contain 'PATH' and b'PATH' keys")
path_list = path_listb
if path_list is not None and isinstance(path_list, bytes):
path_list = fsdecode(path_list)
These exceptions do not signify an error and are thrown under normal conditions. When using exception breakpoints for one of these exceptions, the debugger will also break in these library functions.
Is there a way in PyCharm or in Python in general to have the debugger not break on exceptions that are thrown and caught inside a library without any involvement of my code?
in PyCharm, go to Run-->View Breakpoints, and check "On raise" and "Ignore library files".
The first option makes the debugger stop whenever an exception is raised, instead of just when the program terminates, and the second option gives PyCharm the policy to ignore library files, thus searching mainly in your code.
The solution was found thanks to CrazyCoder's link to the feature request, which has since been added.
For a while I had a complicated scheme which involved something like the following:
try( Closeable ignore = Debugger.newBreakSuppression() )
{
... library call which may throw ...
} <-- exception looks like it is thrown here
This allowed me to never be bothered by exceptions that were thrown and swallowed within library calls. If an exception was thrown by a library call and was not caught, then it would appear as if it occurred at the closing curly bracket.
The way it worked was as follows:
Closeable is an interface which extends AutoCloseable without declaring any checked exceptions.
ignore is just a name that tells IntelliJ IDEA to not complain about the unused variable, and it is necessary because silly java does not support try( Debugger.newBreakSuppression() ).
Debugger is my own class with debugging-related helper methods.
newBreakSuppression() was a method which would create a thread-local instance of some BreakSuppression class which would take note of the fact that we want break-on-exception to be temporarily suspended.
Then I had an exception breakpoint with a break condition that would invoke my Debugger class to ask whether it is okay to break, and the Debugger class would respond with a "no" if any BreakSuppression objects were instantiated.
That was extremely complicated, because the VM throws exceptions before my code has loaded, so the filter could not be evaluated during program startup, and the debugger would pop up a dialog complaining about that instead of ignoring it. (I am not complaining about that, I hate silent errors.) So, I had to have a terrible, horrible, do-not-try-this-at-home hack where the break condition would look like this: java.lang.System.err.equals( this ) Normally, this would never return
true, because System.err is not equal to a thrown exception, therefore the debugger would never break. However, when my Debugger class would get initialized, it would replace System.err with a class of its own,
which provided an implementation for equals(Object) and returned true if the debugger should break. So, essentially, I was using System.err as an eternal global variable.
Eventually I ditched this whole scheme because it is overly complicated and it performs very bad, because exceptions apparently get thrown very often in the java software ecosystem, so evaluating an expression every time an exception is thrown tremendously slows down everything.
This feature is not implemented yet, you can vote for it:
add ability to break (add breakpoint) on exceptions only for my files
There is another SO answer with a solution:
Debugging with pycharm, how to step into project, without entering django libraries
It is working for me, except I still go into the "_pydev_execfile.py" file, but I haven't stepped into other files after adding them to the exclusion in the linked answer.
I am new to python and just downloaded it today. I am using it to work on a web spider, so to test it out and make sure everything was working, I downloaded a sample code. Unfortunately, it does not work and gives me the error:
"AttributeError: 'MyShell' object has no attribute 'loaded' "
I am not sure if the code its self has an error or I failed to do something correctly when installing python. Is there anything you have to do when installing python like adding environmental variables, etc.? And what does that error generally mean?
Here is the sample code I used with imported spider class:
import chilkat
spider = chilkat.CkSpider()
spider.Initialize("www.chilkatsoft.com")
spider.AddUnspidered("http://www.chilkatsoft.com/")
for i in range(0,10):
success = spider.CrawlNext()
if (success == True):
print spider.lastUrl()
else:
if (spider.get_NumUnspidered() == 0):
print "No more URLs to spider"
else:
print spider.lastErrorText()
# Sleep 1 second before spidering the next URL.
spider.SleepMs(1000)
And what does that error generally
mean?
An Attribute in Python is a name belonging to an object - a method or a variable. An AttributeError means that the program tried to use an attribute of an object, but the object did not have the requested attribute.
For instance, string objects have the 'upper' attribute, which is a method that returns the uppercase version of the string. You could write a method that uses it like this:
def get_upper(my_string):
return my_string.upper()
However, note that there's nothing in that method to ensure that you have to give it a string. You could pass in a file object, or a number. Neither of those have the 'upper' attribute, and Python will raise an Attribute Error.
As for why you're seeing it in this instance, you haven't provided enough detail for us to work it out. Add the full error message to your question.
1) Put code in Try ... Except block. get exception details.
2) Could you tell StackTrace details means which line # and method thrown error
And also are you able to run other simple python scripts without any error. Means just try to run some sample script etc.
This problem is partly due to my lack of completely understanding scoping in python, so I'll need to review that. Either way, here is a seriously trivial piece of code that keeps crashing on my Django test app.
Here's a snippet:
#login_required
def someview(request):
try:
usergroup = request.user.groups.all()[0].name
except:
HttpResponseRedirect('/accounts/login')
if 'client' in usergroup:
stafflist = ProxyUserModel.objects.filter(groups__name='staff')
No brain surgery here, the problem is I get an error such as the following:
File "/usr/local/django/myapp/views.py", line 18, in someview
if 'client' in usergroup:
UnboundLocalError: local variable 'usergroup' referenced before assignment
My question here is, why is usergroup unbound? If it's unbound, that means the try statement had an exception thrown at which point an HttpResponseRedirect should happen, but it never does. Instead I get an HTTP 500 error back, which is slightly confusing.
Yes I can write smarter code and ensure that the user logging in definitely has a group associated with them. But this isn't a production app, I'm just trying to understand / learn Python/Django. Why exactly is the above happening when a user that's not associated with a group logs in instead of a redirect to a login page?
In this case I'm intentionally logging in as a user that isn't part of a group. That means that the above code should throw an IndexError exception like the following:
>>> somelist = []
>>> print somelist[0]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: list index out of range
HttpResponseRedirect('/accounts/login')
You're creating it but not returning it. Flow continues to the next line, which references usergroup despite it never having been assigned due to the exception.
The except is also troublesome. In general you should never catch ‘everything’ (except: or except Exception:) as there are lots of odd conditions in there you could be throwing away, making debugging very difficult. Either catch the specific exception subclass that you think is going to happen when the user isn't logged on, or, better, use an if test to see if they're logged on. (It's not really an exceptional condition.)
eg. in Django normally:
if not request.user.is_authenticated():
return HttpResponseRedirect('/accounts/login')
or if your concern is the user isn't in any groups (making the [0] fail):
groups= request.user.groups.all()
if len(groups)==0:
return HttpResponseRedirect('/accounts/login')
usergroup= groups[0].name
Try moving you if 'client' part inside you try block. Either that or define usergroup = None right above the try.
In cases where you have a try…except suite and you want code to run iff no exceptions have occurred, it's a good habit to write the code as follows:
try:
# code that could fail
except Exception1:
# handle exception1
except Exception2:
# handle exception2
else: # the code-that-could-fail didn't
# here runs the code that depends
# on the success of the try clause