Info: I want to upload files using Uppy in frontend and django-tus as backend for file processing. I am getting error TypeError: exceptions must derive from BaseException.
Traceback
Internal Server Error: /tus/upload/6393bfe5-277e-4c68-b9af-c0394be796b9
Traceback (most recent call last):
File "/django_tus/tusfile.py", line 75, in get_tusfile_or_404
raise TusResponse(status=404)
TypeError: exceptions must derive from BaseException
[06/Aug/2022 14:36:42] "HEAD /tus/upload/6393bfe5-277e-4c68-b9af-c0394be796b9 HTTP/1.1" 500 103054
[06/Aug/2022 14:36:42,624] - Broken pipe from ('127.0.0.1', 35814)
[06/Aug/2022 14:36:42] "POST /tus/upload/ HTTP/1.1" 201 0
[06/Aug/2022 14:36:42] "PATCH /tus/upload/8295bef4-c94a-4ab7-9c75-2635c74428d8 HTTP/1.1" 204 0
https://github.com/alican/django-tus/blob/master/django_tus/tusfile.py
class TusUpload(View):
def head(self, request, resource_id):
tus_file = TusFile.get_tusfile_or_404(str(resource_id))
return TusResponse(
status=200,
extra_headers={
'Upload-Offset': tus_file.offset,
'Upload-Length': tus_file.file_size,
})
def create_initial_file(metadata, file_size: int):
resource_id = str(uuid.uuid4())
cache.add("tus-uploads/{}/filename".format(resource_id), "{}".format(metadata.get("filename")), settings.TUS_TIMEOUT)
cache.add("tus-uploads/{}/file_size".format(resource_id), file_size, settings.TUS_TIMEOUT)
cache.add("tus-uploads/{}/offset".format(resource_id), 0, settings.TUS_TIMEOUT)
cache.add("tus-uploads/{}/metadata".format(resource_id), metadata, settings.TUS_TIMEOUT)
tus_file = TusFile(resource_id)
tus_file.write_init_file()
return tus_file
EDIT: Our pull request has now fixed this error.
The django-tus attempts in case of 404 error raise in line 60 TusResponse which does not however inherit from exception.
This is an error of the library. It has not been updated in 2 years and according to the #torek BaseException inheritance was not previously enforced. You could try to wrap it in try-except with TypeError but in my implementation this does not work.
You could attempt to add BaseException to inherited classes of TusResponse in your version of django-tus.
class TusResponse(HttpResponse, BaseException):
which should remove the error.
Related
according to werkzeug docs, InternalServerError has original_exception parameter. That looks cool and I'would like to use it in following way to handle exceptions:
#app.errorhandler(Exception)
def handle_errors(err):
return err if isinstance(err, HTTPException) else InternalServerError(original_exception=err)
and log it in #app.after_request method to pack all additional info from request and response, where standard request logging occurs. Unfortunatelly I'm not able to find out how to use stored original_exception. Where is it stored?
When I check sys.exc_info() in #app.after_request, it's also empty (or (None, None, None)).
One way is probably log exceptions directly in #app.errorhandler, request info should be accessible there and response is not ready (because of error) anyways, but what happens with original_exception after storing in InternalServerError???
Thanks for answer, Michal
I found myself here looking for the same thing - how to access the original_exception, but I eventually gave up and settled on traceback to give me the info I was seeking.
This is the key component, capturing the traceback stack as a list:
formatted_lines = traceback.format_exc().splitlines()
Here's my final code in which I extracted the line number of the error and the specific trigger, then passed them to the page:
from werkzeug.exceptions import InternalServerError
import traceback
#app.errorhandler(InternalServerError)
def handle_500(e):
templateData = {
'errorAt' : 'Internal Server Error',
'errorIs' : 'Check ~/www/gunicorn.error for details'
}
try:
formatted_lines = traceback.format_exc().splitlines()
for error_location in formatted_lines:
if 'intvlm8r.py' in error_location:
templateData['errorAt'] = error_location
templateData['errorIs'] = formatted_lines[-1]
except Exception as e:
app.logger.debug(f'handle_500: Unhandled exception: {e}')
return render_template('500_generic.html', **templateData)
Background - I am trying to create a Twitter bot which listens in real time for certain keywords coming only from certain accounts. When these conditions are met, I would then retweet it with my app, so I can follow with notifications from my main account.
Problem - Sometimes, but not always, I will encounter an error while the script is running which gives me the error listed in the subject of this post. I'm unsure what causes it, since it happens intermittently.
What I've Done - I've searched for the error; most of what I've found refers to "base 10", not "base 16". For the couple cases I've found where it references base 16, I don't understand the solution well enough to adapt it to my code (self teaching myself on this project).
Code
import tweepy
import json
import re
import logging
auth = tweepy.OAuthHandler("xxxx", "xxxx")
auth.set_access_token("yyyyy", "yyyy")
api = tweepy.API(auth, wait_on_rate_limit=True,
wait_on_rate_limit_notify=True)
class MyStreamListener(tweepy.StreamListener):
def __init__(self, api):
self.api = api
self.me = api.me()
def on_status(self, tweet):
keyword = ["Keyword1", "Keyword2"]
accounts = ['account1','account2']
patterns = [r'\b%s\b' % re.escape(s.strip()) for s in keyword]
patterns2 = [r'\b%s\b' % re.escape(s.strip()) for s in accounts]
there = re.compile('|'.join(patterns))
there2 = re.compile('|'.join(patterns2))
if there.search(tweet.text) and there2.search(str(tweet.user.id)):
print("New")
tweet.retweet()
def on_error(self, status):
print("Error detected")
tweets_listener = MyStreamListener(api)
stream = tweepy.Stream(api.auth, tweets_listener)
stream.filter(follow=['account1','account2'])
Error
ValueError Traceback (most recent call last)
~\anaconda3\lib\http\client.py in _get_chunk_left(self)
554 try:
555 chunk_left = self._read_next_chunk_size()
556 except ValueError:
~\anaconda3\lib\http\client.py in _read_next_chunk_size(self)
521 try:
522 return int(line, 16)
523 except ValueError:
ValueError: invalid literal for int() with base 16: b''
During handling of the above exception, another exception occurred:
IncompleteRead Traceback (most recent call last)
I then get a couple of incomplete reads, the last line of the error is:
ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))
This should now be handled in the development version of Tweepy on the master branch by automatically attempting to reconnect. The underlying connection error is likely due to falling too far behind in the stream when processing Tweets. For now, you can attempt to reduce the processing within on_status, handle the error by restarting the stream, wait for Tweepy v4.0, or use the development version of Tweepy.
For more context, see https://github.com/tweepy/tweepy/issues/237 and https://github.com/tweepy/tweepy/issues/448.
I'm using rest framework's Token authentication for my api. Recently I've found an issue that, If the Token provided in the request is invalid, instead of returning Invalid Token response, django throws TypeError at /api/users/: 'exceptions.KeyError' object is not callable.
I checked on the trace, and found this:
In django/db/models/query.py(Django version=1.7) file line no 357, inside raise DoesNotExists exception call, the actual exception TypeError raised by self.model._meta.object_name.
if num == 1:
return clone._result_cache[0]
if not num:
raise self.model.DoesNotExist(
"%s matching query does not exist." %
self.model._meta.object_name)
Does anybody knows why model's object_name became exception.KeyError rather than Token?
Bizarrely, I got this when I had this line in my code:
except KeyError, models.MyModel.DoesNotExist:
where the comma was interpreted as "as", redefining models.MyModel.DoesNotExist as a KeyError and causing "exception.KeyError is not callable" when creating one.
What I meant to do was:
except (KeyError, models.MyModel.DoesNotExist):
This was my code in django 1.6 ( working good)
I upgraded to django 1.7.
First simplejson is depracated : i changed simplejson to json,But i receive the same error always
is not json serializable.
Any ideas ?
Views:
def request_to_json(request):
post_data = request.body
json_data = simplejson.loads(post_data)
return json_data
def receiver(request):
try:
json_data = request_to_json(request)
user_id=json_data['user_id'];
site_id=json_data['site_id'];
# A variable to return to the app
response = 'Ok'
except:
response = sys.exc_info()[0]
return HttpResponse(simplejson.dumps(response))
Error
TypeError at /views/receiver/
<class 'TypeError'> is not JSON serializable
Request Method: POST
Request URL: http://localhost:8000/views/receiver/
Django Version: 1.7
Exception Type: TypeError
Exception Value:
<class 'TypeError'> is not JSON serializable
Exception Location: C:\Python34\lib\json\encoder.py in default, line 173
Python Executable: C:\Python34\python.EXE
Python Version: 3.4.1
Python Path:
['C:\\workspace-eclipse\\IndoorPositioning',
'C:\\Python34\\lib\\site-packages\\setuptools-5.4.2-py3.4.egg',
'C:\\Windows\\system32\\python34.zip',
'C:\\Python34\\DLLs',
'C:\\Python34\\lib',
'C:\\Python34',
'C:\\Python34\\lib\\site-packages']
Server time: Sat, 13 Sep 2014 12:18:36 +0200
Your exception handler is raising an exception. You are trying to serialise the entire exception object to json to return to the user. Instead get a string representation of the exception and return that:
def receiver(request):
try:
...
except:
response = "%s" % sys.exc_info()[0]
return HttpResponse(simplejson.dumps(response))
(It's not a great idea to be directly returning internal exception messages, you should probably try and specifically catch the exception and return a user-friendly message instead of the actual exception)
If you want to see why you are getting an exception in your code, you need to allow the exception to be raised. Get rid of the try..catch block and you will see the actual exception that is being raised with regards the request_to_json function
By the looks of it, the problem is that you are trying to serialize the entire body of the request:
post_data = request.body
which simplejson doesn't like
There are three situations when I need to handle exceptions.
When data validation raised exception
When library/module functions raised exceptions (e.g. database connection abort)
When business logic raises exception such as 500, 503, 401, 403 and 404
def library_func():
try:
...
except HTTPException:
raise TwitterServiceException("Twitter is down!")
#view_config(route_name="home", renderer="json")
#validator
#authorization
def home_view(request):
try:
tweets = library_func()
return {"tweets": tweets}
except TwitterServiceException as e:
LOG.critical(e.msg)
raise ParnterServcieError(e.msg) # this is probably a 503 error
def validator(args):
# I will show the high level of this decorator
try:
decode input as JSON
verify data format
except ValueError as err:
error = {'error': "Missing required parameters."}
except json.JSONDecodeError as err:
error = {'error': "Failed to decode the incoming JSON payload."}
if error is not None:
return HTTPBadRequest(body=json.dumps(error),
content_type='application/json')
def authorization(args):
# very similar to validator except it performs authorization and if failed
# 401 is raised with some helpful message.
The doc suggests Custom Exception Views. In my PoC above, I will tie ParnterServcieError as one. I can even generalize HTTPBadRequest and all praymid.httpexceptions using custom exception so that I no longer need to repeat json.dumps and content_type. I can set a boilerplate error body before I return request.response object.
Idea:
#view_config(context=ParnterServcieError)
def 503_service_error_view(e, request):
request.response.status = 503
request.response.json_body = {"error": e.msg}
return request.response
I can generalize one for all uncaught, unspecified exceptions (which results in 500 Internal Server Error) called 500_internal_server_error_view.
Does this seem sane and clean to people? Is my way of handling high and low level of exceptions proper and Pythonic?
I applied this strategy to ToDoPyramid and could encapsulate error handling in a single custom exception view that was repeated multiple times in the application before. Until you could even improve it, you got a great idea. Pyramid rocks.
References
Catching database connection error in ToDoPyramid