serving blob giving 404 error - python

I'm trying to serve a blob from a blobKey that I've stored in a db.Model as a blobstore.BlobReferenceProperty() and I'm unsure of how to to this properly as my current method gives a 404. I am storing the blobKey value by taking the 'rewritten' request using the regular self.request.get('file_field')
the contents of the key look like this in the SDK console:
Content-Type: video/mp4
MIME-Version: 1.0
Content-Length: 475712
Content-MD5: OTY0MjY4OGI4NDgwYzVlZTI2MGJiNzg0YTA4OTIzNzY=
content-type: video/mp4
content-disposition: form-data; name="video_file"; filename="test_time.mp4"
X-AppEngine-Upload-Creation: 2012-11-10 21:41:12.973934
My video download handler is very standard and looks like this
class VideoServeHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, resource):
resource = str(urllib.unquote(resource))
blob_info = blobstore.BlobInfo.get(resource)
self.send_blob(blob_info)
Here's how i render the url:
'/videos/%s' % time_data_instance.video_key.key()
And the url mapping:
('/videos/([^/]+)?', VideoServeHandler)
And heres the 404 error i get from the console:
INFO 2012-11-10 21:54:11,371 dev_appserver.py:3092] "GET /videos/Content-Type:%20video/mp4MIME-Version:%201.0Content-Length:%20475712Content-MD5:%20OTY0MjY4OGI4NDgwYzVlZTI2MGJiNzg0YTA4OTIzNzY=content-type:%20video/mp4content-disposition:%20form-data;%20name= HTTP/1.1" 404 -
Does anybody know what the problem might be, looking at the url in the 404 its looks as if that is incorrect but i cant find any othehr way to generate it
Update:
Heres the entire code
http://www.pastebucket.com/5163

send_blob needs the BlobKey, not the BlobInfo.
Just pass resource to send_blob
class VideoServeHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, resource):
resource = str(urllib.unquote(resource))
self.send_blob(resource)

The problem is with your request string:
INFO 2012-11-10 21:54:11,371 dev_appserver.py:3092] "GET /videos/Content-Type:%20video/mp4MIME-Version:%201.0Content-Length:%20475712Content-MD5:%20OTY0MjY4OGI4NDgwYzVlZTI2MGJiNzg0YTA4OTIzNzY=content-type:%20video/mp4content-disposition:%20form-data;%20name= HTTP/1.1" 404 -
As far as I can see you request /videos/Content-Type:%20video/mp4[..some garbage...], no wonder you get a 404 back.
Check again how you make a request URL. Definitely it does not look like this:
'/videos/%s' % time_data_instance.video_key.key()

Related

Django return custom HTTP status text

I'm trying to be compatible with an app-side bug(). I want to return a response with custom HTTP message
For example, I get HTTP/1.1 429 TOO_MANY_REQUESTS now, I want get HTTP/1.1 429 CUSTOM_MESSAGE
Howerver, I can get only h2 200
Django has HttpResponse objects to build custom responses.
The following should do the trick:
from django.http import HttpResponse
response = HttpResponse("CUSTOM MESSAGE", status=429, headers={"CUSTOM_MESSAGE": "..."})
see: https://docs.djangoproject.com/en/4.1/ref/request-response/#id4
Updating HttpResponse.reason_phrase work for me, but HTTP/2.0 message will not show reason phrase, I could only find it in HTTP/1.1 message
see: https://docs.djangoproject.com/en/4.1/ref/request-response/#django.http.HttpResponse.reason_phrase

Django Status 200 but 'Failed to Load Response Data" On Certain Image Uploads

I'm making an ajax post request to the Python Django view below which always return a success json response.
# views.py
#csrf_exempt
def success_response(request):
"""
Returns a http response with a JSON success state
:param request:
:return:
"""
return JsonResponse({'success': True})
In each ajax post request, i'm attaching an image file in enctype=multipart/form-data format. As you can see, here is an example request payload:
------WebKitFormBoundarymYCuLcqA6kEkqMA7
Content-Disposition: form-data; name="file"; filename="willprobablycrash.png"
Content-Type: image/png
------WebKitFormBoundarymYCuLcqA6kEkqMA7--
How is it possible that images below a certain image dimension (below 960px x 1141px) return the intended response {"success": true} while images above the said dimension return Failed to Load Response Data?
In both success and error cases, the status_code in the response is 200.
Surprisingly, it is possible for http to return status_code 200 while failing to load the response data. Which resulted in the error i had above: Failed to Load Response Data. In this case, it was my server core code (Django) being able to send the response but there was no data included due to some issues with the resolvers within the framework. After i fixed those issues, i could receive 200 + the response data that i needed.
It because of Advertising blocker chrome extensions on your browser

How to intentionally cause a 400 Bad Request in Nginx?

I've split my previous question into two.
A consumer of my REST API says that on occasion I am returning a 400 Bad Request - The request sent by the client was syntactically incorrect. error.
My application (Python/Flask) logs don't seem to be capturing this, and neither do my webserver/Nginx logs.
I've changed the default 400 and 404 error message HTML's by adding the following line to the config (and adding the corresponding HTML pages in the proper directory):
error_page 404 /404.html;
location = /404.html {
root html;
}
error_page 400 /400.html;
location = /400.html {
root html;
}
This error message plainly states "Nginx" so that I know Nginx is giving the 400 and not my python/flask application (whose 400 has been changed to plainly state "Flask").
Is there a way that I can intentionally cause Nginx to return a 400 error, perhaps through python, CURL, or Postman? For example, I've also changed the 404 error page, and can intentionally get Nginx to return the corresponding error HTML by calling an invalid URL.
Create a location in NGINX's config to return 400 errors:
location /error {
return 400;
}
Presto.
Nginx will return status code 400 if it receives a header field larger than the configured large_client_header_buffers
A request header field cannot exceed the size of one buffer as well,
or the 400 (Bad Request) error is returned to the client. Buffers are
allocated only on demand. By default, the buffer size is equal to 8K
bytes. If after the end of request processing a connection is
transitioned into the keep-alive state, these buffers are released.
Source: http://nginx.org/en/docs/http/ngx_http_core_module.html#large_client_header_buffers
So you just need to create a curl request with a header larger than 8k. Here's an example using a bit of python to generate the header variable to pass into curl:
(nginx)macbook:nginx joeyoung$ myheader=$(python -c "print 'A'*9000")
(nginx)macbook:nginx joeyoung$ curl -vvv --header "X-MyHeader: $myheader" http://my.example.website.com
Results:
...
>
< HTTP/1.1 400 Bad Request
< Server: nginx/1.4.7
< Date: Wed, 02 Sep 2015 22:37:29 GMT
< Content-Type: text/html
< Content-Length: 248
< Connection: close

Flask Json Request 400 logging?

I wrote my own custom client which sends raw http requests via my wifi card to my flask webserver.
This is what a typical requests looks like:
Content-Length: 214
User-Agent: blah
Connection: close
Host: 1.2.3.4:5000
Content-Type: application/json
{"data":[{"scoutId":2,"message":"ph=5.65"},{"scoutId":4,"message":"ph=4.28"},{"scoutId":3,"message":"ph=4.28"},{"scoutId":2,"message":"ph=5.65"},{"scoutId":4,"message":"ph=4.28"},{"scoutId":3,"message":"ph=4.30"}]}
Sometimes, my clients screw up and send malformed JSON requests to my flask server. Typically, flask will just display:
1.2.3.5 - - [01/Sep/2014 22:13:03] "POST / HTTP/1.1" 400 -
and nothing informative about the request.
I would like to track every single request that resulted in 400 in my environment and analyze what is causing these errors.
Where can I place my custom error function in my flask server?
Try turning this on:
app.config['TRAP_BAD_REQUEST_ERRORS'] = True
This should make flask raise an exception instead of just logging the 400 (see documentation here).
If you need to do something more than that, make an event handler:
http://flask.pocoo.org/docs/0.10/patterns/errorpages/
#app.errorhandler(400)
def page_not_found(exc):
#do something with the exception object `exc` here
....
Or try wrapping the body of your view function in try/except.

Is my self.error() being overridden in Google App Engine?

In my Google App Engine application, I have a handler with a method for PUT requests:
def postMethod(self, arg):
response = do_backend_work(arg)
if response.field is None:
self.error(502)
self.response.out.write(json.dumps(
{'message': "you've been a bad boy!"}))
else:
<deal with well-formatted requests here>
.
.
.
However, when I do receive a request where response.field is None the request is returned as a 200. When I caught this error I inserted raise Exception(str(self.response)) just before the else block as a sanity check and found this in the logs:
raise Exception(str(self.response))
Exception: 502 Bad Gateway
Content-Type: text/html; charset=utf-8
Cache-Control: no-cache
Content-Length: 63
{"message": "you've been a bad boy!"}
INFO 2012-08-15 23:25:12,239 dev_appserver.py:2952]
"PUT /url/resource HTTP/1.1" 200 -
is there something I'm missing about how App Engine is processing the request?
After some rooting around, I figured out the problem. in my do_backend_work function, there was a rogue print statement which was hijacking the output stream! The moral of the story is: don't write to stdout on your Google App Engine back-end. I'm posting the question for the benefit of future generations.

Categories

Resources