I wrote my own custom client which sends raw http requests via my wifi card to my flask webserver.
This is what a typical requests looks like:
Content-Length: 214
User-Agent: blah
Connection: close
Host: 1.2.3.4:5000
Content-Type: application/json
{"data":[{"scoutId":2,"message":"ph=5.65"},{"scoutId":4,"message":"ph=4.28"},{"scoutId":3,"message":"ph=4.28"},{"scoutId":2,"message":"ph=5.65"},{"scoutId":4,"message":"ph=4.28"},{"scoutId":3,"message":"ph=4.30"}]}
Sometimes, my clients screw up and send malformed JSON requests to my flask server. Typically, flask will just display:
1.2.3.5 - - [01/Sep/2014 22:13:03] "POST / HTTP/1.1" 400 -
and nothing informative about the request.
I would like to track every single request that resulted in 400 in my environment and analyze what is causing these errors.
Where can I place my custom error function in my flask server?
Try turning this on:
app.config['TRAP_BAD_REQUEST_ERRORS'] = True
This should make flask raise an exception instead of just logging the 400 (see documentation here).
If you need to do something more than that, make an event handler:
http://flask.pocoo.org/docs/0.10/patterns/errorpages/
#app.errorhandler(400)
def page_not_found(exc):
#do something with the exception object `exc` here
....
Or try wrapping the body of your view function in try/except.
Related
I've looked through many SO answers, and can't seem to find this issue. I have a feeling that I'm just missing something obvious.
I have a basic Flask api, and I've implemented both the flask_cors extension and the custom Flask decorator [#crossdomain from Armin Ronacher].1 (http://flask.pocoo.org/snippets/56/) Both show the same issue.
This is my example app:
application = Flask(__name__,
static_url_path='',
static_folder='static')
CORS(application)
application.config['CORS_HEADERS'] = 'Content-Type'
#application.route('/api/v1.0/example')
#cross_origin(origins=['http://example.com'])
# #crossdomain(origin='http://example.com')
def api_example():
print(request.headers)
response = jsonify({'key': 'value'})
print(response.headers)
return response
(EDIT 3 inserted):
When I make a GET request to that endpoint from JS in a browser (from 127.0.0.1), it always returns 200, when I would expect to see:
Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://127.0.0.1:5000' is therefore not allowed access. The response had HTTP status code 403.
CURL:
ACCT:ENVIRON user$ curl -i http://127.0.0.1:5000/api/v1.0/example
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 20
Access-Control-Allow-Origin: http://example.com
Server: Werkzeug/0.11.4 Python/2.7.11
Date: [datetime]
{
"key": "value"
}
LOG:
Content-Length:
User-Agent: curl/7.54.0
Host: 127.0.0.1:5000
Accept: */*
Content-Type:
Content-Type: application/json
Content-Length: 20
127.0.0.1 - - [datetime] "GET /api/v1.0/example HTTP/1.1" 200 -
I'm not even seeing all of the proper headers in the response, and it doesn't seem to care what the origin is in the request.
Any ideas what I'm missing? Thanks!
EDIT:
As a side note, looking at the documentation example here (https://flask-cors.readthedocs.io/en/v1.7.4/#a-more-complicated-example), it shows:
#app.route("/")
def helloWorld():
'''
Since the path '/' does not match the regular expression r'/api/*',
this route does not have CORS headers set.
'''
return '''This view is not exposed over CORS.'''
...which is rather interesting since I already have the root path (and others) exposed without any CORS decoration, and they are working fine from any origin. So it seems that there is something fundamentally wrong with this setup.
Along those lines, the tutorial here (https://blog.miguelgrinberg.com/post/designing-a-restful-api-with-python-and-flask) seems to indicate that Flask apis should naturally be exposed without protection (I would assume that's just since the CORS extension hasn't been applied), but my application is basically just operating like the CORS extension doesn't even exist (other than a few notes in the log that you can see).
EDIT 2:
My comments were unclear, so I created three example endpoints on AWS API Gateway with different CORS settings. They are GET method endpoints that simply return "success":
1) CORS not enabled (default):
Endpoint: https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-default
Response:
XMLHttpRequest cannot load
https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-default.
Response to preflight request doesn't pass access control check: No
'Access-Control-Allow-Origin' header is present on the requested
resource. Origin 'http://127.0.0.1:5000' is therefore not allowed
access. The response had HTTP status code 403.
2) CORS enabled - Origin Restricted:
Access-Control-Allow-Headers: 'Content-Type'
Access-Control-Allow-Origin: 'http://example.com'
Endpoint: https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-enabled-example
Response:
XMLHttpRequest cannot load
https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-enabled-example.
Response to preflight request doesn't pass access control check: The
'Access-Control-Allow-Origin' header has a value 'http://example.com'
that is not equal to the supplied origin. Origin
'http://127.0.0.1:5000' is therefore not allowed access.
3) CORS enabled - Origin Wildcard:
Access-Control-Allow-Headers: 'Content-Type'
Access-Control-Allow-Origin: '*'
Endpoint: https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-enabled-wildcard
Response:
"success"
I'm not that experienced with infrastructure, but my expectation was that enabling the Flask CORS extension would cause my api endpoints to mimic this behavior depending on what I set at the origins= setting. What am I missing in this Flask setup?
SOLUTION EDIT:
Alright, so given that something on my end was obviously not normal, I stripped down my app and re-implemented some very basic APIs for each variation of CORS origin restriction. I've been using AWS's elastic beanstalk to host the test environment, so I re-uploaded those examples and ran a JS ajax request to each. It's now working.
I'm getting the Access-Control-Allow-Origin error on naked endpoints. It appears that when I configured the app for deployment I was uncommenting CORS(application, resources=r'/api/*'), which was obviously allowing all origins for the naked endpoints!
I'm not sure why my route with a specific restriction (origins=[]) was also allowing everything, but that must have been some type of typo or something small, because it's working now.
A special thanks to sideshowbarker for all the help!
From your question as-is, it’s not completely clear what behavior you’re expecting. But as far as how the CORS protocol works, it seems like your server is already behaving as expected.
Specifically, the curl response cited in the question shows this response header:
Access-Control-Allow-Origin: http://example.com
That indicates a server already configured to tell browsers, Only allow cross-origin requests from frontend JavaScript code running in browsers if code’s running at the origin http://example.com.
If the behavior you’re expecting is that the server will now refuse requests from non-browser clients such as curl, then CORS configuration on its own isn’t going to cause a server to do that.
The only thing a server does differently when you configure it with CORS support is just to send the Access-Control-Allow-Origin response header and other CORS response headers. That’s it.
Actual enforcement of CORS restrictions is done only by browsers, not by servers.
So no matter what server-side CORS configuration you make, the server still goes on accepting requests from all clients and origins it would otherwise; in other words, all clients from all origins still keep on getting responses from the server just as they would otherwise.
But browsers will only expose responses from cross-origin requests to frontend JavsScript code running at a particular origin if the server the request was sent to opts-in to permitting the request by responding with an Access-Control-Allow-Origin header that allows that origin.
That’s the only thing you can do using CORS configuration. You can’t make a server only accept and respond to requests from particular origins just by doing any server-side CORS configuration. To do that, you need to use something other than just CORS configuration.
I have a daily cron which handles some of the recurring events at my app, and from time to time I notice one weird error that pops up in logs. The cron, among other things, does a validation of some codes, and it uses the webapp running on the same server, so the validation request is made via POST request with some data.
url = 'https://example.com/validate/'
payload = {'pin': pin, 'sku': sku, 'phone': phone, 'AR': True}
validation_post = requests.post(url, data=payload)
So, this makes the actual request and I log the response. From time to time, and recently up to 50% of the request, the response contains the following message from nginx:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>405 Method Not Allowed</title>
<h1>Method Not Allowed</h1>
<p>The method GET is not allowed for the requested URL.</p>
So, the actual request was made using the GET method, not the POST as it was instructed in the code. In the nginx access.log I can see that entry:
123.123.123.123 - - [18/Feb/2015:12:26:50 -0500] "GET /validate/ HTTP/1.1" 405 182 "-" "python-requests/2.2.1 CPython/2.7.6 Linux/3.13.0-37-generic"
And the uwsgi log for the app shows the similar thing:
[pid: 6888|app: 0|req: 1589/58763] 123.123.123.123 () {40 vars in 613 bytes} [Mon Apr 6 11:42:41 2015] GET /validate/ => generated 182 bytes in 1 msecs (HTTP/1.1 405) 4 headers in 234 bytes (1 switches on core 0)
So, everything points out that the actual request was not made using the POST. The app route that handles this code is simple, and this is an excerpt:
#app.route('/validate/', methods=['POST'])
#login_required
def validate():
if isinstance(current_user.user, Sales):
try:
#do the stuff here
except Exception, e:
app.logger.exception(str(e))
return 0
abort(403)
The app route can fail, and there are some returns inside the try block, but even if those fails or there is an expcetion, there is nothing that could raise the 405 error code in this block, only 403 which rarely happens since I construct and login the user manually from the cron.
I have found similar thing here but the soultion there was that there was a redirect from HTTP to HTTPS version, and I also have that redirect present in the server, but the URL the request is being made has the HTTPS in it, so I doubt this is the cause.
The stack I am running this on is uwsgi+nginx+flask. Can anyone see what might be causing this? To repeat, its not happening always, so sometimes its working as expected, sometimes not. I recently migrated from apache and mod_wsgi to this new stack and from that point I have started encontering this error; can't recally ever seeing it on apache environment.
Thanks!
The only time we ever change a POST request to a GET is when we're handling a redirect. Depending on the redirect code, we will change the request method. If you want to be sure that we don't follow redirects, you need to pass allow_redirects=False. That said, you need to figure out why your application is generating redirects (including if it's redirecting to HTTP or to a different domain, or using a specific status code).
Not sure if it's by design, but removing the forward slash at the end of the URL fixed it for me:
url = 'https://example.com/validate/' # remove the slash
payload = {'pin': pin, 'sku': sku, 'phone': phone, 'AR': True}
validation_post = requests.post(url, data=payload)
I've got my handler setup but after the client requested something via the webbrowser I want the client to be redirected to the default webpage ("/").
I wrote my own http server handler.
How can I rederect a client using python 3 to another address?
Return a 302 redirect to the user's browser.
An example redirect header looks like this:
HTTP/1.1 302 Found
Location: /
Refer to RFC2616, section 10.3 here for all redirect status codes.
In my Google App Engine application, I have a handler with a method for PUT requests:
def postMethod(self, arg):
response = do_backend_work(arg)
if response.field is None:
self.error(502)
self.response.out.write(json.dumps(
{'message': "you've been a bad boy!"}))
else:
<deal with well-formatted requests here>
.
.
.
However, when I do receive a request where response.field is None the request is returned as a 200. When I caught this error I inserted raise Exception(str(self.response)) just before the else block as a sanity check and found this in the logs:
raise Exception(str(self.response))
Exception: 502 Bad Gateway
Content-Type: text/html; charset=utf-8
Cache-Control: no-cache
Content-Length: 63
{"message": "you've been a bad boy!"}
INFO 2012-08-15 23:25:12,239 dev_appserver.py:2952]
"PUT /url/resource HTTP/1.1" 200 -
is there something I'm missing about how App Engine is processing the request?
After some rooting around, I figured out the problem. in my do_backend_work function, there was a rogue print statement which was hijacking the output stream! The moral of the story is: don't write to stdout on your Google App Engine back-end. I'm posting the question for the benefit of future generations.
I have Django project on Dreamhost server which has several views that returns Json response.Yesterday I have ported my Django project from local machine(localhost) to dreamhost server running apache.Now if I call my django view through jquery for
http://www.abc.com/projects/
It should return me all projects that i have in my mongodb database but instead of that it returns :
On Firefox - just headers with no response
Connection Keep-Alive
Content-Type application/json
Date Thu, 19 Jan 2012 09:03:34 GMT
Keep-Alive timeout=2, max=100
Server Apache
Status 200 OK
Transfer-Encoding chunked
On Chrome - No headers and response data.It throws an error:
XMLHttpRequest cannot load http://abc.com/Projects/. Origin null is not allowed by Access-Control-Allow-Origin.**
If I just access the http://www.abc.com/projects/ through my web-browser it returns me results in json format,but not in case if I use JavaScript/Jquery.
Earlier I was using this middleware to allow other domains to request and get response on my local-machine with django in-built server.But now when I am running on apache server It stops working so I removed It from Settings.py file.
I don't know why is this error coming .Please help
*EDIT*
As #burhan suggested I used jsonp on client side and now my server is returning json but browser is giving error before parsing it.Error is : unexpected token
JSON i am getting in reply is :
{"projects": [{"projectName": "carmella", "projectId": "4f13c7475fcff30710000000"}, {"projectName": "SeaMonkey", "projectId": "4f1677b75fcff37c03000001"}]}
You are running into the same origin policy sandbox. Since your server is www.abc.com and you are accessing abc.com - the origin is not the same, which is why the script is not executing.
You have a few options:
Make sure the URL matches exactly - to avoid the same origin policy sandbox.
Use jsonp in your javascript libary.