Develop Reference

Django failing to load 400.html

I've defined custom templates for errors 400, and 404 for my Django project. When I try to access the production version of my site, the error 404 template is correctly loaded for missing pages. However, if I send a bad request to my Apache/Django server (e.g. http://mysite.example.com/%), the template for the error 400 is not loaded, instead, the regular Apache error page is rendered:
Bad Request
Your browser sent a request that this server could not understand.
Apache/2.4.18 (Ubuntu) Server at mysite.example.com Port 80
Is apache relaying this request to Django at all, or do I need to define handler400 in my Django project in order for this to work (though I didn't have to do that for the 404.html)?

The crucial point here is that your apache is acting as a proxy for your usgi server. It's forwarding all valid requests to usgi, a request for a non existent request is a valid request as far as apache is concerned and needs the forwarded to the django router - which will find that the url mapping does not exist and raise a 404 error. This error is done internally by django and results in the django 404 page being shown.
Some requests, most notably the django rest framework produce 400 responses internally when the serializers fail to validate the incoming json request. Those will also result in the django 400 page being shown.
However if the request itself is malformed, it will never be forwarded to the usgi server and django will never see it. it will be handled internally by apache hence the reason that the apache 400 html is shown.
The simplest solution would be to replace all the apache error pages with the corresponding django one (if these are templates, render them and save the html)

Python Flask CORS - API always allows any origin

I've looked through many SO answers, and can't seem to find this issue. I have a feeling that I'm just missing something obvious.
I have a basic Flask api, and I've implemented both the flask_cors extension and the custom Flask decorator [#crossdomain from Armin Ronacher].1 (http://flask.pocoo.org/snippets/56/) Both show the same issue.
This is my example app:
application = Flask(__name__,
static_url_path='',
static_folder='static')
CORS(application)
application.config['CORS_HEADERS'] = 'Content-Type'
#application.route('/api/v1.0/example')
#cross_origin(origins=['http://example.com'])
# #crossdomain(origin='http://example.com')
def api_example():
print(request.headers)
response = jsonify({'key': 'value'})
print(response.headers)
return response
(EDIT 3 inserted):
When I make a GET request to that endpoint from JS in a browser (from 127.0.0.1), it always returns 200, when I would expect to see:
Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://127.0.0.1:5000' is therefore not allowed access. The response had HTTP status code 403.
CURL:
ACCT:ENVIRON user$ curl -i http://127.0.0.1:5000/api/v1.0/example
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 20
Access-Control-Allow-Origin: http://example.com
Server: Werkzeug/0.11.4 Python/2.7.11
Date: [datetime]
{
"key": "value"
}
LOG:
Content-Length:
User-Agent: curl/7.54.0
Host: 127.0.0.1:5000
Accept: */*
Content-Type:
Content-Type: application/json
Content-Length: 20
127.0.0.1 - - [datetime] "GET /api/v1.0/example HTTP/1.1" 200 -
I'm not even seeing all of the proper headers in the response, and it doesn't seem to care what the origin is in the request.
Any ideas what I'm missing? Thanks!
EDIT:
As a side note, looking at the documentation example here (https://flask-cors.readthedocs.io/en/v1.7.4/#a-more-complicated-example), it shows:
#app.route("/")
def helloWorld():
'''
Since the path '/' does not match the regular expression r'/api/*',
this route does not have CORS headers set.
'''
return '''This view is not exposed over CORS.'''
...which is rather interesting since I already have the root path (and others) exposed without any CORS decoration, and they are working fine from any origin. So it seems that there is something fundamentally wrong with this setup.
Along those lines, the tutorial here (https://blog.miguelgrinberg.com/post/designing-a-restful-api-with-python-and-flask) seems to indicate that Flask apis should naturally be exposed without protection (I would assume that's just since the CORS extension hasn't been applied), but my application is basically just operating like the CORS extension doesn't even exist (other than a few notes in the log that you can see).
EDIT 2:
My comments were unclear, so I created three example endpoints on AWS API Gateway with different CORS settings. They are GET method endpoints that simply return "success":
1) CORS not enabled (default):
Endpoint: https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-default
Response:
XMLHttpRequest cannot load
https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-default.
Response to preflight request doesn't pass access control check: No
'Access-Control-Allow-Origin' header is present on the requested
resource. Origin 'http://127.0.0.1:5000' is therefore not allowed
access. The response had HTTP status code 403.
2) CORS enabled - Origin Restricted:
Access-Control-Allow-Headers: 'Content-Type'
Access-Control-Allow-Origin: 'http://example.com'
Endpoint: https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-enabled-example
Response:
XMLHttpRequest cannot load
https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-enabled-example.
Response to preflight request doesn't pass access control check: The
'Access-Control-Allow-Origin' header has a value 'http://example.com'
that is not equal to the supplied origin. Origin
'http://127.0.0.1:5000' is therefore not allowed access.
3) CORS enabled - Origin Wildcard:
Access-Control-Allow-Headers: 'Content-Type'
Access-Control-Allow-Origin: '*'
Endpoint: https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-enabled-wildcard
Response:
"success"
I'm not that experienced with infrastructure, but my expectation was that enabling the Flask CORS extension would cause my api endpoints to mimic this behavior depending on what I set at the origins= setting. What am I missing in this Flask setup?
SOLUTION EDIT:
Alright, so given that something on my end was obviously not normal, I stripped down my app and re-implemented some very basic APIs for each variation of CORS origin restriction. I've been using AWS's elastic beanstalk to host the test environment, so I re-uploaded those examples and ran a JS ajax request to each. It's now working.
I'm getting the Access-Control-Allow-Origin error on naked endpoints. It appears that when I configured the app for deployment I was uncommenting CORS(application, resources=r'/api/*'), which was obviously allowing all origins for the naked endpoints!
I'm not sure why my route with a specific restriction (origins=[]) was also allowing everything, but that must have been some type of typo or something small, because it's working now.
A special thanks to sideshowbarker for all the help!

From your question as-is, it’s not completely clear what behavior you’re expecting. But as far as how the CORS protocol works, it seems like your server is already behaving as expected.
Specifically, the curl response cited in the question shows this response header:
Access-Control-Allow-Origin: http://example.com
That indicates a server already configured to tell browsers, Only allow cross-origin requests from frontend JavaScript code running in browsers if code’s running at the origin http://example.com.
If the behavior you’re expecting is that the server will now refuse requests from non-browser clients such as curl, then CORS configuration on its own isn’t going to cause a server to do that.
The only thing a server does differently when you configure it with CORS support is just to send the Access-Control-Allow-Origin response header and other CORS response headers. That’s it.
Actual enforcement of CORS restrictions is done only by browsers, not by servers.
So no matter what server-side CORS configuration you make, the server still goes on accepting requests from all clients and origins it would otherwise; in other words, all clients from all origins still keep on getting responses from the server just as they would otherwise.
But browsers will only expose responses from cross-origin requests to frontend JavsScript code running at a particular origin if the server the request was sent to opts-in to permitting the request by responding with an Access-Control-Allow-Origin header that allows that origin.
That’s the only thing you can do using CORS configuration. You can’t make a server only accept and respond to requests from particular origins just by doing any server-side CORS configuration. To do that, you need to use something other than just CORS configuration.

https site with Django in text browser throws CSRF verification failed

I have a Django site that works well on a server using HTTPS protocol, I can use it with no problem with all kind of browsers.
The thing is that every time I try to use a text browser, I get a
Forbidden (403)
CSRF verification failed. Request aborted.
You are seeing this message because this HTTPS site requires a 'Referer header' to be sent by your Web browser, but none was sent.
This header is required for security reasons, to ensure that your browser is not being hijacked by third parties.
If you have configured your browser to disable 'Referer' headers, please re-enable them, at least for this site, or for HTTPS
connections, or for 'same-origin' requests.
Help
Reason given for failure:
Referer checking failed - no Referer.
I have tried links, lynx, even w3m and eww on emacs, to no avail.
When I use a HTTP site (like when I'm using the manage.py runserver) I can use the site on text browsers with no problem, but my production server needs a HTTPS protocol and that's when I get this error.
[ EDIT: just for testing purposes, I deployed an HTTP server for my django site on the production server. It works well on text browsers... ]
[ EDIT: given the message the server throws, why are Referer headers not been given? ]

Lynx is likely configured to not send the Referer header. Check /etc/lynx.cfg for "REFERER".
There are entries like NO_REFERER_HEADER. Make sure that's set to false. If that's not it, check around in that config for any other disabled referer headers.
Also related, the CSRF and Referer header debate: https://code.djangoproject.com/ticket/16870

Are you setting SECURE_PROXY_SSL_HEADER, SESSION_COOKIE_SECURE and CSRF_COOKIE_SECURE in your settings?
https://docs.djangoproject.com/en/1.7/topics/security/#ssl-https

Not able to make post request to native domain from chrome extension

Description - I have a website www.robustest.com which bind with http://robustestom.appspot.com.
When I am tying to make post request to /user/signup (robustest) from a chrome extension Postman I am getting following error
Request URL:http://robustest.com/user/signup
Request Method:POST
Status Code:301 Moved Permanently
**Response Header -**
Alternate-Protocol:80:quic
Content-Length:233
Content-Type:text/html; charset=UTF-8
Date:Mon, 30 Dec 2013 03:43:43 GMT
Location:http://www.robustest.com/user/signup
Server:ghs
X-Frame-Options:SAMEORIGIN
X-XSS-Protection:1; mode=block
But its working as expected when I am firing against http://robustestom.appspot.com/user/signup.
Why We need - We are making an extension and there we need post request against our doamin.
Debugging - I might be wrong but it seems , all post request are redirecting to their counter part 'GET' because of origin is not 'robustest.com' but its a 'chrome extension'

The 301 is redirecting from robustest.com to www.robustest.com. Add the www to the domain the extension is making requests to and the 301 error should go away.

Google App Engine OAuth endpoints throwing 400 in production

I implemented the experimental OAuth support for Google App Engine using Python, and have it working locally, but the endpoints are throwing a 400 when I deploy to appspot.
For example, the url http(s)://my-app.appspot.com/_ah/OAuthGetRequestToken returns a 400, but locally that url pattern behaves as expected.
I have tried both http and https, and assumed that appspot handles the ssl cert.
UPDATE
I've been using the OAuth Playground to test my code. Despite documentation, it seems Registering your app is required. Go here for instructions on how to register. According to documentation during the registration process, certificate is not required when running on App Engine. Playground is showing more detail on the error - "signature invalid". If I understand correctly, the signature is produced from a signature base string. In this case I am using the base string 7DYB6MJ2s-IQcd7VJYJUmcct .
GET /accounts/OAuthGetRequestToken?scope=https%3A%2F%2Fmail.google.com%2Fmail%2Ffeed%2Fatom HTTP/1.1
Host: www.google.com
Accept: */*
Authorization: OAuth oauth_version="1.0", oauth_nonce="168cfd60a93a46caa38dddfdcedd9de9", oauth_timestamp="1305315895", oauth_consumer_key="xxxxxxx.appspot.com", oauth_callback="http%3A%2F%2Fgooglecodesamples.com%2Foauth_playground%2Findex.php", oauth_signature_method="HMAC-SHA1", oauth_signature="4J5faUujE0VNaybyvFCiEPY7DQ8%3D"
HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=UTF-8
Date: Fri, 13 May 2011 19:44:55 GMT
Expires: Fri, 13 May 2011 19:44:55 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Length: 451
Server: GSE
**signature_invalid**
base_string:GET&https%3A%2F%2Fwww.google.com%2Faccounts%2FOAuthGetRequestToken&oauth_callback%3Dhttp%253A%252F%252Fgooglecodesamples.com%252Foauth_playground%252Findex.php%26oauth_consumer_key%3Dxxxxxx.appspot.com%26oauth_nonce%3D168cf60a94caa38e2defdcedd9de9%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1305315895%26oauth_version%3D1.0%26scope%3Dhttps%253A%252F%252Fmail.google.com%252Fmail%252Ffeed%252Fatom
FINAL UPDATE
There were two things causing the 400. First, the app was not registered. Google's documentation says it's optional, but that is not the case apparently. Secondly, the Request was not properly signed. Here is an excellent debugging tool to test your OAuth Requests: Oauth Playground

You must register your domain in order to have OAuth working on production.
Although the following docs state that Registering is Optional:
http://code.google.com/apis/accounts/docs/RegistrationForWebAppsAuto.html
It doesn't work without the Registration since January.
Look at the link above and Register your domain/application. You don't need to submit a certificate, this is still optional.

Which request method it is? In dev_appserver_oauth.py I see:
if method != 'GET' and method != 'POST':
outfile.write('Status: 400\r\n')
return
So it'll only work for GET or POST requests.

First ensure you have enabled Federated Login in your Application Settings.
From your description it sounds like you might just be performing a direct GET request to /_ah/OAuthGetRequestToken without any of the other required parameters of oAuth. This will work on the dev_appserver as it is simply a mockup of oAuth to let you flesh out your code.
See the parameters listed on the OAuthGetRequestToken description page for what is required and how to deal with signing. I believe you can ignore scope for GAE though