Develop Reference

Python Flask CORS - API always allows any origin

I've looked through many SO answers, and can't seem to find this issue. I have a feeling that I'm just missing something obvious.
I have a basic Flask api, and I've implemented both the flask_cors extension and the custom Flask decorator [#crossdomain from Armin Ronacher].1 (http://flask.pocoo.org/snippets/56/) Both show the same issue.
This is my example app:
application = Flask(__name__,
static_url_path='',
static_folder='static')
CORS(application)
application.config['CORS_HEADERS'] = 'Content-Type'
#application.route('/api/v1.0/example')
#cross_origin(origins=['http://example.com'])
# #crossdomain(origin='http://example.com')
def api_example():
print(request.headers)
response = jsonify({'key': 'value'})
print(response.headers)
return response
(EDIT 3 inserted):
When I make a GET request to that endpoint from JS in a browser (from 127.0.0.1), it always returns 200, when I would expect to see:
Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://127.0.0.1:5000' is therefore not allowed access. The response had HTTP status code 403.
CURL:
ACCT:ENVIRON user$ curl -i http://127.0.0.1:5000/api/v1.0/example
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 20
Access-Control-Allow-Origin: http://example.com
Server: Werkzeug/0.11.4 Python/2.7.11
Date: [datetime]
{
"key": "value"
}
LOG:
Content-Length:
User-Agent: curl/7.54.0
Host: 127.0.0.1:5000
Accept: */*
Content-Type:
Content-Type: application/json
Content-Length: 20
127.0.0.1 - - [datetime] "GET /api/v1.0/example HTTP/1.1" 200 -
I'm not even seeing all of the proper headers in the response, and it doesn't seem to care what the origin is in the request.
Any ideas what I'm missing? Thanks!
EDIT:
As a side note, looking at the documentation example here (https://flask-cors.readthedocs.io/en/v1.7.4/#a-more-complicated-example), it shows:
#app.route("/")
def helloWorld():
'''
Since the path '/' does not match the regular expression r'/api/*',
this route does not have CORS headers set.
'''
return '''This view is not exposed over CORS.'''
...which is rather interesting since I already have the root path (and others) exposed without any CORS decoration, and they are working fine from any origin. So it seems that there is something fundamentally wrong with this setup.
Along those lines, the tutorial here (https://blog.miguelgrinberg.com/post/designing-a-restful-api-with-python-and-flask) seems to indicate that Flask apis should naturally be exposed without protection (I would assume that's just since the CORS extension hasn't been applied), but my application is basically just operating like the CORS extension doesn't even exist (other than a few notes in the log that you can see).
EDIT 2:
My comments were unclear, so I created three example endpoints on AWS API Gateway with different CORS settings. They are GET method endpoints that simply return "success":
1) CORS not enabled (default):
Endpoint: https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-default
Response:
XMLHttpRequest cannot load
https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-default.
Response to preflight request doesn't pass access control check: No
'Access-Control-Allow-Origin' header is present on the requested
resource. Origin 'http://127.0.0.1:5000' is therefore not allowed
access. The response had HTTP status code 403.
2) CORS enabled - Origin Restricted:
Access-Control-Allow-Headers: 'Content-Type'
Access-Control-Allow-Origin: 'http://example.com'
Endpoint: https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-enabled-example
Response:
XMLHttpRequest cannot load
https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-enabled-example.
Response to preflight request doesn't pass access control check: The
'Access-Control-Allow-Origin' header has a value 'http://example.com'
that is not equal to the supplied origin. Origin
'http://127.0.0.1:5000' is therefore not allowed access.
3) CORS enabled - Origin Wildcard:
Access-Control-Allow-Headers: 'Content-Type'
Access-Control-Allow-Origin: '*'
Endpoint: https://t9is0yupn4.execute-api.us-east-1.amazonaws.com/prod/cors-enabled-wildcard
Response:
"success"
I'm not that experienced with infrastructure, but my expectation was that enabling the Flask CORS extension would cause my api endpoints to mimic this behavior depending on what I set at the origins= setting. What am I missing in this Flask setup?
SOLUTION EDIT:
Alright, so given that something on my end was obviously not normal, I stripped down my app and re-implemented some very basic APIs for each variation of CORS origin restriction. I've been using AWS's elastic beanstalk to host the test environment, so I re-uploaded those examples and ran a JS ajax request to each. It's now working.
I'm getting the Access-Control-Allow-Origin error on naked endpoints. It appears that when I configured the app for deployment I was uncommenting CORS(application, resources=r'/api/*'), which was obviously allowing all origins for the naked endpoints!
I'm not sure why my route with a specific restriction (origins=[]) was also allowing everything, but that must have been some type of typo or something small, because it's working now.
A special thanks to sideshowbarker for all the help!

From your question as-is, it’s not completely clear what behavior you’re expecting. But as far as how the CORS protocol works, it seems like your server is already behaving as expected.
Specifically, the curl response cited in the question shows this response header:
Access-Control-Allow-Origin: http://example.com
That indicates a server already configured to tell browsers, Only allow cross-origin requests from frontend JavaScript code running in browsers if code’s running at the origin http://example.com.
If the behavior you’re expecting is that the server will now refuse requests from non-browser clients such as curl, then CORS configuration on its own isn’t going to cause a server to do that.
The only thing a server does differently when you configure it with CORS support is just to send the Access-Control-Allow-Origin response header and other CORS response headers. That’s it.
Actual enforcement of CORS restrictions is done only by browsers, not by servers.
So no matter what server-side CORS configuration you make, the server still goes on accepting requests from all clients and origins it would otherwise; in other words, all clients from all origins still keep on getting responses from the server just as they would otherwise.
But browsers will only expose responses from cross-origin requests to frontend JavsScript code running at a particular origin if the server the request was sent to opts-in to permitting the request by responding with an Access-Control-Allow-Origin header that allows that origin.
That’s the only thing you can do using CORS configuration. You can’t make a server only accept and respond to requests from particular origins just by doing any server-side CORS configuration. To do that, you need to use something other than just CORS configuration.

How to view Boto3 HTTPS request string

I have been able to view the attributes of the PreparedRequest that botocore sends, but I'm wondering how I can view the exact request string that is sent to AWS. I need the exact request string to be able to compare it to another application I'm testing AWS calls with.

You could also enable debug logging in boto3. That will log all requests and responses as well as lots of other things. Its a bit obscure to enable it:
import boto3
boto3.set_stream_logger(name='botocore')
The reason you have to specify botocore as the name to log is that all of the actual requests and responses happen at the botocore layer.

So what you probably want to do is to send your request through the proxy (mitmproxy, squid). Then check the proxy for what was sent.
Since HTTPS data is encrypted you must first decrypt it, then log the response, then encrypt it back and send to AWS. One of the options is to use mitmproxy. ( It's really easy to install )
Run mitmproxy
Open up another terminal and point proxy to mitmproxys port:
export http_proxy=127.0.0.1:8080
export https_proxy=$http_proxy
Then set verify=False when creating session/client
In [1]: import botocore.session
In [2]: client = botocore.session.Session().create_client('elasticache', verify=False)
Send request and look at the output of mitmproxy
In [3]: client.describe_cache_engine_versions()
The result should be similar to this:
Host: elasticache.us-east-1.amazonaws.com
Accept-Encoding: identity
Content-Length: 53
Content-Type: application/x-www-form-urlencoded
Authorization: AWS4-HMAC-SHA256 Credential=FOOOOOO/20150428/us-east-1/elasticache/aws4_request, SignedHeaders=host;user-agent;x-amz-date, Signature=BAAAAAAR
X-Amz-Date: 20150428T213004Z
User-Agent: Botocore/0.103.0 Python/2.7.6 Linux/3.13.0-49-generic
<?xml version='1.0' encoding='UTF-8'?>
<DescribeCacheEngineVersionsResponse
xmlns="http://elasticache.amazonaws.com/doc/2015-02-02/">
<DescribeCacheEngineVersionsResult>
<CacheEngineVersions>
<CacheEngineVersion>
<CacheParameterGroupFamily>memcached1.4</CacheParameterGroupFamily>
<Engine>memcached</Engine>
<CacheEngineVersionDescription>memcached version 1.4.14</CacheEngineVersionDescription>
<CacheEngineDescription>memcached</CacheEngineDescription>
<EngineVersion>1.4.14</EngineVersion>

Testing web-tornado using Firefox's HttpRequest addon

I am testing my web-tornado application using Firefox's HttpRequest add-on but after I log in and receive my secure cookie data, I am not able to re-use it to consume protected methods.
This is my response data:
POST http://mylocalurl:8888/user/login
Content-Type: application/x-www-form-urlencoded
Login=mylogin;Pass=123
-- response -- 200 OK Content-Length: 33
Content-Type: text/html; charset=UTF-8
Server: TornadoServer/2.2.1
Set-Cookie:
IdUser="Mjk=|1395170421|ffaf0d6fecf2f91c0dccca7cab03d799ef6637a0";
expires=Thu, 17 Apr 2014 19:20:21 GMT; Path=/
{
"Success": true }
-- end response --
Now why I am trying to do is to configure HttpRequester to use this cookie for my new requests. I tried to add it using the "Headers" tab but my server keeps sending me a 403, Forbidden.
Can anyone help me on this ? It could be with another tool (for linux) too.

I really like fiddler2 for these kind of things and there's an alpha build for mono that you may wish to try out: http://www.telerik.com/download/fiddler
If you don't mind paid software you can use Charles, for which there is a free trial.
And if you are testing and already using python, why not use a simple python script with requests and its Session object with cookie-persistence..

Django view sending empty reply with proper headers

I have Django project on Dreamhost server which has several views that returns Json response.Yesterday I have ported my Django project from local machine(localhost) to dreamhost server running apache.Now if I call my django view through jquery for
http://www.abc.com/projects/
It should return me all projects that i have in my mongodb database but instead of that it returns :
On Firefox - just headers with no response
Connection Keep-Alive
Content-Type application/json
Date Thu, 19 Jan 2012 09:03:34 GMT
Keep-Alive timeout=2, max=100
Server Apache
Status 200 OK
Transfer-Encoding chunked
On Chrome - No headers and response data.It throws an error:
XMLHttpRequest cannot load http://abc.com/Projects/. Origin null is not allowed by Access-Control-Allow-Origin.**
If I just access the http://www.abc.com/projects/ through my web-browser it returns me results in json format,but not in case if I use JavaScript/Jquery.
Earlier I was using this middleware to allow other domains to request and get response on my local-machine with django in-built server.But now when I am running on apache server It stops working so I removed It from Settings.py file.
I don't know why is this error coming .Please help
*EDIT*
As #burhan suggested I used jsonp on client side and now my server is returning json but browser is giving error before parsing it.Error is : unexpected token
JSON i am getting in reply is :
{"projects": [{"projectName": "carmella", "projectId": "4f13c7475fcff30710000000"}, {"projectName": "SeaMonkey", "projectId": "4f1677b75fcff37c03000001"}]}

You are running into the same origin policy sandbox. Since your server is www.abc.com and you are accessing abc.com - the origin is not the same, which is why the script is not executing.
You have a few options:
Make sure the URL matches exactly - to avoid the same origin policy sandbox.
Use jsonp in your javascript libary.

OAuth and the YouTube API

I am trying to use the YouTube services with OAuth. I have been able to obtain request tokens, authorize them and transform them into access tokens.
Now I am trying to use those tokens to actually do requests to the YouTube services. For instance I am trying to add a video to a playlist. Hence I am making a POST request to
https://gdata.youtube.com/feeds/api/playlists/XXXXXXXXXXXX
sending a body of
<?xml version="1.0" encoding="UTF-8"?>
<entry xmlns="http://www.w3.org/2005/Atom"
xmlns:yt="http://gdata.youtube.com/schemas/2007">
<id>XXXXXXXXX</id>
</entry>
and with the headers
Gdata-version: 2
Content-type: application/atom+xml
Authorization: OAuth oauth_consumer_key="www.xxxxx.xx",
oauth_nonce="xxxxxxxxxxxxxxxxxxxxxxxxx",
oauth_signature="XXXXXXXXXXXXXXXXXXX",
oauth_signature_method="HMAC-SHA1",
oauth_timestamp="1310985770",
oauth_token="1%2FXXXXXXXXXXXXXXXXXXXX",
oauth_version="1.0"
X-gdata-key: key="XXXXXXXXXXXXXXXXXXXXXXXXX"
plus some standard headers (Host and Content-Length) which are added by urllib2 (I am using Python) at the moment of the request.
Unfortunately, I get an Error 401: Unknown authorization header, and the headers of the response are
X-GData-User-Country: IT
WWW-Authenticate: GoogleLogin service="youtube",realm="https://www.google.com/youtube/accounts/ClientLogin"
Content-Type: text/html; charset=UTF-8
Content-Length: 179
Date: Mon, 18 Jul 2011 10:42:50 GMT
Expires: Mon, 18 Jul 2011 10:42:50 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Connection: close
In particular I do not know how to interpret the WWW-Authenticate header, whose realm hints to ClientLogin.
I have also tried to play with the OAuth Playground and the Authorization header sent by that site looks exactly like mine, except for the order of the fields. Still, on the plyaground everything works. Well, almost: I get an error telling that a Developer key is missing, but that is reasonable since there is no way to add one on the playground. Still, I go past the Error 401.
I have also tried to manually copy the Authorization header from there, and I got an Error 400: Bad request.
What am I doing wrong?

Turns out the problem was the newline before xmlns:yt. I was able to debug this using ncat, as suggeested here, and inspecting the full response.

i would suggest using the oauth python module, because it much more simple and takes care of the auth headers :) https://github.com/simplegeo/python-oauth2, as a solution i suggest you encode your parameters with 'utf-8' , i had a similar problem, and the solution was that google was expecting utf-8 encoded strings