Overview
When creating a post request from my website to my Python server running CherryPy, I receive the error Access to XMLHttpRequest has been blocked by CORS policy: Request header field content-type is not allowed by Access-Control-Allow-Headers in preflight response. . I was able to get away with the problem temporarily with one of the "CORS Everywhere" browser extensions, but
Due to recent updates, the extensions have not yet been updated to be working again.
The website involved needs to eventually be used by many in my local complex without the browser extension, so once the extensions get updated, it does not really matter one way or another, as I cannot rely on these extensions, and force everyone to use them (when there is obviously a fix that would make an extension not necessary).
I figure that perhaps the solutions are outdated, but am not sure.
Here is the relevant code:
On the server side (CherryPy/Python):
The CherryPy Python function being called, from the website post request
#cherrypy.expose
#cherrypy.tools.json_in()
def add_meeting(self):
data = None
id = None
start_time = None
end_time = None
title = None
userlist = None
result = {"operation": "request", "result": "success"}
if cherrypy.request.method == "POST":
data = cherrypy.request.json
id = data["id"]
start_time = data["start_time"]
end_time = data["end_time"]
title = data["title"]
userlist = data["userlist"]
# Rest of relevant code in function is left out, to take up less
# space and not post irrelevant code. That being said, I am
# positive the logic is correct, as it originally ran smoothly
# with a "Cors Everywhere" Browser Extension.
return result
Here is the area where I set up and run CherryPy
def main():
# Create the configuration file parser object and start the CherryPy server
config = ConfigParser.ConfigParser()
config.read(CONFIG_FILE)
port = config.getint('Meta', 'port')
host = config.get('Meta', 'host')
cherrypy.config.update({'server.socket_port': port,
'server.socket_host': host,
'tools.CORS.on': True})
cherrypy.quickstart(Coordinator(config))
main()
Here is the config file mentioned in the code above (CONFIG_FILE)
[Meta]
host = 0.0.0.0
port = 3000
# Rest is left out, as it is irrelevant with problem
The solutions I have tried implementing
The inclusion of the following function above the main function:
def CORS():
cherrypy.response.headers["Access-Control-Allow-Origin"] = "*"
with cherrypy.tools.CORS = cherrypy.Tool('before_handler', CORS)
2. Adding " 'cors.expose.on': True " to cherrypy.config.update above
3. Using this cherrypy-cors Python library I found online: https://pypi.org/project/cherrypy-cors/
4. The inclusion of headers in the config.update portion of the Python file
5. Adding "#cherrypy.tools.accept(media='application/json')" before "def add_meeting"
Conclusion
I've tried the solutions above together, separately, some with and without the others, and I am still stuck. Maybe some of these solutions are partially correct, and there is something extra needed with my code. I am not sure; I just cannot get it working. I do not have much experience with web development before this, so maybe (and hopefully) the solution is extremely simple. I know the code works, I just cannot get it running without a working "Cors Everywhere" browser extension for every user.
As for the versions I am running: I am using CherryPy 14.2.0 and Python 2.7.6
Any help would mean the absolute world to me, thank you.
So first, you need to set pre-flight headers when processing OPTIONS request, you can list allowed methods there.
Then, you also need to enable the cors.expose tool.
There's some usage hints in the docstring of cherrypy-cors. For example, when using a MethodDispatcher, you could just decorate an OPTIONS handler method with #cherrypy_cors.tools.preflight() instead of doing this in every HTTP handler.
Here's a simple traversal example (without a method dispatcher). To test it, visit http://127.0.0.1/ and it will make requests against http://localhost:3333/add_meeting which is a different Origin in terms of CORS ('localhost' != '127.0.0.1').
"""Example of CORS setup using cherrypy-cors library."""
import cherrypy
import cherrypy_cors
# Python 2 compat: make all classes new-style by default
__metaclass__ = type # pylint: disable=invalid-name
class WebRoot:
"""Root node for HTTP handlers."""
#cherrypy.expose
def index(self): # pylint: disable=no-self-use
"""Render a web page handling request against ``/``.
Contains client JS snippet which will query the API endpoint.
It will be executed by the browser while loading the page.
"""
return """<html>
<script type="text/javascript">
async function addMeeting() {
/*
* Example coroutine for querying /add_meeing
* HTTP endpoint. It uses localhost as in the URL.
* For testing CORS, make sure to visit
* http://127.0.0.1/ which is a different origin
* from browser's perspective.
* /
const request_payload = {
some: 'data',
listed: ['h', 'er', 'e'],
}
try {
const resp = await fetch(
'http://localhost:3333/add_meeting',
{
method: 'POST',
mode: 'cors', // Required for customizing HTTP request headers
credentials: 'same-origin',
headers: {
'Content-Type': 'application/json; charset=UTF-8', // Required for ``cherrypy.tools.json_in`` to identify JSON payload and parse it automatically
},
body: JSON.stringify(request_payload),
},
)
const json_resp = await resp.json()
console.log(json_resp) // Will print: {"method": "POST", "payload": {"listed": ["h", "er", "e"], "some": "data"}}
} catch (e) {
console.warn('Exception: ' + e)
}
}
async function main() {
await addMeeting()
}
main() // Entry point
</script>
</html>""" # noqa: E501
#cherrypy.expose
#cherrypy.tools.json_in() # turn HTTP payload into an object; also checking the Content-Type header
#cherrypy.tools.json_out() # turn ``return``ed Python object into a JSON string; also setting corresponding Content-Type
def add_meeting(self):
"""Handle HTTP requests against ``/add_meeting`` URI."""
if cherrypy.request.method == 'OPTIONS':
# This is a request that browser sends in CORS prior to
# sending a real request.
# Set up extra headers for a pre-flight OPTIONS request.
cherrypy_cors.preflight(allowed_methods=['GET', 'POST'])
if cherrypy.request.method == 'POST':
return {'method': 'POST', 'payload': cherrypy.request.json}
return {'method': 'non-POST'}
def main():
"""Set up and run the web app.
Initializes CORS tools.
Sets up web server socket.
Enables the CORS tool.
"""
cherrypy_cors.install()
cherrypy.config.update({
'server.socket_host': '127.0.0.1',
'server.socket_port': 3333,
'cors.expose.on': True,
})
cherrypy.quickstart(WebRoot())
__name__ == '__main__' and main() # pylint: disable=expression-not-assigned
Related
I am working with an app that sends data to a server with a POST request,
POST https://www.somedomain.com//sendImage HTTP/2.0
looking like this:
{
"user": {
"consent": true,
"currentNumberIs": 1,
"images": {
"data": "BASE64ENCODED IMAGE",
"docType": "avatar"
},
"totalNumberOfImages": 1
}
}
I want to replace the data part of this Json, but only if the docType is avatar. Trying to use a python script for that, that I found here and edited:
def response(flow: http.HTTPFlow) -> None:
if "somedomain.com" in flow.request.pretty_url:
request_data = json.loads(flow.request.get_text())
if request_data["user"]["images"]["docType"] == "avatar":
data = json.loads(flow.response.get_text())
data["user"]["images"]["data"] = "NEWDATA"
flow.response.text = json.dumps(data)
Launched mitmproxy with -s script.py, but according to the web console, the specific request does not trigger the script at all. Which kinda limits the scope to debug.
Would glady appreciate any help.
As #nneonneo mentioned in the comments, I would first recommend to make extensive use of mitmproxy.ctx.log() to make sure that your event hook is triggered properly. Second, if I understand things correctly, you intend to modify the request and not the response? If you want to modify request contents before they are sent to the server, you need to use the request hook and not the response hook:
def request(flow: http.HTTPFlow) -> None:
# this is executed after we have received the request
# from the client, but before it is sent to the server.
def response(flow: http.HTTPFlow) -> None:
# this is executed after we have sent the request
# to the server and received the response at the proxy.
Finally, you currently read from flow.request.text and then later assign to flow.response.text. I don't know your specific use case, but usually that should be flow.request.text as well.
You're altering the flow variable in a function, but not using the edited flow. If you return the new flow you can then use it and post it.
def response(flow: http.HTTPFlow) -> http.HTTPFlow:
if "somedomain.com" in flow.request.pretty_url:
request_data = json.loads(flow.request.get_text())
if request_data["user"]["images"]["docType"] == "avatar":
data = json.loads(flow.response.get_text())
data["user"]["images"]["data"] = "NEWDATA"
flow.response.text = json.dumps(data)
return flow
UPDATE
For me the Problem got fixed as soon as I was putting "encoding: URLEncoding(destination: .queryString)" in my request. Maybe this helps somebody else. link
I struggled the whole day to find the problem in my Alamofire PUT Request or the Flask Restful API. Request like GET, DELETE and POST are working fine with Alamofire, except the PUT Request.
When I'm using PUT Requests in combination with Postman and Flask-Restful everything is also working fine. But as soon as I'm trying to achieve the same Result with Alamofire, I'm not getting any parameters in Flask. I tried to illustrate this in the code examples.
So in short my example illustrates the following:
DELETE Request(Same with GET and POST)
Postman: success
Alamofire: success
PUT Request
Postman: success
Alamofire: failure (parameter dictionary empty in Flask-Restful)
Here is my Python Code [API Server]:
from flask import Flask, request, jsonify
from flask_restful import Resource, Api, reqparse
app = Flask(__name__)
api = Api(app)
class Stackoverflow(Resource):
def delete(self):
print(request.args)
if request.args.get('test-key') is None:
return jsonify({"message": "failure"})
else:
return jsonify({"message": "success"})
def put(self):
print(request.args)
if request.args.get('test-key') is None:
return jsonify({"message": "failure"})
else:
return jsonify({"message": "success"})
api.add_resource(Stackoverflow, '/stackoverflow')
if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0')
If I'm using Postman, I get this result (like expected):
Result in Postman
But now I'm trying to do the same with Alamofire in Swift. Same Server, nothing changed.
SWIFT demo Code [IOS APP]:
import UIKit
import Alamofire
import SwiftyJSON
class ViewController: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view
simplePUTRequest()
simpleDELETERequest()
}
func simplePUTRequest(){
AF.request("http://localhost:5000/stackoverflow", method: .put, parameters: ["test-key":"testvalue"])
.validate(statusCode: 200..<300)
.responseJSON { response in
if let data = response.data {
print("Result PUT Request:")
print(String(data: data, encoding: .utf8)!)
//print(utf8Text)
}else{
}
}
}
func simpleDELETERequest(){
AF.request("http://localhost:5000/stackoverflow", method: .delete, parameters: ["test-key":"testvalue"])
.validate(statusCode: 200..<300)
.responseJSON { response in
if let data = response.data {
print("Result DELETE Request:")
print(String(data: data, encoding: .utf8)!)
//print(utf8Text)
}else{
}
}
}
Xcode Console:
Result PUT Request:
{
"message": "failure"
}
Result DELETE Request:
{
"message": "success"
}
python Console (both Alamofire Requests):
ImmutableMultiDict([])
127.0.0.1 - - [15/Jun/2019 21:17:31] "PUT /stackoverflow HTTP/1.1" 200 -
ImmutableMultiDict([('test-key', 'testvalue')])
127.0.0.1 - - [15/Jun/2019 21:17:31] "DELETE /stackoverflow?test-key=testvalue HTTP/1.1" 200 -
As you can see, I'm getting the success message only while using the DELETE method.
Till now I tried using different encodings like URLEncoding.httpbody and URLEncoding.default, but nothing really helped.
For me it seems like it's a Alamofire/Swift Problem, because in Postman the same request method is working fine.
I would really appreciate your help, because I'm stuck and don't know anything further to do. I hope I didn't misunderstood something essential.
Thank you in advance!
I am currently using the same version AlamoFire, and when I use the PUT method, I use it as follows:
let request = AF.request(url, method: .put, parameters: ["uid": uid],
encoding: JSONEncoding.default, headers: headers)
request.responseJSON(completionHandler: { response in
guard response.error == nil else {
//Handle error
}
if let json = response.value as? [String: Any]
// Handle result.
}
The only difference to your post is that I used the encoding option. You can try to put the option and see what happens.
It looks like your server is expecting your PUT parameters to be URL form encoded into the URL. You may be hitting the version of the request method that uses JSON encoding by default, so adding encoder: URLEncodedFormParameterEncoder.default at the end of your request call should fix that. A future release will make that the default, as it's safe across all request types.
If that's not the issue, I suggest you investigate more closely to see what the differences between the requests may be. Since you control the server you should have easy access to the traffic.
I want to create a Flask error handler that returns a JSON response if the request was from JavaScript but returns a redirect otherwise. I tried using request.is_xhr, but it is false even for JavaScript requests. How can I check if the request is from JavaScript?
#app.errorhandler(Exception)
def unhandled_exception(error):
if request.is_xhr:
return flask.jsonify(error='yes')
return redirect(url_for('error'))
There is no standard or reliable way to detect if a request comes from a particular source, such as JavaScript.
is_xhr was only true when a certain header was set by some JavaScript libraries, such as jQuery. The header is not sent by most JavaScript. is_xhr has been deprecated for that reason.
You can check the Accept header to see if the client is asking for application/json, but that too is unreliable.
if request.is_xhr or request.accept_mimetypes.accept_json:
return jsonify(...)
return redirect(...)
Answer by #davidism makes sense. is_xhr was only true when a certain header was set by some JavaScript libraries. So, I have set header 'X-Requested-With' to 'XMLHttpRequest' manually in '$httpProvider' config in AngularJs. This ensures that on the back end I will get 'is_xhr' true for AJAX request.
app.config([
'$httpProvider',
function ($httpProvider) {
$httpProvider.defaults.headers.common['X-Requested-With'] = 'XMLHttpRequest';
var interceptor = [
'$q',
'$rootScope',
function ($q, $rootScope) {
'responseError': function (rejection) {
if(rejection.status != undefined && rejection.status != 'undefined') {
window.location.href = '/error';
}
}
};
return service;
}
];
$httpProvider.interceptors.push(interceptor);
}
]);
Rather than tie my app to a non-standard header I added an Accept: header to my Javascript instead:
let req = new XMLHttpRequest();
req.open('POST', location);
// signal back-end to return json instead of rendering a full page:
req.setRequestHeader('Accept', 'application/json');
req.send(…);
And in my Python:
# if an ajax-like request, return json instead of html
if request.accept_mimetypes.best == 'application/json':
log.debug('client prefers json, skipping page render.')
return jsonify(status='errror', detail='…')
This should handle other use cases as they come up.
If the request is from a javascript/jquery code, is it most certainly from a browser so you can check the flask.request.user_agent object which is a instance of werkzeug.useragents.UserAgent to verify that.
I have read that cherrypy uses a threadpool of its own. But I am unable to see the advantage of that.
Let's say I fire off a request which will take a long time and after that in another tab I fire off a request which will take a short time. If it really uses multithreading, the short request should complete before the long one. But I am seeing that first the long request gets completed and then the short time, as if everything is processed sequentially.
I have tried integrating with different uWSGI frameworks like Tornado and twistd, but still I don't see a difference.
http://cherrypy.readthedocs.org/en/latest/deploy.html#tornado
This is my starter code. Can anyone help me out here?
cfg = {
'global' : {
'server.socket_host' : Utils.gflags.FLAGS.bind_addr,
'server.socket_port' : Utils.gflags.FLAGS.bind_port,
'server.thread_pool' : 10,
'engine.timeout_monitor.frequency' : gflags.FLAGS.request_time_out_secs,
},
'/static' : {"tools.sessions.on": False, 'tools.auth.on': False},
'/favicon.ico' : {"tools.sessions.on": False, 'tools.auth.on': False},
}
# To turn off the cherrypy errors on screen.
cfg['global'].update({'log.screen': False})
cfg['/static'].update({'tools.staticdir.on': True})
cfg['/static'].update({'tools.staticdir.dir': Utils.gflags.FLAGS.static_dir})
cfg['/favicon.ico'].update({'tools.staticfile.on': True})
cfg['/favicon.ico'].update({'tools.staticfile.filename':
Utils.gflags.FLAGS.favicon_file})
# Disable the auto reload on code change.
cherrypy.engine.autoreload.unsubscribe()
# Start the cherrypy
#Root() is defined somewhere else. Don't worry about that
cherrypy.quickstart(Root(), config = cfg)
Yes it looks like you're having the same issue mentioned in this blog post about session locking: http://blog.schmichael.com/2007/09/20/session-locking-and-performance-in-cherrypy/
Basically the solution is to explicitly lock your sessions at a different point in the code where it won't block all other requests.
cherrypy.session.acquire_lock()
cherrypy.session.release_lock()
I see that you are disabling the session tool on the static files, so I assume that the blocking is caused by blocking sessions, check out the documentation related to session locking.
This example may be illustrative:
"""Show the difference between explicit and implicit locking
on the cherrypy sessions.
To see the effects make sure your client can handle cookies,
for example any conventional web browser or curl with
a cookie jar.
The exposed routes are:
/e/
/e/block/[minutes]
/e/blocked_hi/
/e/unblocked_hi
/i/
/i/block/[minutes]
/i/blocked_hi/
/i/unblocked_hi
The application mounted on /e/ has the sessions *explicitly* locked and
the applicaiton mounted on /i/ has the sessions *implicitly* locked.
You can make any concurrent request on the /e branch and you
will not have any blocking.
If you do the requests on the following steps:
1. /i/
2. /i/block
3. /i/blocked_hi
The step 3 is going to be blocked because of the step 2, you can wait a minute
and when the request on step 2 ends it will inmediatly complete the step 3.
Also any request that you do to /i/unblocked_hi will respond immediately regardless
of any blocking.
In general if you call:
1. /i/ or /e/ and then
2. /i/block
3. Any request to:
/i/
/i/blocked_hi
/e/
are going to be blocked in until /i/block finish.
"""
import time
import cherrypy as cp
class _App:
#cp.expose
def block(self, m=1):
"""Sleep for `m` minutes and return."""
time.sleep(float(m) * 60)
return "I have blocked this request {}".format(m)
#cp.expose
def blocked_hi(self):
"""It can be blocked if the blocked method is executing,
the session have content and is locked.
"""
return """Hi, I could have been blocked by a session.
Session content: {}\n""".format(dict(cp.session))
#cp.expose
def unblocked_hi(self):
return "Hi, I'm not blocked!"
class ImplicitlyLockedApp(_App):
#cp.expose
def index(self):
cp.session['foo'] = 'bar'
return "I've just set the session content to {}".format(dict(cp.session))
class ExplicitlyLockedApp(_App):
#cp.expose
def index(self):
# This method can be blocked by /i/block because of the
# acquire_lock/release_lock calls.
cp.session.acquire_lock()
cp.session['foo'] = 'bar'
cp.session.release_lock()
return "I've just set the session content to {}".format(dict(cp.session))
if __name__ == '__main__':
cp.tree.mount(ImplicitlyLockedApp(), '/i', config={
'/': {
'tools.sessions.on': True
},
'/unblocked_hi': { # Disable the session tool to avoid any locking
'tools.sessions.on': False
}
})
cp.tree.mount(ExplicitlyLockedApp(), '/e', config={
'/': {
'tools.sessions.on': True,
'tools.sessions.locking': 'explicit' # This is the magic bit.
},
'/unblocked_hi': { # Rather irrelevant on this case
'tools.sessions.on': False
}
})
cp.engine.start()
cp.engine.block()
I am using autobahnpython with twisted (wamp) on server side and autobahnjs in browser. Is there a straight-forward way to allow/restrict subscriptions on a per session basis? For example, a client should not be able to subscribe to topics relavant to other users.
While I am NOT using crossbar.io, I tried using the Python code shown in the 'Example' section at the end of this page http://crossbar.io/docs/Authorization/ where a RPC call is first used to give authorization to a client. Of course, I am using my own authorization logic. Once this authorization is successful, I'd like to give the client privileges to subscribe to topics related only to this client, like 'com.example.user_id'. My issue is that even if auth passes, however, I have not found a way to limit subscription requests in the ApplicationSession class which is where the authorization takes place. How can I prevent a client who authorizes with user_id=user_a from subscribing to 'com.example.user_b'?
You can authorize by creating your own router. To do that, subclass Router() and override (at a minumum) the authorize() method:
def authorize(self, session, uri, action):
return True
This method is pretty simple, if you return a True then the session is authorized to do whatever it is attempting. You could make a rule that all subscriptions must start with 'com.example.USER_ID', so, your python code would split the uri, take the third field, and compare it to the current session id, returning True if they match, false otherwise. This is where things get a little weird though. I have code that does a similar thing, here is my authorize() method:
#inlineCallbacks
def authorize(self, session, uri, action):
authid = session._authid
if authid is None:
authid = 1
log.msg("AuthorizeRouter.authorize: {} {} {} {} {}".format(authid,
session._session_id, uri, IRouter.ACTION_TO_STRING[action], action))
if authid != 1:
rv = yield self.check_permission(authid, uri, IRouter.ACTION_TO_STRING[action])
else:
rv = yield True
log.msg("AuthorizeRouter.authorize: rv is {}".format(rv))
if not uri.startswith(self.svar['topic_base']):
self.sessiondb.activity(session._session_id, uri, IRouter.ACTION_TO_STRING[action], rv)
returnValue(rv)
return
Note that I dive into the session to get the _authid, which is bad karma (I think) because I should not be looking at these private variables. I don't know where else to get it, though.
Also, of note, this goes hand in hand with Authentication. In my implementation, the _authid is the authenticated user id, which is similar to a unix user id (positive unique integer). I am pretty sure this can be anything, like a string, so you should be ok with your 'user_b' as the _auth_id if you wish.
-g
I found a relatively simple solution using a Node guest. Here's the code:
// crossbar setup
var autobahn = require('autobahn');
var connection = new autobahn.Connection({
url: 'ws://127.0.0.1:8080/ws',
realm: 'realm1'
}
);
// Websocket to Scratch setup
// pull in the required node packages and assign variables for the entities
var WebSocketServer = require('websocket').server;
var http = require('http');
var ipPort = 1234; // ip port number for Scratch to use
// this connection is a crossbar connection
connection.onopen = function (session) {
// create an http server that will be used to contain a WebSocket server
var server = http.createServer(function (request, response) {
// We are not processing any HTTP, so this is an empty function. 'server' is a wrapper for the
// WebSocketServer we are going to create below.
});
// Create an IP listener using the http server
server.listen(ipPort, function () {
console.log('Webserver created and listening on port ' + ipPort);
});
// create the WebSocket Server and associate it with the httpServer
var wsServer = new WebSocketServer({
httpServer: server
});
// WebSocket server has been activated and a 'request' message has been received from client websocket
wsServer.on('request', function (request) {
// accept a connection request from Xi4S
//myconnection is the WS connection to Scratch
myconnection = request.accept(null, request.origin); // The server is now 'online'
// Process Xi4S messages
myconnection.on('message', function (message) {
console.log('message received: ' + message.utf8Data);
session.publish('com.serial.data', [message.utf8Data]);
// Process each message type received
myconnection.on('close', function (myconnection) {
console.log('Client closed connection');
boardReset();
});
});
});
};
connection.open();