Develop Reference

How to set up CORS in CherryPy

Overview
When creating a post request from my website to my Python server running CherryPy, I receive the error Access to XMLHttpRequest has been blocked by CORS policy: Request header field content-type is not allowed by Access-Control-Allow-Headers in preflight response. . I was able to get away with the problem temporarily with one of the "CORS Everywhere" browser extensions, but
Due to recent updates, the extensions have not yet been updated to be working again.
The website involved needs to eventually be used by many in my local complex without the browser extension, so once the extensions get updated, it does not really matter one way or another, as I cannot rely on these extensions, and force everyone to use them (when there is obviously a fix that would make an extension not necessary).
I figure that perhaps the solutions are outdated, but am not sure.
Here is the relevant code:
On the server side (CherryPy/Python):
The CherryPy Python function being called, from the website post request
#cherrypy.expose
#cherrypy.tools.json_in()
def add_meeting(self):
data = None
id = None
start_time = None
end_time = None
title = None
userlist = None
result = {"operation": "request", "result": "success"}
if cherrypy.request.method == "POST":
data = cherrypy.request.json
id = data["id"]
start_time = data["start_time"]
end_time = data["end_time"]
title = data["title"]
userlist = data["userlist"]
# Rest of relevant code in function is left out, to take up less
# space and not post irrelevant code. That being said, I am
# positive the logic is correct, as it originally ran smoothly
# with a "Cors Everywhere" Browser Extension.
return result
Here is the area where I set up and run CherryPy
def main():
# Create the configuration file parser object and start the CherryPy server
config = ConfigParser.ConfigParser()
config.read(CONFIG_FILE)
port = config.getint('Meta', 'port')
host = config.get('Meta', 'host')
cherrypy.config.update({'server.socket_port': port,
'server.socket_host': host,
'tools.CORS.on': True})
cherrypy.quickstart(Coordinator(config))
main()
Here is the config file mentioned in the code above (CONFIG_FILE)
[Meta]
host = 0.0.0.0
port = 3000
# Rest is left out, as it is irrelevant with problem
The solutions I have tried implementing
The inclusion of the following function above the main function:
def CORS():
cherrypy.response.headers["Access-Control-Allow-Origin"] = "*"
with cherrypy.tools.CORS = cherrypy.Tool('before_handler', CORS)
2. Adding " 'cors.expose.on': True " to cherrypy.config.update above
3. Using this cherrypy-cors Python library I found online: https://pypi.org/project/cherrypy-cors/
4. The inclusion of headers in the config.update portion of the Python file
5. Adding "#cherrypy.tools.accept(media='application/json')" before "def add_meeting"
Conclusion
I've tried the solutions above together, separately, some with and without the others, and I am still stuck. Maybe some of these solutions are partially correct, and there is something extra needed with my code. I am not sure; I just cannot get it working. I do not have much experience with web development before this, so maybe (and hopefully) the solution is extremely simple. I know the code works, I just cannot get it running without a working "Cors Everywhere" browser extension for every user.
As for the versions I am running: I am using CherryPy 14.2.0 and Python 2.7.6
Any help would mean the absolute world to me, thank you.

So first, you need to set pre-flight headers when processing OPTIONS request, you can list allowed methods there.
Then, you also need to enable the cors.expose tool.
There's some usage hints in the docstring of cherrypy-cors. For example, when using a MethodDispatcher, you could just decorate an OPTIONS handler method with #cherrypy_cors.tools.preflight() instead of doing this in every HTTP handler.
Here's a simple traversal example (without a method dispatcher). To test it, visit http://127.0.0.1/ and it will make requests against http://localhost:3333/add_meeting which is a different Origin in terms of CORS ('localhost' != '127.0.0.1').
"""Example of CORS setup using cherrypy-cors library."""
import cherrypy
import cherrypy_cors
# Python 2 compat: make all classes new-style by default
__metaclass__ = type # pylint: disable=invalid-name
class WebRoot:
"""Root node for HTTP handlers."""
#cherrypy.expose
def index(self): # pylint: disable=no-self-use
"""Render a web page handling request against ``/``.
Contains client JS snippet which will query the API endpoint.
It will be executed by the browser while loading the page.
"""
return """<html>
<script type="text/javascript">
async function addMeeting() {
/*
* Example coroutine for querying /add_meeing
* HTTP endpoint. It uses localhost as in the URL.
* For testing CORS, make sure to visit
* http://127.0.0.1/ which is a different origin
* from browser's perspective.
* /
const request_payload = {
some: 'data',
listed: ['h', 'er', 'e'],
}
try {
const resp = await fetch(
'http://localhost:3333/add_meeting',
{
method: 'POST',
mode: 'cors', // Required for customizing HTTP request headers
credentials: 'same-origin',
headers: {
'Content-Type': 'application/json; charset=UTF-8', // Required for ``cherrypy.tools.json_in`` to identify JSON payload and parse it automatically
},
body: JSON.stringify(request_payload),
},
)
const json_resp = await resp.json()
console.log(json_resp) // Will print: {"method": "POST", "payload": {"listed": ["h", "er", "e"], "some": "data"}}
} catch (e) {
console.warn('Exception: ' + e)
}
}
async function main() {
await addMeeting()
}
main() // Entry point
</script>
</html>""" # noqa: E501
#cherrypy.expose
#cherrypy.tools.json_in() # turn HTTP payload into an object; also checking the Content-Type header
#cherrypy.tools.json_out() # turn ``return``ed Python object into a JSON string; also setting corresponding Content-Type
def add_meeting(self):
"""Handle HTTP requests against ``/add_meeting`` URI."""
if cherrypy.request.method == 'OPTIONS':
# This is a request that browser sends in CORS prior to
# sending a real request.
# Set up extra headers for a pre-flight OPTIONS request.
cherrypy_cors.preflight(allowed_methods=['GET', 'POST'])
if cherrypy.request.method == 'POST':
return {'method': 'POST', 'payload': cherrypy.request.json}
return {'method': 'non-POST'}
def main():
"""Set up and run the web app.
Initializes CORS tools.
Sets up web server socket.
Enables the CORS tool.
"""
cherrypy_cors.install()
cherrypy.config.update({
'server.socket_host': '127.0.0.1',
'server.socket_port': 3333,
'cors.expose.on': True,
})
cherrypy.quickstart(WebRoot())
__name__ == '__main__' and main() # pylint: disable=expression-not-assigned

How to write a GRPC python unittest

We are using grpc as the RPC protocol for all our internal system. Most of the system is written in Java.
In Java, we can use InprocessServerBuilder for unittest. However, I haven't find a similar class in Python.
Can any one provide a sample code for how to do GRPC unittest in python?

How serendipitous that you have asked this question today; our unit test framework just entered code review. So for the time being the way to test is to use the full production stack to connect your client-side and server-side code (or to violate the API and mock a lot of internal stuff) but hopefully in days to weeks the much better solution will be available to you.

Some example code to get started:
proto
syntax = "proto3";
service MyLibrary {
rpc Search (Request) returns (Response);
}
message Request {
string id = 1;
}
message Response {
string status = 1;
}
python unit test
#!/usr/bin/env python
# coding=utf-8
import unittest
from grpc import StatusCode
from grpc_testing import server_from_dictionary, strict_real_time
import mylibrary_pb2
class TestCase(unittest.TestCase):
def __init__(self, methodName) -> None:
super().__init__(methodName)
myServicer = MyLibraryServicer()
servicers = {
mylibrary_pb2.DESCRIPTOR.services_by_name['MyLibrary']: myServicer
}
self.test_server = server_from_dictionary(
servicers, strict_real_time())
def test_search(self):
request = mylibrary_pb2.Request(
id=2,
)
method = self.test_server.invoke_unary_unary(
method_descriptor=(mylibrary_pb2.DESCRIPTOR
.services_by_name['Library']
.methods_by_name['Search']),
invocation_metadata={},
request=request, timeout=1)
response, metadata, code, details = method.termination()
self.assertTrue(bool(response.status))
self.assertEqual(code, StatusCode.OK)
if __name__ == '__main__':
unittest.main()

I find pytest-grpc is easy to follow and get it works in few minutes.
src: https://pypi.org/project/pytest-grpc/

Django test VS pytest

I am new to django unittest and pytest. However, I started to feel that pytest test case is more compact and clearer.
Here is my test cases:
class OrderEndpointTest(TestCase):
def setUp(self):
user = User.objects.create_superuser(username='admin', password='password', email='pencil#gmail.com')
mommy.make(CarData, _quantity=1)
mommy.make(UserProfile, _quantity=1, user=user)
def test_get_order(self):
mommy.make(Shop, _quantity=1)
mommy.make(Staff, _quantity=1, shop=Shop.objects.first())
mommy.make(Order, _quantity=1, car_info={"color": "Black"}, customer={"name": "Lord Elcolie"},
staff=Staff.objects.first(), shop=Shop.objects.first())
factory = APIRequestFactory()
user = User.objects.get(username='admin')
view = OrderViewSet.as_view({'get': 'list'})
request = factory.get('/api/orders/')
force_authenticate(request, user=user)
response = view(request)
assert 200 == response.status_code
assert 1 == len(response.data.get('results'))
And here is the pytest version
def test_get_order(car_data, admin_user, orders):
factory = APIRequestFactory()
user = User.objects.get(username='admin')
view = OrderViewSet.as_view({'get': 'list'})
request = factory.get('/api/orders/')
force_authenticate(request, user=user)
response = view(request)
assert 200 == response.status_code
assert 1 == len(response.data.get('results'))
The benefit from pytest is fixture in another file. It makes my test case compact by let them be my input parameters.
Are they any benefit of using Django unittest than pytest?
Update: 1July2017
Update: 5July2017
Update: 1Sep2017
Update: 29Sep2017
Update: 26Dec2017
Pytest reduces your problem when fixtures got mutated over the test.
I got testcases that run individually passed, but fail when run
thoroughly.
Pytest will show you the assertion output if the error occur. Django
unittest does not. I have to put the breakpoint on my own and
investigate the error.
Pytest allow you to use real database with simple decorator. Django
test does not. You have to create your own customized command for
your job
Pytest is generic. Being an generic it means you feel comfortable to
work with project outside the Django. For example when you have to
build micro-service such as Flask + 3rd parties like APScheduler,
PyRad, ... etc. I mention this because my backend life uses Django 50%
The rest of the is Python and infra
Pytest is not using multiple inheritance to create my fixtures
Unittest takes advantage on gitlab-ci over Pytest when used with Docker as a runner by smoothly execute without any extra configurations. problem

i've used Django test for my entire life and now i am using Py.test. I agree that pytest is much cleaner than django itself.
The benefit from pytest is fixture in another file. It makes my test case compact by let them be my input parameters.
In Django unittest you can still use fixtures in other file by using the attribute fixtures = ['appname/fixtures/my_fixture.json']
Pytest will show you the assertion output if the error occur. Django unittest does not. I have to put the breakpoint on my own and investigate the error.
Did you tried to change the --verbose param on python manage.py tests?
A few tips:
There is a package called pytest-django that will help you integrate and using django with pytest.
I think that if you use classes will you not need to use the factory = APIRequestFactory(), the test methods itself they have a parameter called client that is a interface to the python requests module to access your views.
import pytest
from model_mommy import mommy
#pytest.fixture()
def user(db):
return mommy.make(User)
class SiteAPIViewTestSuite:
def test_create_view(self, client, user):
assert Site.objects.count() == 0
post_data = {
'name': 'Stackoverflow'
'url': 'http://stackoverflow.com',
'user_id': user.id,
}
response = client.post(
reverse('sites:create'),
json.dumps(post_data),
content_type='application/json',
)
data = response.json()
assert response.status_code == 201
assert Site.objects.count() == 1
assert data == {
'count': 1,
'next': None,
'previous': None
'results': [{
'pk': 1,
'name': 'Stackoverflow',
'url': 'http://stackoverflow.com',
'user_id': user.id
}]
}

Breakpoints inside Flask-RESTful class methods aren't hit on PTVS

I'm using Python Tools for Visual Studio, and I've set up a project with a virtual environment and installed Flask-RESTful there.
Then, I just copied their hello world example
from flask import Flask
from flask.ext.restful import reqparse, abort, Api, Resource
app = Flask(__name__)
app.debug = True
api = Api(app)
TODOS = {
'todo1': {'task': 'build an API'},
'todo2': {'task': '?????'},
'todo3': {'task': 'profit!'},
}
def abort_if_todo_doesnt_exist(todo_id):
if todo_id not in TODOS:
abort(404, message="Todo {} doesn't exist".format(todo_id))
parser = reqparse.RequestParser()
parser.add_argument('task', type=str)
# Todo
# show a single todo item and lets you delete them
class Todo(Resource):
def get(self, todo_id):
abort_if_todo_doesnt_exist(todo_id)
return TODOS[todo_id]
def delete(self, todo_id):
abort_if_todo_doesnt_exist(todo_id)
del TODOS[todo_id]
return '', 204
def put(self, todo_id):
args = parser.parse_args()
task = {'task': args['task']}
TODOS[todo_id] = task
return task, 201
# TodoList
# shows a list of all todos, and lets you POST to add new tasks
class TodoList(Resource):
def get(self):
return TODOS
def post(self):
args = parser.parse_args()
todo_id = 'todo%d' % (len(TODOS) + 1)
TODOS[todo_id] = {'task': args['task']}
return TODOS[todo_id], 201
##
## Actually setup the Api resource routing here
##
api.add_resource(TodoList, '/todos')
api.add_resource(Todo, '/todos/<string:todo_id>')
if __name__ == '__main__':
app.run(debug=True)
Everything works fine, and if I put breakpoints on the lines that are executed before starting the app.run(debug=True) they're hit (with F10 and F11 working great, and local variables being updated as expected)
However, I'd love to debug what happens when a request is processed, but if I add breakpoints to the methods of the Todo or TodoList classes, they're never hit. I added code (like print('here')) to see if they're being processed and they are... also, they're returning what I expect when opened from a browser.
Is there something I'm missing from the setup?
Thanks!
UPDATE: I found out that if I attach VS to the python.exe process that's running my code, I'm able to debug those methods... so I guess that the question is now: can I force VS to attach to the process once it's launched as it does with regular .NET apps?

I had the same exact problem (but with PyCharm on Mac).
I believe this has something to do with the way Flask reloads itself when debug=True. Setting debug to False allowed me to break inside the view methods.

Disabling Debug resolves the issue with the breakpoint hit, but has the disadvantage that you are unable to read exception trace output.
One way to bypass this limitation is to add:
DEBUG = False
PROPAGATE_EXCEPTIONS = True
to your config.
When an exception occurs the browser still displays "Internal Server Error" message, but the console Window will receive the exception trace normally.

Creating a basic JSONRPC API in CherryPy - Getting 404s

The title is fairly self-explanatory, so I'll just show some code I've tried so far:
From https://stackoverflow.com/a/713950,
import cherrypy
from cherrypy import expose
cherrypy.config.update({'server.socket_port': 80})
class Test:
#expose
def test_call(self):
return "Testing"
cherrypy.quickstart(Test())
Also, from another SO post, two variants on the following:
cherrypy.config.update({
'server.socket_port': 80,
'/': {
'request.dispatch': cherrypy.dispatch.MethodDispatcher(),
'tools.trailing_slash.on': False
}
})
class Test:
def test_call(self, *args):
return json.dumps(args)
test_call.exposed = True
class API:
def __init__(self):
self.Test = Test()
class Root:
def __init__(self):
self.API = API()
cherrypy.tree.mount(Root())
cherrypy.quickstart(Root())
with a variation suggested here: Path Not Found in CherryPy
cherrypy.quickstart(cherrypy.Application(Root(), '/', {}))
I run these and access http://mysite.com/test_call or, in the other case, mysite.com/api/test/test_call, and neither of these seem to be doing much of anything except returning a 404. Ideas?
I am completely open to trying a different framework if it'll just let me expose a few function calls to dump JSON. I don't need anything fancy or cool, just functioning.
EDIT: Apparently my problem was that the server was by default expecting to be localhost, which basically makes me an idiot. Adding cherrypy.server.socket_host = "mydomain.com" fixes this.

The title doesn't match with examples and is telling that you were likely misguided by REST adepts, in the answer's comments you linked, who tend call everything a "RPC", which deviates from their CRUD-limited resource perspective. JSON RPC is the certain specification, which defines JSON structures for request and response. It looks like this.
--> {"jsonrpc": "2.0", "method": "subtract", "params": [42, 23], "id": 1}
<-- {"jsonrpc": "2.0", "result": 19, "id": 1}
Your examples have nothing to do with it, as well as with REST. They are a copy-paste without understanding the subject. Let's sort out a little.
CherryPy has serveral dispatching options. Default dispatcher maps request URL segments to Python object tree, like /API/Test to root.API.Test in your second example. Its common use is regular GET/POST web flow.
If you want to implement a RESTful API, here's dedicated manual page for it: Creating RESTful applications in CherryPy. It briefly explains the style and usage of MethodDispatcher.
If actual JSON RPC is what you want, then you can look at python-jsonrpc package, which has CherryPy adapter.
Though, if all you want to achieve is to return a JSON string with appropriate content-type header, CherryPy has a specific tool for it, so there's no point to do it manually.
Here follows example for #4.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import cherrypy
config = {
'global' : {
'server.socket_host' : '127.0.0.1',
'server.socket_port' : 8080,
'server.thread_pool' : 8
}
}
class Api:
#cherrypy.expose
#cherrypy.tools.json_out()
def oneway(self):
'''Just open http://127.0.0.1:8080/api/oneway'''
return {
'foo' : 'bar',
'baz' : 'another one'
}
#cherrypy.expose
#cherrypy.tools.json_in()
#cherrypy.tools.json_out()
def twoway(self):
'''You can call it like:
curl -X POST -H "Content-Type: application/json" \
-d '{"foo":123,"bar":"baz"}' http://127.0.0.1:8080/api/twoway
'''
data = cherrypy.request.json
return data.items()
if __name__ == '__main__':
cherrypy.quickstart(Api(), '/api', config)

I tried the first script as below (with 2 notes, using root user if using port 80). Access it by "http:// 127.0.0.1 /test_call". It works.
You should be more specific to raise your question (giving your codes), so the audience knows how to help solve it.
#! /usr/bin/python3
import cherrypy
from cherrypy import expose
## NOTE 1 - using 0.0.0.0
cherrypy.config.update({'server.socket_host' : '0.0.0.0', 'server.socket_port': 80})
class Test:
#expose
def test_call(self):
return "Testing"
## NOTE 2 - a work around for python3 and cherrypy v3.x
## work around OSERR issue - OSError: Port 7766 not bound on '10.220.203.233'
## refer to http://stackoverflow.com/questions/767575/crp-hello-world-error
def fake_wait_for_occupied_port(host, port): return
cherrypy.process.servers.wait_for_occupied_port = fake_wait_for_occupied_port
cherrypy.quickstart(Test())