The title is fairly self-explanatory, so I'll just show some code I've tried so far:
From https://stackoverflow.com/a/713950,
import cherrypy
from cherrypy import expose
cherrypy.config.update({'server.socket_port': 80})
class Test:
#expose
def test_call(self):
return "Testing"
cherrypy.quickstart(Test())
Also, from another SO post, two variants on the following:
cherrypy.config.update({
'server.socket_port': 80,
'/': {
'request.dispatch': cherrypy.dispatch.MethodDispatcher(),
'tools.trailing_slash.on': False
}
})
class Test:
def test_call(self, *args):
return json.dumps(args)
test_call.exposed = True
class API:
def __init__(self):
self.Test = Test()
class Root:
def __init__(self):
self.API = API()
cherrypy.tree.mount(Root())
cherrypy.quickstart(Root())
with a variation suggested here: Path Not Found in CherryPy
cherrypy.quickstart(cherrypy.Application(Root(), '/', {}))
I run these and access http://mysite.com/test_call or, in the other case, mysite.com/api/test/test_call, and neither of these seem to be doing much of anything except returning a 404. Ideas?
I am completely open to trying a different framework if it'll just let me expose a few function calls to dump JSON. I don't need anything fancy or cool, just functioning.
EDIT: Apparently my problem was that the server was by default expecting to be localhost, which basically makes me an idiot. Adding cherrypy.server.socket_host = "mydomain.com" fixes this.
The title doesn't match with examples and is telling that you were likely misguided by REST adepts, in the answer's comments you linked, who tend call everything a "RPC", which deviates from their CRUD-limited resource perspective. JSON RPC is the certain specification, which defines JSON structures for request and response. It looks like this.
--> {"jsonrpc": "2.0", "method": "subtract", "params": [42, 23], "id": 1}
<-- {"jsonrpc": "2.0", "result": 19, "id": 1}
Your examples have nothing to do with it, as well as with REST. They are a copy-paste without understanding the subject. Let's sort out a little.
CherryPy has serveral dispatching options. Default dispatcher maps request URL segments to Python object tree, like /API/Test to root.API.Test in your second example. Its common use is regular GET/POST web flow.
If you want to implement a RESTful API, here's dedicated manual page for it: Creating RESTful applications in CherryPy. It briefly explains the style and usage of MethodDispatcher.
If actual JSON RPC is what you want, then you can look at python-jsonrpc package, which has CherryPy adapter.
Though, if all you want to achieve is to return a JSON string with appropriate content-type header, CherryPy has a specific tool for it, so there's no point to do it manually.
Here follows example for #4.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import cherrypy
config = {
'global' : {
'server.socket_host' : '127.0.0.1',
'server.socket_port' : 8080,
'server.thread_pool' : 8
}
}
class Api:
#cherrypy.expose
#cherrypy.tools.json_out()
def oneway(self):
'''Just open http://127.0.0.1:8080/api/oneway'''
return {
'foo' : 'bar',
'baz' : 'another one'
}
#cherrypy.expose
#cherrypy.tools.json_in()
#cherrypy.tools.json_out()
def twoway(self):
'''You can call it like:
curl -X POST -H "Content-Type: application/json" \
-d '{"foo":123,"bar":"baz"}' http://127.0.0.1:8080/api/twoway
'''
data = cherrypy.request.json
return data.items()
if __name__ == '__main__':
cherrypy.quickstart(Api(), '/api', config)
I tried the first script as below (with 2 notes, using root user if using port 80). Access it by "http:// 127.0.0.1 /test_call". It works.
You should be more specific to raise your question (giving your codes), so the audience knows how to help solve it.
#! /usr/bin/python3
import cherrypy
from cherrypy import expose
## NOTE 1 - using 0.0.0.0
cherrypy.config.update({'server.socket_host' : '0.0.0.0', 'server.socket_port': 80})
class Test:
#expose
def test_call(self):
return "Testing"
## NOTE 2 - a work around for python3 and cherrypy v3.x
## work around OSERR issue - OSError: Port 7766 not bound on '10.220.203.233'
## refer to http://stackoverflow.com/questions/767575/crp-hello-world-error
def fake_wait_for_occupied_port(host, port): return
cherrypy.process.servers.wait_for_occupied_port = fake_wait_for_occupied_port
cherrypy.quickstart(Test())
Related
Overview
When creating a post request from my website to my Python server running CherryPy, I receive the error Access to XMLHttpRequest has been blocked by CORS policy: Request header field content-type is not allowed by Access-Control-Allow-Headers in preflight response. . I was able to get away with the problem temporarily with one of the "CORS Everywhere" browser extensions, but
Due to recent updates, the extensions have not yet been updated to be working again.
The website involved needs to eventually be used by many in my local complex without the browser extension, so once the extensions get updated, it does not really matter one way or another, as I cannot rely on these extensions, and force everyone to use them (when there is obviously a fix that would make an extension not necessary).
I figure that perhaps the solutions are outdated, but am not sure.
Here is the relevant code:
On the server side (CherryPy/Python):
The CherryPy Python function being called, from the website post request
#cherrypy.expose
#cherrypy.tools.json_in()
def add_meeting(self):
data = None
id = None
start_time = None
end_time = None
title = None
userlist = None
result = {"operation": "request", "result": "success"}
if cherrypy.request.method == "POST":
data = cherrypy.request.json
id = data["id"]
start_time = data["start_time"]
end_time = data["end_time"]
title = data["title"]
userlist = data["userlist"]
# Rest of relevant code in function is left out, to take up less
# space and not post irrelevant code. That being said, I am
# positive the logic is correct, as it originally ran smoothly
# with a "Cors Everywhere" Browser Extension.
return result
Here is the area where I set up and run CherryPy
def main():
# Create the configuration file parser object and start the CherryPy server
config = ConfigParser.ConfigParser()
config.read(CONFIG_FILE)
port = config.getint('Meta', 'port')
host = config.get('Meta', 'host')
cherrypy.config.update({'server.socket_port': port,
'server.socket_host': host,
'tools.CORS.on': True})
cherrypy.quickstart(Coordinator(config))
main()
Here is the config file mentioned in the code above (CONFIG_FILE)
[Meta]
host = 0.0.0.0
port = 3000
# Rest is left out, as it is irrelevant with problem
The solutions I have tried implementing
The inclusion of the following function above the main function:
def CORS():
cherrypy.response.headers["Access-Control-Allow-Origin"] = "*"
with cherrypy.tools.CORS = cherrypy.Tool('before_handler', CORS)
2. Adding " 'cors.expose.on': True " to cherrypy.config.update above
3. Using this cherrypy-cors Python library I found online: https://pypi.org/project/cherrypy-cors/
4. The inclusion of headers in the config.update portion of the Python file
5. Adding "#cherrypy.tools.accept(media='application/json')" before "def add_meeting"
Conclusion
I've tried the solutions above together, separately, some with and without the others, and I am still stuck. Maybe some of these solutions are partially correct, and there is something extra needed with my code. I am not sure; I just cannot get it working. I do not have much experience with web development before this, so maybe (and hopefully) the solution is extremely simple. I know the code works, I just cannot get it running without a working "Cors Everywhere" browser extension for every user.
As for the versions I am running: I am using CherryPy 14.2.0 and Python 2.7.6
Any help would mean the absolute world to me, thank you.
So first, you need to set pre-flight headers when processing OPTIONS request, you can list allowed methods there.
Then, you also need to enable the cors.expose tool.
There's some usage hints in the docstring of cherrypy-cors. For example, when using a MethodDispatcher, you could just decorate an OPTIONS handler method with #cherrypy_cors.tools.preflight() instead of doing this in every HTTP handler.
Here's a simple traversal example (without a method dispatcher). To test it, visit http://127.0.0.1/ and it will make requests against http://localhost:3333/add_meeting which is a different Origin in terms of CORS ('localhost' != '127.0.0.1').
"""Example of CORS setup using cherrypy-cors library."""
import cherrypy
import cherrypy_cors
# Python 2 compat: make all classes new-style by default
__metaclass__ = type # pylint: disable=invalid-name
class WebRoot:
"""Root node for HTTP handlers."""
#cherrypy.expose
def index(self): # pylint: disable=no-self-use
"""Render a web page handling request against ``/``.
Contains client JS snippet which will query the API endpoint.
It will be executed by the browser while loading the page.
"""
return """<html>
<script type="text/javascript">
async function addMeeting() {
/*
* Example coroutine for querying /add_meeing
* HTTP endpoint. It uses localhost as in the URL.
* For testing CORS, make sure to visit
* http://127.0.0.1/ which is a different origin
* from browser's perspective.
* /
const request_payload = {
some: 'data',
listed: ['h', 'er', 'e'],
}
try {
const resp = await fetch(
'http://localhost:3333/add_meeting',
{
method: 'POST',
mode: 'cors', // Required for customizing HTTP request headers
credentials: 'same-origin',
headers: {
'Content-Type': 'application/json; charset=UTF-8', // Required for ``cherrypy.tools.json_in`` to identify JSON payload and parse it automatically
},
body: JSON.stringify(request_payload),
},
)
const json_resp = await resp.json()
console.log(json_resp) // Will print: {"method": "POST", "payload": {"listed": ["h", "er", "e"], "some": "data"}}
} catch (e) {
console.warn('Exception: ' + e)
}
}
async function main() {
await addMeeting()
}
main() // Entry point
</script>
</html>""" # noqa: E501
#cherrypy.expose
#cherrypy.tools.json_in() # turn HTTP payload into an object; also checking the Content-Type header
#cherrypy.tools.json_out() # turn ``return``ed Python object into a JSON string; also setting corresponding Content-Type
def add_meeting(self):
"""Handle HTTP requests against ``/add_meeting`` URI."""
if cherrypy.request.method == 'OPTIONS':
# This is a request that browser sends in CORS prior to
# sending a real request.
# Set up extra headers for a pre-flight OPTIONS request.
cherrypy_cors.preflight(allowed_methods=['GET', 'POST'])
if cherrypy.request.method == 'POST':
return {'method': 'POST', 'payload': cherrypy.request.json}
return {'method': 'non-POST'}
def main():
"""Set up and run the web app.
Initializes CORS tools.
Sets up web server socket.
Enables the CORS tool.
"""
cherrypy_cors.install()
cherrypy.config.update({
'server.socket_host': '127.0.0.1',
'server.socket_port': 3333,
'cors.expose.on': True,
})
cherrypy.quickstart(WebRoot())
__name__ == '__main__' and main() # pylint: disable=expression-not-assigned
UPDATE
For me the Problem got fixed as soon as I was putting "encoding: URLEncoding(destination: .queryString)" in my request. Maybe this helps somebody else. link
I struggled the whole day to find the problem in my Alamofire PUT Request or the Flask Restful API. Request like GET, DELETE and POST are working fine with Alamofire, except the PUT Request.
When I'm using PUT Requests in combination with Postman and Flask-Restful everything is also working fine. But as soon as I'm trying to achieve the same Result with Alamofire, I'm not getting any parameters in Flask. I tried to illustrate this in the code examples.
So in short my example illustrates the following:
DELETE Request(Same with GET and POST)
Postman: success
Alamofire: success
PUT Request
Postman: success
Alamofire: failure (parameter dictionary empty in Flask-Restful)
Here is my Python Code [API Server]:
from flask import Flask, request, jsonify
from flask_restful import Resource, Api, reqparse
app = Flask(__name__)
api = Api(app)
class Stackoverflow(Resource):
def delete(self):
print(request.args)
if request.args.get('test-key') is None:
return jsonify({"message": "failure"})
else:
return jsonify({"message": "success"})
def put(self):
print(request.args)
if request.args.get('test-key') is None:
return jsonify({"message": "failure"})
else:
return jsonify({"message": "success"})
api.add_resource(Stackoverflow, '/stackoverflow')
if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0')
If I'm using Postman, I get this result (like expected):
Result in Postman
But now I'm trying to do the same with Alamofire in Swift. Same Server, nothing changed.
SWIFT demo Code [IOS APP]:
import UIKit
import Alamofire
import SwiftyJSON
class ViewController: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view
simplePUTRequest()
simpleDELETERequest()
}
func simplePUTRequest(){
AF.request("http://localhost:5000/stackoverflow", method: .put, parameters: ["test-key":"testvalue"])
.validate(statusCode: 200..<300)
.responseJSON { response in
if let data = response.data {
print("Result PUT Request:")
print(String(data: data, encoding: .utf8)!)
//print(utf8Text)
}else{
}
}
}
func simpleDELETERequest(){
AF.request("http://localhost:5000/stackoverflow", method: .delete, parameters: ["test-key":"testvalue"])
.validate(statusCode: 200..<300)
.responseJSON { response in
if let data = response.data {
print("Result DELETE Request:")
print(String(data: data, encoding: .utf8)!)
//print(utf8Text)
}else{
}
}
}
Xcode Console:
Result PUT Request:
{
"message": "failure"
}
Result DELETE Request:
{
"message": "success"
}
python Console (both Alamofire Requests):
ImmutableMultiDict([])
127.0.0.1 - - [15/Jun/2019 21:17:31] "PUT /stackoverflow HTTP/1.1" 200 -
ImmutableMultiDict([('test-key', 'testvalue')])
127.0.0.1 - - [15/Jun/2019 21:17:31] "DELETE /stackoverflow?test-key=testvalue HTTP/1.1" 200 -
As you can see, I'm getting the success message only while using the DELETE method.
Till now I tried using different encodings like URLEncoding.httpbody and URLEncoding.default, but nothing really helped.
For me it seems like it's a Alamofire/Swift Problem, because in Postman the same request method is working fine.
I would really appreciate your help, because I'm stuck and don't know anything further to do. I hope I didn't misunderstood something essential.
Thank you in advance!
I am currently using the same version AlamoFire, and when I use the PUT method, I use it as follows:
let request = AF.request(url, method: .put, parameters: ["uid": uid],
encoding: JSONEncoding.default, headers: headers)
request.responseJSON(completionHandler: { response in
guard response.error == nil else {
//Handle error
}
if let json = response.value as? [String: Any]
// Handle result.
}
The only difference to your post is that I used the encoding option. You can try to put the option and see what happens.
It looks like your server is expecting your PUT parameters to be URL form encoded into the URL. You may be hitting the version of the request method that uses JSON encoding by default, so adding encoder: URLEncodedFormParameterEncoder.default at the end of your request call should fix that. A future release will make that the default, as it's safe across all request types.
If that's not the issue, I suggest you investigate more closely to see what the differences between the requests may be. Since you control the server you should have easy access to the traffic.
We are using grpc as the RPC protocol for all our internal system. Most of the system is written in Java.
In Java, we can use InprocessServerBuilder for unittest. However, I haven't find a similar class in Python.
Can any one provide a sample code for how to do GRPC unittest in python?
How serendipitous that you have asked this question today; our unit test framework just entered code review. So for the time being the way to test is to use the full production stack to connect your client-side and server-side code (or to violate the API and mock a lot of internal stuff) but hopefully in days to weeks the much better solution will be available to you.
Some example code to get started:
proto
syntax = "proto3";
service MyLibrary {
rpc Search (Request) returns (Response);
}
message Request {
string id = 1;
}
message Response {
string status = 1;
}
python unit test
#!/usr/bin/env python
# coding=utf-8
import unittest
from grpc import StatusCode
from grpc_testing import server_from_dictionary, strict_real_time
import mylibrary_pb2
class TestCase(unittest.TestCase):
def __init__(self, methodName) -> None:
super().__init__(methodName)
myServicer = MyLibraryServicer()
servicers = {
mylibrary_pb2.DESCRIPTOR.services_by_name['MyLibrary']: myServicer
}
self.test_server = server_from_dictionary(
servicers, strict_real_time())
def test_search(self):
request = mylibrary_pb2.Request(
id=2,
)
method = self.test_server.invoke_unary_unary(
method_descriptor=(mylibrary_pb2.DESCRIPTOR
.services_by_name['Library']
.methods_by_name['Search']),
invocation_metadata={},
request=request, timeout=1)
response, metadata, code, details = method.termination()
self.assertTrue(bool(response.status))
self.assertEqual(code, StatusCode.OK)
if __name__ == '__main__':
unittest.main()
I find pytest-grpc is easy to follow and get it works in few minutes.
src: https://pypi.org/project/pytest-grpc/
Lets say I have a very simple web app which is presented as blue if the current president is a democrat and red if they are a republican. A REST API is used to get the current president, via the endpoint:
/presidents/current
which currently returns the json object:
{name: "Donald Trump", party: "Republican"}
So when my page loads I call the endpoint and I show red or blue depending on who is returned.
I wish to test this HTML/javascript page and I wish to mock the back-end so that I can control from within the test environment the API responses. For example:
def test_republican():
# configure the response for this test that the web app will receive when it connects to this endpoint
configure_endpoint(
"/presidents/current",
jsonify(
name="Donald Trump",
party="Republican"
)
)
# start the web app in the browser using selenium
load_web_app(driver, "http://localhost:8080")
e = driver.find_element_by_name("background")
assert(e.getCssValue("background-color") == "red")
def test_democrat():
# configure the response for this test that the web app will receive when it connects to this endpoint
configure_endpoint(
"/presidents/current",
jsonify(
name="Barack Obama",
party="Democrat"
)
)
# start the web app in the browser using selenium
load_web_app(driver, "http://localhost:8080")
e = driver.find_element_by_name("background")
assert(e.getCssValue("background-color") == "blue")
So the question is how should I implement the function configure_endpoint() and what libraries can you recommend me?
As #Kie mentioned, configure_endpoint implementation won't be enough, if you're going to stub the whole server-side within Selenium Python code. You would need a web server or whatever that will response via HTTP to requests from within testing environment.
It looks like the question is partially about testing of client-side code. What I see is that you're trying to make unit-test for client-side logic, but use integration testing suite in order to check this logic (it's strange).
The main idea is as follows.
You're trying to test client-side code. So, let's make mocks client-side too! Because this part of code is completely client-side related stuff.
If you actually want to have mocks, not stubs (watch the difference here: https://stackoverflow.com/a/3459491/882187) it is a better way to mock out HTTP requests inside your Javascript code. Just because you're testing a client-side piece of code, not some parts of server-side logic.
Having it isolated from whatever server-side is - is a great idea that you would love when your project become grow, while more and more endpoints will be appearing.
For example, you can use the following approach:
var restResponder = function() { // the original responder your client-side app will use
this.getCurrentPresident = function(successCallback) {
$.get('/presidents/current', callback);
}
};
var createMockResponder = function(president, party){ // factory that creates mocks
var myPresident = president;
var myParty = party;
return function() {
this.getCurrentPresident = function (successCallback) {
successCallback({"name": myPresident, "party": myParty});
}
};
}
// somewhere swap the original restResponder with new mockResponder created by 'createMockResponder'
// then use it in your app:
function drawColor(restResponder, backgroundEl) {
restResponder.getCurrentPresident(function(data){
if (data.party == "Democrat") $(backgroundEl).style('background-color', 'blue')
else if (data.party == "Republican") $(backgroundEl).style('background-color', 'red')
else console.info('Some strange response from server... Nevermind...');
});
}
Practically, this implementation depends on what do you have at the client-side as a framework. If jQuery, then my example is enough, but it looks very wordy. In case you have something more advanced, like AngularJS, you can do the same in 2-3 lines of code:
// Set up the mock http service responses
$httpBackend = $injector.get('$httpBackend');
// backend definition common for all tests
authRequestHandler = $httpBackend.when('GET', '/auth.py')
.respond({userId: 'userX'}, {'A-Token': 'xxx'});
Check out the docs: https://docs.angularjs.org/api/ngMock/service/$httpBackend
If you're still stick to the idea, that you need mocks inside Selenium tests, please
try this project: https://turq.readthedocs.io/en/latest/
It serves with Python DSL for describing REST responders.
Using turq your mocks will look as follows:
path('/presidents/current').json({'name':'Barack Obama', 'party': 'Democrat'}, jsonp=False)
Also, I would recommend to try stubs instead of mocks and use this Python module: mock-server https://pypi.python.org/pypi/mock-server/0.3.7
You are required to create the directory layout containing corresponding pre-populated JSON responses and to add some boilerplate code in order to make the mock-server respond on 'localhost:8080'. The directory layout for your example will look like this:
stub_obama/
presidents/
current/
GET_200.json # will contain {"name": "Barack Obama", "party": "Democrat"}
stub_trump/
presidents/
current/
GET_200.json # will contain {"name": "Donald Trump", "party": "Republican"}
But the mock_server is based on Tornado, it is very heavy solution for using in tests I think.
I hope, my answer is helpful and informative. Welcome to discuss it! I made tons of projects with Selenium, big and small tests, tested client-side and server-side.
I would use tornado web framework.
import json
import functools
import operator
from tornado import ioloop, web, gen
from tornado.options import define, options
define("data_file", default='default/mock.json', type=str)
class Handler(web.RequestHandler):
def data_received(self, chunk):
pass
def initialize(self, data):
self.data = data
#gen.coroutine
def get(self, *args, **kwargs):
path = self.request.path.split("/")[1:]
path = functools.reduce(
operator.add,
[[k, v[0].decode("utf-8")] for k, v in self.request.query_arguments.items()],
path
)
try:
self.write(functools.reduce(operator.getitem, path, self.data))
except KeyError:
self.set_status(404)
class Application(web.Application):
def __init__(self):
data = {}
with open(options.data_file) as data_file:
data = json.load(data_file)
handlers = [
('(.*)', Handler, {"data": data})
]
settings = dict(
gzip=True,
static_hash_cache=True,
)
web.Application.__init__(self, handlers, **settings)
def main():
io_loop = ioloop.IOLoop.instance()
backend_application = Application()
backend_application.listen(8001)
io_loop.start()
if __name__ == "__main__":
main()
This is a code I used for mocking a REST-API which is a standalone script, but it can be embedded into your test environment as well.
I defined a JSON file which defines the different path components and what should be returned. Like this:
{
"presidents": {
"current": {
"name": "Donald Trump",
"party": "Republican"
}
}
}
I saved this to a mock.json and called the script with a parameter mock_rest.py --data-file="./mock.json".
I hope that gives you a starting point and a good example.
If your load_web_app function uses the requests library to access the REST API, using requests-mock is a convenient way to fake that library's functionality for test purposes.
For those who stumble upon this question, and do not want to end up writing the code to create their own mock server implementations of the API, you can use Mocktastic, which is a downloadable desktop application for Windows, MacOS and Linux, which provides an easy to use GUI to setup your mock API servers.
I have written my own custom dispatcher that uses regular expressions to map routes, however, I can no longer host static files in /static. Here is the dispatcher and the config:
class Dispatcher(object):
def __init__(self):
self.urls = {}
def __call__(self, path_info):
print('Dispatcher called: ' + path_info)
func = self.find_handler(path_info)
cherrypy.serving.request.handler = func
def find_handler(self, path_info):
request = cherrypy.serving.request
request.config = cherrypy.config.copy()
for url in self.urls:
args = re.findall(url, path_info)
if len(args) > 0:
# in the case that the route is just a URL, we don't want
# an extra argument in the method function
try:
args.remove(path_info)
except ValueError:
pass
controller = self.urls[url]
method = request.method.lower()
return cherrypy._cpdispatch.LateParamPageHandler(getattr(controller, method), *args)
return cherrypy.NotFound()
def connect(self, url, controller):
if not url.endswith("$"):
url += "$"
self.urls[url] = controller
And the config:
config = {
'global': {
'server.socket_host': '0.0.0.0',
'server.socket_port': port,
},
'/static': {
'tools.staticdir.on': True,
'tools.staticdir.dir': os.path.join(os.getcwd(), 'static'),
},
'/': {
'request.dispatch': self.dispatcher,
}
}
If I use the standard dispatcher, static files work as they should, however if I use my own, they no longer work. Having done debugging in the dispatcher, static files go through the dispatcher, even though I have specific that only in '/' does the dispatcher get used.
I'm not familiar with cherrypy, but it seems obvious: everything in /static is also in /, so it is anyone's guess which config entry it will use. I would hope that "more specific has priority", but according to your description, this is not the case. Looking at the documentation also doesn't help, there's no mention of ambiguous path handling.
You would think that changing the order might help, but since this is a dictionary, the order isn't preserved.
It seems that cherrypy is unable to do this. If it has a default dispatcher which is overloaded with others, that could solve the problem. Another option is that your custom dispatcher can call the static one if it detects the path.
Finally, the documentation talks about "mounting an application to a path". If you do this, you may want to change the order. If you don't do this, it might be done automatically, and doing it manually may solve your problem.
Not all of this may make sense, since as I wrote I'm not familiar with cherrypy, but I hope it helps you a bit anyway.