Develop Reference

Fix potential XSS on Python using Bleach module

I am working on a cloud based API implemented using Lambda functions on AWS. We are scanning it for securities issues and Veracode SAST tool has identified a potential XSS on one of the endpoints.
The code looks as follow:
def lambda_handler(event: dict, _):
event = Event(event)
body = event.body
id_xss = body[“id”]
…
return {
status_code: 200,
body:{
"message": f"Successfully endpoint execution ${id_xss}.”
}
}
I am trying to fix it by using the Python module Bleach. So my new code would look as follows:
def lambda_handler(event: dict, _):
event = Event(event)
body = event.body
# SANITIZE THE INPUT
id_xss = bleach.clean(body[“id”], BLEACH_VALID_TAGS,
BLEACH_VALID_ATTRS, BLEACH_VALID_STYLES)
…
return {
status_code: 200,
body:{
"message": f"Successfully endpoint execution ${id_xss}.”
}
}
I've also tried checking that body[“id”] is a natural number.
However, Veracode SAST tool still consider it an issue. Am I using it properly? What would be the right approach?
Any comment or feedback will be welcomed.

How to set up CORS in CherryPy

Overview
When creating a post request from my website to my Python server running CherryPy, I receive the error Access to XMLHttpRequest has been blocked by CORS policy: Request header field content-type is not allowed by Access-Control-Allow-Headers in preflight response. . I was able to get away with the problem temporarily with one of the "CORS Everywhere" browser extensions, but
Due to recent updates, the extensions have not yet been updated to be working again.
The website involved needs to eventually be used by many in my local complex without the browser extension, so once the extensions get updated, it does not really matter one way or another, as I cannot rely on these extensions, and force everyone to use them (when there is obviously a fix that would make an extension not necessary).
I figure that perhaps the solutions are outdated, but am not sure.
Here is the relevant code:
On the server side (CherryPy/Python):
The CherryPy Python function being called, from the website post request
#cherrypy.expose
#cherrypy.tools.json_in()
def add_meeting(self):
data = None
id = None
start_time = None
end_time = None
title = None
userlist = None
result = {"operation": "request", "result": "success"}
if cherrypy.request.method == "POST":
data = cherrypy.request.json
id = data["id"]
start_time = data["start_time"]
end_time = data["end_time"]
title = data["title"]
userlist = data["userlist"]
# Rest of relevant code in function is left out, to take up less
# space and not post irrelevant code. That being said, I am
# positive the logic is correct, as it originally ran smoothly
# with a "Cors Everywhere" Browser Extension.
return result
Here is the area where I set up and run CherryPy
def main():
# Create the configuration file parser object and start the CherryPy server
config = ConfigParser.ConfigParser()
config.read(CONFIG_FILE)
port = config.getint('Meta', 'port')
host = config.get('Meta', 'host')
cherrypy.config.update({'server.socket_port': port,
'server.socket_host': host,
'tools.CORS.on': True})
cherrypy.quickstart(Coordinator(config))
main()
Here is the config file mentioned in the code above (CONFIG_FILE)
[Meta]
host = 0.0.0.0
port = 3000
# Rest is left out, as it is irrelevant with problem
The solutions I have tried implementing
The inclusion of the following function above the main function:
def CORS():
cherrypy.response.headers["Access-Control-Allow-Origin"] = "*"
with cherrypy.tools.CORS = cherrypy.Tool('before_handler', CORS)
2. Adding " 'cors.expose.on': True " to cherrypy.config.update above
3. Using this cherrypy-cors Python library I found online: https://pypi.org/project/cherrypy-cors/
4. The inclusion of headers in the config.update portion of the Python file
5. Adding "#cherrypy.tools.accept(media='application/json')" before "def add_meeting"
Conclusion
I've tried the solutions above together, separately, some with and without the others, and I am still stuck. Maybe some of these solutions are partially correct, and there is something extra needed with my code. I am not sure; I just cannot get it working. I do not have much experience with web development before this, so maybe (and hopefully) the solution is extremely simple. I know the code works, I just cannot get it running without a working "Cors Everywhere" browser extension for every user.
As for the versions I am running: I am using CherryPy 14.2.0 and Python 2.7.6
Any help would mean the absolute world to me, thank you.

So first, you need to set pre-flight headers when processing OPTIONS request, you can list allowed methods there.
Then, you also need to enable the cors.expose tool.
There's some usage hints in the docstring of cherrypy-cors. For example, when using a MethodDispatcher, you could just decorate an OPTIONS handler method with #cherrypy_cors.tools.preflight() instead of doing this in every HTTP handler.
Here's a simple traversal example (without a method dispatcher). To test it, visit http://127.0.0.1/ and it will make requests against http://localhost:3333/add_meeting which is a different Origin in terms of CORS ('localhost' != '127.0.0.1').
"""Example of CORS setup using cherrypy-cors library."""
import cherrypy
import cherrypy_cors
# Python 2 compat: make all classes new-style by default
__metaclass__ = type # pylint: disable=invalid-name
class WebRoot:
"""Root node for HTTP handlers."""
#cherrypy.expose
def index(self): # pylint: disable=no-self-use
"""Render a web page handling request against ``/``.
Contains client JS snippet which will query the API endpoint.
It will be executed by the browser while loading the page.
"""
return """<html>
<script type="text/javascript">
async function addMeeting() {
/*
* Example coroutine for querying /add_meeing
* HTTP endpoint. It uses localhost as in the URL.
* For testing CORS, make sure to visit
* http://127.0.0.1/ which is a different origin
* from browser's perspective.
* /
const request_payload = {
some: 'data',
listed: ['h', 'er', 'e'],
}
try {
const resp = await fetch(
'http://localhost:3333/add_meeting',
{
method: 'POST',
mode: 'cors', // Required for customizing HTTP request headers
credentials: 'same-origin',
headers: {
'Content-Type': 'application/json; charset=UTF-8', // Required for ``cherrypy.tools.json_in`` to identify JSON payload and parse it automatically
},
body: JSON.stringify(request_payload),
},
)
const json_resp = await resp.json()
console.log(json_resp) // Will print: {"method": "POST", "payload": {"listed": ["h", "er", "e"], "some": "data"}}
} catch (e) {
console.warn('Exception: ' + e)
}
}
async function main() {
await addMeeting()
}
main() // Entry point
</script>
</html>""" # noqa: E501
#cherrypy.expose
#cherrypy.tools.json_in() # turn HTTP payload into an object; also checking the Content-Type header
#cherrypy.tools.json_out() # turn ``return``ed Python object into a JSON string; also setting corresponding Content-Type
def add_meeting(self):
"""Handle HTTP requests against ``/add_meeting`` URI."""
if cherrypy.request.method == 'OPTIONS':
# This is a request that browser sends in CORS prior to
# sending a real request.
# Set up extra headers for a pre-flight OPTIONS request.
cherrypy_cors.preflight(allowed_methods=['GET', 'POST'])
if cherrypy.request.method == 'POST':
return {'method': 'POST', 'payload': cherrypy.request.json}
return {'method': 'non-POST'}
def main():
"""Set up and run the web app.
Initializes CORS tools.
Sets up web server socket.
Enables the CORS tool.
"""
cherrypy_cors.install()
cherrypy.config.update({
'server.socket_host': '127.0.0.1',
'server.socket_port': 3333,
'cors.expose.on': True,
})
cherrypy.quickstart(WebRoot())
__name__ == '__main__' and main() # pylint: disable=expression-not-assigned

get trace ID of sent request

I'm using the Open Tracing Python library for GRPC and am trying to build off of the example script here: https://github.com/opentracing-contrib/python-grpc/blob/master/examples/trivial/trivial_client.py.
Once I have sent a request through the intercepted channel, how do I find the trace-id value for the request? I want to use this to look at the traced data in the Jaeger UI.

I had missed a key piece of documentation. In order to get a trace ID, you have to create a span on the client side. This span will have the trace ID that can be used to examine data in the Jaeger UI. The span has to be added into the GRPC messages via an ActiveSpanSource instance.
# opentracing-related imports
from grpc_opentracing import open_tracing_client_interceptor, ActiveSpanSource
from grpc_opentracing.grpcext import intercept_channel
from jaeger_client import Config
# dummy class to hold span data for passing into GRPC channel
class FixedActiveSpanSource(ActiveSpanSource):
def __init__(self):
self.active_span = None
def get_active_span(self):
return self.active_span
config = Config(
config={
'sampler': {
'type': 'const',
'param': 1,
},
'logging': True,
},
service_name='foo')
tracer = config.initialize_tracer()
# ...
# In the method where GRPC requests are sent
# ...
active_span_source = FixedActiveSpanSource()
tracer_interceptor = open_tracing_client_interceptor(
tracer,
log_payloads=True,
active_span_source=active_span_source)
with tracer.start_span('span-foo') as span:
print(f"Created span: trace_id:{span.trace_id:x}, span_id:{span.span_id:x}, parent_id:{span.parent_id}, flags:{span.flags:x}")
# provide the span to the GRPC interceptor here
active_span_source.active_span = span
with grpc.insecure_channel(...) as channel:
channel = intercept_channel(channel, tracer_interceptor)
Of course, you could switch the ordering of the with statements so that the span is created after the GRPC channel. That part doesn't make any difference.

Correct me, if I'm wrong. If you mean how to find the trace-id on the server side, you can try to access the OpenTracing span by get_active_span. The trace-id, I suppose, should be one of the tags in it.

How to write a GRPC python unittest

We are using grpc as the RPC protocol for all our internal system. Most of the system is written in Java.
In Java, we can use InprocessServerBuilder for unittest. However, I haven't find a similar class in Python.
Can any one provide a sample code for how to do GRPC unittest in python?

How serendipitous that you have asked this question today; our unit test framework just entered code review. So for the time being the way to test is to use the full production stack to connect your client-side and server-side code (or to violate the API and mock a lot of internal stuff) but hopefully in days to weeks the much better solution will be available to you.

Some example code to get started:
proto
syntax = "proto3";
service MyLibrary {
rpc Search (Request) returns (Response);
}
message Request {
string id = 1;
}
message Response {
string status = 1;
}
python unit test
#!/usr/bin/env python
# coding=utf-8
import unittest
from grpc import StatusCode
from grpc_testing import server_from_dictionary, strict_real_time
import mylibrary_pb2
class TestCase(unittest.TestCase):
def __init__(self, methodName) -> None:
super().__init__(methodName)
myServicer = MyLibraryServicer()
servicers = {
mylibrary_pb2.DESCRIPTOR.services_by_name['MyLibrary']: myServicer
}
self.test_server = server_from_dictionary(
servicers, strict_real_time())
def test_search(self):
request = mylibrary_pb2.Request(
id=2,
)
method = self.test_server.invoke_unary_unary(
method_descriptor=(mylibrary_pb2.DESCRIPTOR
.services_by_name['Library']
.methods_by_name['Search']),
invocation_metadata={},
request=request, timeout=1)
response, metadata, code, details = method.termination()
self.assertTrue(bool(response.status))
self.assertEqual(code, StatusCode.OK)
if __name__ == '__main__':
unittest.main()

I find pytest-grpc is easy to follow and get it works in few minutes.
src: https://pypi.org/project/pytest-grpc/

Creating a basic JSONRPC API in CherryPy - Getting 404s

The title is fairly self-explanatory, so I'll just show some code I've tried so far:
From https://stackoverflow.com/a/713950,
import cherrypy
from cherrypy import expose
cherrypy.config.update({'server.socket_port': 80})
class Test:
#expose
def test_call(self):
return "Testing"
cherrypy.quickstart(Test())
Also, from another SO post, two variants on the following:
cherrypy.config.update({
'server.socket_port': 80,
'/': {
'request.dispatch': cherrypy.dispatch.MethodDispatcher(),
'tools.trailing_slash.on': False
}
})
class Test:
def test_call(self, *args):
return json.dumps(args)
test_call.exposed = True
class API:
def __init__(self):
self.Test = Test()
class Root:
def __init__(self):
self.API = API()
cherrypy.tree.mount(Root())
cherrypy.quickstart(Root())
with a variation suggested here: Path Not Found in CherryPy
cherrypy.quickstart(cherrypy.Application(Root(), '/', {}))
I run these and access http://mysite.com/test_call or, in the other case, mysite.com/api/test/test_call, and neither of these seem to be doing much of anything except returning a 404. Ideas?
I am completely open to trying a different framework if it'll just let me expose a few function calls to dump JSON. I don't need anything fancy or cool, just functioning.
EDIT: Apparently my problem was that the server was by default expecting to be localhost, which basically makes me an idiot. Adding cherrypy.server.socket_host = "mydomain.com" fixes this.

The title doesn't match with examples and is telling that you were likely misguided by REST adepts, in the answer's comments you linked, who tend call everything a "RPC", which deviates from their CRUD-limited resource perspective. JSON RPC is the certain specification, which defines JSON structures for request and response. It looks like this.
--> {"jsonrpc": "2.0", "method": "subtract", "params": [42, 23], "id": 1}
<-- {"jsonrpc": "2.0", "result": 19, "id": 1}
Your examples have nothing to do with it, as well as with REST. They are a copy-paste without understanding the subject. Let's sort out a little.
CherryPy has serveral dispatching options. Default dispatcher maps request URL segments to Python object tree, like /API/Test to root.API.Test in your second example. Its common use is regular GET/POST web flow.
If you want to implement a RESTful API, here's dedicated manual page for it: Creating RESTful applications in CherryPy. It briefly explains the style and usage of MethodDispatcher.
If actual JSON RPC is what you want, then you can look at python-jsonrpc package, which has CherryPy adapter.
Though, if all you want to achieve is to return a JSON string with appropriate content-type header, CherryPy has a specific tool for it, so there's no point to do it manually.
Here follows example for #4.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import cherrypy
config = {
'global' : {
'server.socket_host' : '127.0.0.1',
'server.socket_port' : 8080,
'server.thread_pool' : 8
}
}
class Api:
#cherrypy.expose
#cherrypy.tools.json_out()
def oneway(self):
'''Just open http://127.0.0.1:8080/api/oneway'''
return {
'foo' : 'bar',
'baz' : 'another one'
}
#cherrypy.expose
#cherrypy.tools.json_in()
#cherrypy.tools.json_out()
def twoway(self):
'''You can call it like:
curl -X POST -H "Content-Type: application/json" \
-d '{"foo":123,"bar":"baz"}' http://127.0.0.1:8080/api/twoway
'''
data = cherrypy.request.json
return data.items()
if __name__ == '__main__':
cherrypy.quickstart(Api(), '/api', config)

I tried the first script as below (with 2 notes, using root user if using port 80). Access it by "http:// 127.0.0.1 /test_call". It works.
You should be more specific to raise your question (giving your codes), so the audience knows how to help solve it.
#! /usr/bin/python3
import cherrypy
from cherrypy import expose
## NOTE 1 - using 0.0.0.0
cherrypy.config.update({'server.socket_host' : '0.0.0.0', 'server.socket_port': 80})
class Test:
#expose
def test_call(self):
return "Testing"
## NOTE 2 - a work around for python3 and cherrypy v3.x
## work around OSERR issue - OSError: Port 7766 not bound on '10.220.203.233'
## refer to http://stackoverflow.com/questions/767575/crp-hello-world-error
def fake_wait_for_occupied_port(host, port): return
cherrypy.process.servers.wait_for_occupied_port = fake_wait_for_occupied_port
cherrypy.quickstart(Test())