Develop Reference

Pytest - Mock and Testing Django url with call to external package

HI I need to test my endpoint, it is expecting a json from another call to external library
url
path("check-services", views.check_services, name="check_services"),
views
from api.services import Services
services = Services()
def check_services(request):
payload = json.loads(request.GET.get("the_params"))
error, message, response = services.check_services(payload)
if error:
return JsonResponse({"res": [], 'error': True, 'message': message})
return JsonResponse({"res": response, 'error': False, "message": "Success"})
tests
import pytest
from api.services import Services
#patch('api.services.Services', side_effect=[True, 'Success', []])
def test_check_services(client):
payload = {}
url = reverse('check_services')
res = client.get(url)
//really dont know how to mock the external service here
error,message, response = Services().check_services(payload)
print(response)
//assertion
assert error ==True
Got error
AssertionError: assert <MagicMock name='Services.get().status_code' id='140222676793616'> == 200
E + where <MagicMock name='Services.get().status_code' id='140222676793616'> = <MagicMock name='Services.get()' id='140222676785840'>.status_code
But it is still actually calling the external package when I commented the line that says `res=client.get(url).
Inspecting the package Service class
class Services:
def __init__(self):
//
def check_services(payload: dict):
try
response = //some manipulation here ex calling another api
return False, 'success', response
except Exception as e:
return True,e.message,[]
Update ,
Ok so I updated my test using unittest.mock
class SimpleTestCase(TestCase):
def setUp(self):
self.client = Client()
#patch('Courier.views.check_services')
def test_mocked_test_case_should_succeed(self, mocked_check_services):
"""
This is a simple testcase using mock feature.
1 - Decorate the method with path.
2 - Reference the mock in test method as param (mocked_check_services).
3 - Define a value of the return of method (sub method or property) to mock.
4 - Call the mocked method and compare.
"""
mocked_check_services.get.content.return_value = b'{"res": [], "error": true, "messages": "Invalid service"}'
test_address = '{"name": "37 Jamaica Drive"}'
url = f'{reverse("check_services")}?the_params={test_address}'
response = self.client.get(url)
self.assertEqual(response.status_code, 200)
self.assertEqual(
response.content, b'{"res": [], "error": true, "message": "Invalid service"}')
Bu still it keeps calling the Services class directly.
How to test in this scenario without actually running the imported external method from a library?

FastApi : traceback.exc_format return none when using add_exception_handler

I develop with FastApi, and want to contain traceback info in response when error occur;
To do so, I define exception handlers in exception_handler.py :
from fastapi.responses import JSONResponse
from fastapi import status
from fastapi import FastAPI, Request
from traceback import format_exc, print_exc
def general_exception_handler(req: Request, exc: Exception):
'''
Exception handler for unspecified exceptions
'''
tracback_msg = format_exc()
return JSONResponse(
{
"code": status.HTTP_500_INTERNAL_SERVER_ERROR,
"message": f"error info: {tracback_msg}",
# "message": f"error info: {str(exc)}",
"data": "",
},
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
)
And attach those handler to fastappi app instance in server.py:
server.py is where I create app instance and attach extra function to it like middlewares or exception handlers.
from core import router # api routers are defined in router.py
from fastapi import FastAPI
from core.exception_handler import general_exception_handler
app = FastAPI(
debug=False,
docs_url=None,
redoc_url=None
)
# attach exception handler to app instance
app.add_exception_handler(Exception, general_exception_handler)
# include routers to app intance
app.include_router(router.router)
The problem is, when exception was raised, traceback message return by format_exc() is None;But when I used str(exc) like the annotated code, I got the exception info properly but of course without traceback info.

It will not work because the exception handler receives the exception as parameter instead of catching the exception itself, meaning that there is no stacktrace in this case.
If you want to have the stacktrace, you should create a middleware or a custom API router that will actually capture the exception and return the message the way you want. I usually prefer to use a custom API Route instead of using middleware because it is more explicit and gives you more flexibility.
You can write something like this
class MyRoute(APIRoute):
def get_route_handler(self) -> Callable:
original_route_handler = super().get_route_handler()
async def custom_route_handler(request: Request) -> Response:
try:
return await original_route_handler(request)
except Exception as exc:
tracback_msg = format_exc()
# your return goes here
return custom_route_handler
Then you override the default route handler from fastapi
app = FastAPI()
app.router.route_class = MyRoute
It should give you want you want

There's always format_exception, which takes an explicit exception argument, rather than grabbing the current one from sys.exc_info().

How to call a normal python function along with header information without using requests

Details of application:
UI: Angular
Backend: Python Flask (using Swagger)
Database: MongoDB
We have a few backend python methods which will be called from the UI side to do CURD operations on the database.
Each of the methods has a decorator which will check the header information to ensure that only a genuine person can call the methods.
From the UI side when these API's are called, this authorization decorator is not creating any problem and a proper response is returned to the UI (as we are passing the header information also to the request)
But now we are writing unit test cases for the API's. Here each test case will call the backend method and because of the authorization decorator, I am getting errors and not able to proceed. How can I handle this issue?
backend_api.py
--------------
from commonlib.auth import require_auth
#require_auth
def get_records(record_id):
try:
record_details = records_coll.find_one({"_id": ObjectId(str(record_id))})
if record_details is not None:
resp = jsonify({"msg": "Found Record", "data": str(record_details)})
resp.status_code = 200
return resp
else:
resp = jsonify({"msg": "Record not found"})
resp.status_code = 404
return resp
except Exception as ex:
resp = jsonify({"msg": "Exception Occured",'Exception Details': ex}))
resp.status_code = 500
return resp
commonlib/auth.py
-----------------
### some lines of code here
def require_auth(func):
"""
Decorator that can be added to a function to check for authorization
"""
def wrapper(*args, **kwargs):
print(*args,**kwargs)
username = get_username()
security_log = {
'loginId': username,
'securityProtocol': _get_auth_type(),
}
try:
if username is None:
raise SecurityException('Authorization header or cookie not found')
if not is_auth_valid():
raise SecurityException('Authorization header or cookie is invalid')
except SecurityException as ex:
log_security(result='DENIED', message=str(ex))
unauthorized(str(ex))
return func(*args, **kwargs)
return wrapper
test_backend_api.py
-------------------
class TestBackendApi(unittest.TestCase):
### some lines of code here
#mock.patch("pymongo.collection.Collection.find_one", side_effect=[projects_json])
def test_get_records(self, mock_call):
from backend_api import get_records
ret_resp = get_records('61729c18afe7a83268c6c9b8')
final_response = ret_resp.get_json()
message1 = "return response status code is not 200"
self.assertEqual(ret_resp.status_code, 200, message1)
Error snippet :
---------------
E RuntimeError: Working outside of request context.
E
E This typically means that you attempted to use functionality that needed
E an active HTTP request. Consult the documentation on testing for
E information about how to avoid this problem.

How to set up CORS in CherryPy

Overview
When creating a post request from my website to my Python server running CherryPy, I receive the error Access to XMLHttpRequest has been blocked by CORS policy: Request header field content-type is not allowed by Access-Control-Allow-Headers in preflight response. . I was able to get away with the problem temporarily with one of the "CORS Everywhere" browser extensions, but
Due to recent updates, the extensions have not yet been updated to be working again.
The website involved needs to eventually be used by many in my local complex without the browser extension, so once the extensions get updated, it does not really matter one way or another, as I cannot rely on these extensions, and force everyone to use them (when there is obviously a fix that would make an extension not necessary).
I figure that perhaps the solutions are outdated, but am not sure.
Here is the relevant code:
On the server side (CherryPy/Python):
The CherryPy Python function being called, from the website post request
#cherrypy.expose
#cherrypy.tools.json_in()
def add_meeting(self):
data = None
id = None
start_time = None
end_time = None
title = None
userlist = None
result = {"operation": "request", "result": "success"}
if cherrypy.request.method == "POST":
data = cherrypy.request.json
id = data["id"]
start_time = data["start_time"]
end_time = data["end_time"]
title = data["title"]
userlist = data["userlist"]
# Rest of relevant code in function is left out, to take up less
# space and not post irrelevant code. That being said, I am
# positive the logic is correct, as it originally ran smoothly
# with a "Cors Everywhere" Browser Extension.
return result
Here is the area where I set up and run CherryPy
def main():
# Create the configuration file parser object and start the CherryPy server
config = ConfigParser.ConfigParser()
config.read(CONFIG_FILE)
port = config.getint('Meta', 'port')
host = config.get('Meta', 'host')
cherrypy.config.update({'server.socket_port': port,
'server.socket_host': host,
'tools.CORS.on': True})
cherrypy.quickstart(Coordinator(config))
main()
Here is the config file mentioned in the code above (CONFIG_FILE)
[Meta]
host = 0.0.0.0
port = 3000
# Rest is left out, as it is irrelevant with problem
The solutions I have tried implementing
The inclusion of the following function above the main function:
def CORS():
cherrypy.response.headers["Access-Control-Allow-Origin"] = "*"
with cherrypy.tools.CORS = cherrypy.Tool('before_handler', CORS)
2. Adding " 'cors.expose.on': True " to cherrypy.config.update above
3. Using this cherrypy-cors Python library I found online: https://pypi.org/project/cherrypy-cors/
4. The inclusion of headers in the config.update portion of the Python file
5. Adding "#cherrypy.tools.accept(media='application/json')" before "def add_meeting"
Conclusion
I've tried the solutions above together, separately, some with and without the others, and I am still stuck. Maybe some of these solutions are partially correct, and there is something extra needed with my code. I am not sure; I just cannot get it working. I do not have much experience with web development before this, so maybe (and hopefully) the solution is extremely simple. I know the code works, I just cannot get it running without a working "Cors Everywhere" browser extension for every user.
As for the versions I am running: I am using CherryPy 14.2.0 and Python 2.7.6
Any help would mean the absolute world to me, thank you.

So first, you need to set pre-flight headers when processing OPTIONS request, you can list allowed methods there.
Then, you also need to enable the cors.expose tool.
There's some usage hints in the docstring of cherrypy-cors. For example, when using a MethodDispatcher, you could just decorate an OPTIONS handler method with #cherrypy_cors.tools.preflight() instead of doing this in every HTTP handler.
Here's a simple traversal example (without a method dispatcher). To test it, visit http://127.0.0.1/ and it will make requests against http://localhost:3333/add_meeting which is a different Origin in terms of CORS ('localhost' != '127.0.0.1').
"""Example of CORS setup using cherrypy-cors library."""
import cherrypy
import cherrypy_cors
# Python 2 compat: make all classes new-style by default
__metaclass__ = type # pylint: disable=invalid-name
class WebRoot:
"""Root node for HTTP handlers."""
#cherrypy.expose
def index(self): # pylint: disable=no-self-use
"""Render a web page handling request against ``/``.
Contains client JS snippet which will query the API endpoint.
It will be executed by the browser while loading the page.
"""
return """<html>
<script type="text/javascript">
async function addMeeting() {
/*
* Example coroutine for querying /add_meeing
* HTTP endpoint. It uses localhost as in the URL.
* For testing CORS, make sure to visit
* http://127.0.0.1/ which is a different origin
* from browser's perspective.
* /
const request_payload = {
some: 'data',
listed: ['h', 'er', 'e'],
}
try {
const resp = await fetch(
'http://localhost:3333/add_meeting',
{
method: 'POST',
mode: 'cors', // Required for customizing HTTP request headers
credentials: 'same-origin',
headers: {
'Content-Type': 'application/json; charset=UTF-8', // Required for ``cherrypy.tools.json_in`` to identify JSON payload and parse it automatically
},
body: JSON.stringify(request_payload),
},
)
const json_resp = await resp.json()
console.log(json_resp) // Will print: {"method": "POST", "payload": {"listed": ["h", "er", "e"], "some": "data"}}
} catch (e) {
console.warn('Exception: ' + e)
}
}
async function main() {
await addMeeting()
}
main() // Entry point
</script>
</html>""" # noqa: E501
#cherrypy.expose
#cherrypy.tools.json_in() # turn HTTP payload into an object; also checking the Content-Type header
#cherrypy.tools.json_out() # turn ``return``ed Python object into a JSON string; also setting corresponding Content-Type
def add_meeting(self):
"""Handle HTTP requests against ``/add_meeting`` URI."""
if cherrypy.request.method == 'OPTIONS':
# This is a request that browser sends in CORS prior to
# sending a real request.
# Set up extra headers for a pre-flight OPTIONS request.
cherrypy_cors.preflight(allowed_methods=['GET', 'POST'])
if cherrypy.request.method == 'POST':
return {'method': 'POST', 'payload': cherrypy.request.json}
return {'method': 'non-POST'}
def main():
"""Set up and run the web app.
Initializes CORS tools.
Sets up web server socket.
Enables the CORS tool.
"""
cherrypy_cors.install()
cherrypy.config.update({
'server.socket_host': '127.0.0.1',
'server.socket_port': 3333,
'cors.expose.on': True,
})
cherrypy.quickstart(WebRoot())
__name__ == '__main__' and main() # pylint: disable=expression-not-assigned

GAE: How can a handler return a webapp2.Response when using sessions and overriding `dispatch`?

I adapted this sample code in order to get webapp2 sessions to work on Google App Engine.
What do I need to do to be able to return webapp2.Response objects from a handler that's inheriting from a BaseHandler that overrides the dispatch method?
Here's a demonstration of the kind of handler I want to write:
import webapp2
import logging
from webapp2_extras import sessions
class BaseHandler(webapp2.RequestHandler):
def dispatch(self):
# Get a session store for this request.
self.session_store = sessions.get_store(request=self.request)
try:
# Dispatch the request.
webapp2.RequestHandler.dispatch(self)
finally:
# Save all sessions.
self.session_store.save_sessions(self.response)
class HomeHandler(BaseHandler):
def get(self):
logging.debug('In homehandler')
response = webapp2.Response()
response.write('Foo')
return response
config = {}
config['webapp2_extras.sessions'] = {
'secret_key': 'some-secret-key',
}
app = webapp2.WSGIApplication([
('/test', HomeHandler),
], debug=True, config=config)
This code is obviously not working, since BaseHandler always calls dispatch with self. I've looked through the code of webapp2.RequestHandler, but it seriously eludes me how to modify my BaseHandler (or perhaps set a custom dispatcher) such that I can simply return response objects from inheriting handlers.
Curiously, the shortcut of assigning self.response = copy.deepcopy(response) does not work either.

You're mixing the two responses in one method. Use either
return webapp2.Response('Foo')
or
self.response.write('Foo')
...not both.

I took a look at webapp2.RequestHandler and noticed that returned values are just passed up the stack.
A solution which works for me is to use the returned Response when one is returned from the handler, or self.response when nothing is returned.
class BaseHandler(webapp2.RequestHandler):
def dispatch(self):
# Get a session store for this request.
self.session_store = sessions.get_store(request=self.request)
response = None
try:
# Dispatch the request.
response = webapp2.RequestHandler.dispatch(self)
return response
finally:
# Save all sessions.
if response is None:
response = self.response
self.session_store.save_sessions(response)
While I was playing I noticed that my session stored as a secure cookie was not getting updated when exceptions were raised in the handler.