Develop Reference

Django OAuth with Xero SDK can't access session for oauth2_token_getter and oauth2_token_saver as no request object can be passed

I am using Django with requests_oauthlib to access the Xero API using the xero-python SDK.
Xero Starter Demo
requests-oauthlib
I am trying to use request.session to store the oauth token information. This works well on the initial call and callback, but in subsequent views when I want to load the correct token from the request.session I am hitting a problem.
The Xero ApiClient (xero_python.api_client) wants me to configure two functions it can call to get and save the token in the future (oauth2_token_getter, oauth2_token_saver) (decorators are used in the demo). Those functions are called by the Xero API and take no parameters. I can create the functions no problem, and they get called correctly, but once inside the functions I have no request object so I can't access request.session to get or save the token to the session.
The Xero demonstration projects use Flask which imports a root level session object (from flask import session) which can be accessed in any function, so this works fine for Flask. What is the correct way to solve this problem in Django?
I have tried creating a SessionStore object but it was empty despite me knowing the data is in the request.session
I have also tried importing requests.session but this doesn't seem to be what I am looking for as I can't get it to read/write any data.
Some Code:
Saving the token in the callback works as I have a request.session:
def xero_callback(request):
...
request.session["oauth_token"] = token
...
The Xero ApiClient is configured with the methods for getting and saving:
api_client = ApiClient(
Configuration(
debug=True,
oauth2_token=OAuth2Token(client_id=config.XERO_CLIENT_ID, client_secret=config.XERO_CLIENT_SECRET),
),
pool_threads=1,
oauth2_token_getter=oauth2_token_getter,
oauth2_token_saver=oauth2_token_saver)
This doesn't work as I have no request object and can't pass one in:
def oauth2_token_getter():
return request.session.get("token") #! FAIL, I don't have access to request here
The error is thrown after calling identity_api.get_connections() as it tries to get the token using oauth2_token_getter and fails
def xero_tenants(request):
identity_api = IdentityApi(api_client)
accounting_api = AccountingApi(api_client)
available_tenants = []
for connection in identity_api.get_connections():
...
Any help appreciated, Thanks

Are you able to store the token in the django cache? (django.core.cache import cache) Then instead of specifying oauth2_token_getter=oauth2_token_getter in the ApiClient constructor you can do:
#api_client.oauth2_token_getter
def obtain_xero_oauth2_token():
return cache.get('token')
and something similar for the setter/saver.

How to debug Fastapi openapi generation error

I spend some time going over this error but had no success.
File "C:\Users\ebara.conda\envs\asci\lib\site-packages\fastapi\openapi\utils.py", line 388, in get_openapi
flat_models=flat_models, model_name_map=model_name_map
File "C:\Users\ebara.conda\envs\asci\lib\site-packages\fastapi\utils.py", line 28, in get_model_definitions
model_name = model_name_map[model]
KeyError: <class 'pydantic.main.Body_login_access_token_api_v1_login_access_token_post'>
The problem is that I'm trying to build a project with user authentication from OpenAPI form to create new users in database.
I've used backend part of this template project https://github.com/tiangolo/full-stack-fastapi-postgresql
Everything works except for Authentication like here.
#router.post("/login/access-token", response_model=schemas.Token)
def login_access_token(
db: Session = Depends(deps.get_db), form_data: OAuth2PasswordRequestForm = Depends()) -> Any:
When I add this part form_data: OAuth2PasswordRequestForm = Depends() - and go to /docs page - this error appears (Failed to load API definition. Fetch error. Internal Server Error /openapi.json)
.
The server itself runs in normal mode, but it can't load the open API. If I remove the aforementioned formdata part - then everything works smoothly, but without Authorisation. I tried to debug it, but I have no success. I think it might be connected to a dependency graph or some start-up issues, but have no guess how to trace it back.
Here is the full working example which will reproduce the error. The link points out the code which causes the problem. If you will comment out lines 18-39 - the docs will open without any problems.
https://github.com/BEEugene/fastapi_error_demo/blob/master/fastapi_service/api/api_v1/endpoints/login.py
Any ideas on how to debug or why this error happens?

You are using Depends function without an argument. Maybe in the console, you were having the error provoked by a function. You must pass the OAuth2PasswordRequestForm function after importing it from fastapi.security to get the result you were expecting.
from fastapi.security import OAuth2PasswordRequestForm
form_data: OAuth2PasswordRequestForm = Depends(OAuth2PasswordRequestForm)
it might work.

It seems that in my case - the main error issue was that I was an idiot.
As said, if you will comment out lines 18-39 - the docs will open without any problems. But, you will notice this warning:
UserWarning: Duplicate Operation ID read_users_api_v1_users__get for
function read_users at
...\fastapi_error\fastapi_service\api\api_v1\endpoints\users.py
warnings. Warn(message)
I've started to compare all the files and it is appeared that I included router to the fastapi twice:
import logging
from fastapi import FastAPI
from starlette.middleware.cors import CORSMiddleware
from fastapi_service.api.api_v1.api import api_router
from fastapi_service.core.config import settings
from fastapi_service.core.event_handlers import (start_app_handler,
stop_app_handler)
log = logging.getLogger(__name__)
def get_app(mode="prod") -> FastAPI:
fast_app = FastAPI(title=settings.PROJECT_NAME,
version=settings.APP_VERSION,
debug=settings.IS_DEBUG)
# openapi_url=f"{settings.API_V1_STR}/openapi.json")
# first time when I included the router
fast_app.include_router(api_router, prefix=f"{settings.API_V1_STR}")
fast_app.mode = mode
logger = log.getChild("get_app")
logger.info("adding startup")
fast_app.add_event_handler("startup", start_app_handler(fast_app))
logger.info("adding shutdown")
fast_app.add_event_handler("shutdown", stop_app_handler(fast_app))
return fast_app
app = get_app()
# Set all CORS enabled origins
if settings.BACKEND_CORS_ORIGINS:
app.add_middleware(
CORSMiddleware,
allow_origins=[str(origin) for origin in settings.BACKEND_CORS_ORIGINS],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# second time when I included the router
app.include_router(api_router, prefix=settings.API_V1_STR)
So, if you comment out (or just delete) the second router introduction - the app will work normally.
It seems, that the answer to my question on how to debug this error - is to find the point where the bug appears in fastapi and compare the values in it to the version where there is no error. In my case the number of keys in different dictionaries differed in the function get_model_definitions.

I had the same problem. For me it was because I had code like this
from pydantic import BaseModel
class A(BaseModel):
b: B
class B(BaseModel):
c: int
but instead, Class B should have been defined above class A. This fixed it:
from pydantic import BaseModel
class B(BaseModel):
c: int
class A(BaseModel):
b: B
More info: https://stackoverflow.com/a/70384637/9439097
Regaring your original question on how to debug these or similar errors:
You probably have your routes defined somewhere. Comment all of your routers/routes out, then the openapi docs should generate (and they should show you have no routes. Then, enable the routes one by one and see which one causes the error. THis is how I debuged my situation.

Flask authentication using htpassword with routes in seperate files

I'm working on updating an existing university department internal website to run using Flask. Before we completely release the project where it will use shibboleth to manage authentication, I've been trying to setup authentication for testing using htpassword using this tutorial. I'm able to get authentication using htpasswords working for a single file, but I can't figure out how to add the authentication to separate files. Currently, the project looks like this:
This is my main file called app.py:
#app.py
from flask import Flask
from flask_htpasswd import HtPasswdAuth
app = Flask(__name__)
app.config['FLASK_HTPASSWD_PATH'] = '.htpasswd'
app.config['FLASK_SECRET'] = "Super secret key that I will change later"
htpasswd = HtPasswdAuth(app)
import exampleRoute
#app.route('/')
#htpasswd.required
def home(user):
return "This is the homepage"
This is an example of one of my route files called exampleRoute.py:
# exampleRoute.py
from app import app
from flask import Flask
#app.route('/exampleRoute')
def exampleRoute():
return "This is an example route in another file"
When I place the #htpassword.required decorator in front of exampleRoute(), I get an error saying that htpassword is not defined (which makes sense). I've tried to import the app configuration a few different ways and at best, I end up with a circular dependency. I know that I could place the authentication code in a separate file and then import that into all my endpoints, but I didn't think this was possible since this method is incorporated into the app configuration.

I ended up getting an answer from the reddit user alexisprince. The solution was to use Blueprints that import htpasswd from another file (called extensions.py).

Webob / Pyramid query string parameters out of order upon receipt

I am running Pyramid as my API server. Recently we started getting query string parameters out of order when handed to the RESTful API server. For example, a GET to /v1/finishedGoodRequests?exact=true&id=39&join=OR&exact=false&name=39
is logged by the RESTful api module upon init as request.url:
v1/finishedGoodRequests?join=OR&name=39&exact=true&exact=false&id=39
with request.query_string: join=OR&name=39&exact=true&exact=false&id=39
I process the query params in order to qualify the search, in this case id exactly 39 or 39 anywhere in the name. What kind of possible server setting or bug could have crept in to the server code to cause such a thing? It is still a MultiDict...

As a simple example, this works fine for me, and the MultiDict has always preserved the order and so I suspect something is getting rewritten by something you're using in your stack.
from pyramid.config import Configurator
from pyramid.view import view_config
from waitress import serve
#view_config(renderer='json')
def view(request):
return list(request.GET.items())
config = Configurator()
config.scan(__name__)
app = config.make_wsgi_app()
serve(app, listen='127.0.0.1:8080')
$ curl http://localhost:8080\?join=OR\&name=39\&exact=true\&exact=false\&id=39
[["join", "OR"], ["name", "39"], ["exact", "true"], ["exact", "false"], ["id", "39"]]
Depending on which WSGI server you are using, often you can view environ vars to see the original url which may be handy. Waitress does not, so instead just put something high up in the pipeline (wsgi middleware) that can log out the environ['QUERY_STRING'] and see if it doesn't match somewhere lower down in your stack.

Testing Flask routes do and don't exist

I'm creating a large number of Flask routes using regular expressions. I'd like to have a unit test that checks that the correct routes exist and that incorrect routes 404.
One way of doing this would be to spin up a local server and use urllib2.urlopen or the like. However, I'd like to be able to run this test on Travis, and I'm assuming that's not an option.
Is there another way for me to test routes on my application?

Use the Flask.test_client() object in your unittests. The method returns a FlaskClient instance (a werkzeug.test.TestClient subclass), making it trivial to test routes.
The result of a call to the TestClient is a Response object, to see if it as 200 or 404 response test the Response.status_code attribute:
with app.test_client() as c:
response = c.get('/some/path/that/exists')
self.assertEquals(response.status_code, 200)
or
with app.test_client() as c:
response = c.get('/some/path/that/doesnt/exist')
self.assertEquals(response.status_code, 404)
See the Testing Flask Applications chapter of the Flask documentation.

Martjin's answer surely solve your issue, but some times you don't have the time (or will) to mock all the components you call in a route you want to test for existence.
And why would you need to mock? Well, the call get('some_route') will first check for this route to exists and then ... it will be executed!
If the view is a complex one and you need to have fixtures, envs variables and any other preparation just for test if the route exists, then you need to think again about your test design.
How to avoid this:
You can list all the routes in your app. An check the one you're testing is in the list.
In the following example, you can see this in practice with the implementation of a site-map.
from flask import Flask, url_for
app = Flask(__name__)
def has_no_empty_params(rule):
defaults = rule.defaults if rule.defaults is not None else ()
arguments = rule.arguments if rule.arguments is not None else ()
return len(defaults) >= len(arguments)
#app.route("/site-map")
def site_map():
links = []
for rule in app.url_map.iter_rules():
# Filter out rules we can't navigate to in a browser
# and rules that require parameters
if "GET" in rule.methods and has_no_empty_params(rule):
url = url_for(rule.endpoint, **(rule.defaults or {}))
links.append((url, rule.endpoint))
# links is now a list of url, endpoint tuples
references:
get a list of all routes defined in the app
Helper to list routes (like Rail's rake routes)

Another way of testing a URL without executing the attached view function is using the method match of MapAdapter.
from flask import Flask
app = Flask(__name__)
#app.route('/users')
def list_users():
return ''
#app.route('/users/<int:id>')
def get_user(id):
return ''
Testing
# Get a new MapAdapter instance. For testing purpose, an empty string is fine
# for the server name.
adapter = app.url_map.bind('')
# This raise werkzeug.exceptions.NotFound.
adapter.match('/unknown')
# This raises werkzeug.exceptions.MethodNotAllowed,
# Although the route exists, the POST method was not defined.
adapter.match('/users', method='POST')
# No exception occurs when there is a match..
adapter.match('/users')
adapter.match('/users/1')
From Werkzeug documentation:
you get a tuple in the form (endpoint, arguments) if there is a match.
Which may be useful in certain testing scenarios.