Develop Reference

Unable to fetch OAuth2.0 Token from Azure-DevOps , using Python

I am trying to use OAuth2 to access the Azure DevopsAPI, to query work-items.
But I am unable to get the access tokene.
I am using Python and Flask. My approach is based on these resources:
Microsoft documentation , there currently Step 3 is relevant
OAuth Tutorial, which worked fine for Github, but is not working for Azure.
Relevant libraries:
from requests_oauthlib import OAuth2Session
from flask import Flask, request, redirect, session, url_for
Parameters:
client_id = "..."
client_secret = "..."
authorization_base_url = "https://app.vssps.visualstudio.com/oauth2/authorize"
token_url = "https://app.vssps.visualstudio.com/oauth2/token"
callback_url = "..."
Step 1: User Authorization. (works fine)
#app.route("/")
def demo():
azure = OAuth2Session(client_id)
authorization_url, state = azure.authorization_url(authorization_base_url)
session['oauth_state'] = state
authorization_url += "&scope=" + authorized_scopes + "&redirect_uri=" + callback_url
print(authorization_url)
return redirect(authorization_url)
Step 2: Retrieving an access token (generates an error)
#app.route("/callback", methods=["GET"])
def callback():
fetch_body = "client_assertion_type=urn:ietf:params:oauth:client-assertion-type:jwt-bearer" \
"&client_assertion=" + client_secret + \
"&grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer" \
"&assertion=" + request.args["code"] + \
"&redirect_uri=" + callback_url
azure = OAuth2Session(client_id, state=session['oauth_state'])
token = azure.fetch_token(token_url=token_url, client_secret=client_secret,
body=fetch_body,
authorization_response=request.url)
azure.request()
session['oauth_token'] = token
return redirect(url_for('.profile'))
The application-registration and adhoc-SSL-certification are working fine (using it just temporary).
When I use the client_assertion in Postman, I get a correct response from Azure:
But when I execute the code, this error is thrown:
oauthlib.oauth2.rfc6749.errors.MissingTokenError: (missing_token) Missing access token parameter.
Which only lets me know, that no token was received.
There is one issue in the generated request body, where the grant_type is added twice:
grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer
grant_type=authorization_code
The first value is expected by Azure, but the second one is generated automatically by the library.
Now when I specify the grant_type in the fetch_token call, like this:
token = azure.fetch_token(token_url=token_url, client_secret=client_secret,
body=fetch_body, grant_type="urn:ietf:params:oauth:grant-type:jwt-bearer",
authorization_response=request.url)
I get this error
TypeError: prepare_token_request() got multiple values for argument 'grant_type'
And the actual request to Azure is not even sent.
I see in the web_application.py that is used by oauth2_session.py, that grant_type ='authorization_code' is set fixed, so I guess this library is generally incompatible with Azure.
Is that the case?
If so, what would be the simplest way to connect to Azure-OAuth with Python (Flask)?
I would be very grateful for any advice and help that point me in the right direction.

I just found the azure.devops library that solves my problem.
Ressources
https://github.com/Microsoft/azure-devops-python-api
https://github.com/microsoft/azure-devops-python-samples/blob/main/src/samples/work_item_tracking.py
azure-devops-python-api query for work item where field == string
from azure.devops.connection import Connection
from azure.devops.v5_1.work_item_tracking import Wiql
from msrest.authentication import BasicAuthentication
import pprint
# Fill in with your personal access token and org URL
personal_access_token = '... PAT'
organization_url = 'https://dev.azure.com/....'
# Create a connection to the org
credentials = BasicAuthentication('', personal_access_token)
connection = Connection(base_url=organization_url, creds=credentials)
# Get a client (the "core" client provides access to projects, teams, etc)
core_client = connection.clients.get_core_client()
wit_client = connection.clients.get_work_item_tracking_client()
query = "SELECT [System.Id], [System.WorkItemType], [System.Title], [System.AssignedTo], [System.State]," \
"[System.Tags] FROM workitems WHERE [System.TeamProject] = 'Test'"
wiql = Wiql(query=query)
query_results = wit_client.query_by_wiql(wiql).work_items
for item in query_results:
work_item = wit_client.get_work_item(item.id)
pprint.pprint(work_item.fields['System.Title'])

Unit Testing Replace remote API Server with predefined response

So, I have a server running FastAPI which will make a API call to a remote API upon request.
I am developping unit-testing for this application, but here comes the question:
Can I, for the purpose of the test, replace a legit remote API server response by a predefined response ?
Example of the tests runned:
from fastapi.testclient import TestClient
from web_api import app
client = TestClient(app)
def test_get_root():
response = client.get('/')
assert response.status_code == 200
assert response.json() == {"running": True}
And the my server
from fastapi import FastAPI
app = FastAPI()
#app.get("/")
def home():
return {"running": True}
This is a simple example, but on other endpoints of my API I would call an external remote API
def call_api(self, endpoint:str, params:dict):
url = self.BASEURL + urllib.parse.quote(endpoint)
try:
response = requests.get(url, params=params)
response.raise_for_status()
except requests.exceptions.HTTPError as error:
print(error)
return response
Because I want to test the response of MY API, I would like to replace the remote API with a predefined response.
Also, one user request can end-up in multiple background API requests with transformed pieces of data.
Edit
Here are some more details on the structure of the application:
#app.get("/stuff/.......",
# lots of params
)
def get_stuff_from_things(stuff:list, params):
api = API(api_key=...)
# Do some stuff with the params
things = generate_things_list(params)
api.search_things(params)
# Check the result
# do some other stuff
return some_response
class API:
BASE_URL = 'https://api.example.com/'
def search_things(self, params):
# Do some stuff
# like putting stuff in the params
for s in stuff:
s.update(self.get_thing(params)) # -> get_thing()
# Do some more stuff
return stuff
# get_thing <- search_things
def get_thing(self, params...):
# Some stuff
results = self.call_api('something', params) # -> call_api()
json = results.json()
# Some more stuff
things = []
for thing in json['things']:
t = Thing(thing)
things.append(t)
return things
# call_api <- get_thing
def call_api(self, endpoint:str, params:dict):
url = self.BASEURL + urllib.parse.quote(endpoint)
try:
response = requests.get(url, params=params)
response.raise_for_status()
except requests.exceptions.HTTPError as error:
print(error)
self.last_response = response
return response
Nb. That is pseudo-code, I simplified the functions by removing the parameters, etc.
I hope it is clear, thanks for your help.

A complex API method might look like this (please pay attention to the depends mechanism - it is crucial):
import urllib
import requests
from fastapi import FastAPI, Depends
app = FastAPI()
# this can be in a different file
class RemoteCallWrapper:
def call_api(self, baseurl: str, endpoint: str, params: dict):
url = baseurl + urllib.parse.quote(endpoint)
try:
response = requests.get(url, params=params)
response.raise_for_status()
except requests.exceptions.HTTPError as error:
print(error)
return response
#app.get("/complex_api")
def calls_other_api(remote_call_wrapper=Depends(RemoteCallWrapper)):
response = remote_call_wrapper.call_api("https://jsonplaceholder.typicode.com",
"/todos/1", None)
return {"result": response.json()}
Now, we wish to replace the remote call class. I wrote a helper library that simplifies the replacement for tests - pytest-fastapi-deps:
from fastapi.testclient import TestClient
from mock.mock import Mock
from requests import Response
from web_api import app, RemoteCallWrapper
client = TestClient(app)
class MyRemoteCallWrapper:
def call_api(self, baseurl: str, endpoint: str, params: dict):
the_response = Mock(spec=Response)
the_response.json.return_value = {"my": "response"}
return the_response
def test_get_root(fastapi_dep):
with fastapi_dep(app).override({RemoteCallWrapper: MyRemoteCallWrapper}):
response = client.get('/complex_api')
assert response.status_code == 200
assert response.json() == {"result": {"my": "response"}}
You override the RemoteCallWrapper with your MyRemoteCallWrapper implementation for the test, which has the same spec.
As asserted - the response changed to our predefined response.

It sounds like you'd want to mock your call_api() function.
With a small modification to call_api() (returning the result of .json()), you can easily mock the whole function while calling the endpoint in your tests.
I'll use two files, app.py and test_app.py, to demonstrate how I would do this:
# app.py
import requests
import urllib
from fastapi import FastAPI
app = FastAPI()
def call_api(self, endpoint: str, params: dict):
url = self.BASEURL + urllib.parse.quote(endpoint)
try:
response = requests.get(url, params=params)
response.raise_for_status()
except requests.exceptions.HTTPError as error:
print(error)
return response.json() # <-- This is the only change. Makes it easier to test things.
#app.get("/")
def home():
return {"running": True}
#app.get("/call-api")
def make_call_to_external_api():
# `endpoint` and `params` could be anything here and could be different
# depending on the query parameters when calling this endpoint.
response = call_api(endpoint="something", params={})
# Do something with the response...
result = response["some_parameter"]
return result
# test_app.py
from unittest import mock
from fastapi import status
from fastapi.testclient import TestClient
import app as app_module
from app import app
def test_call_api_endpoint():
test_response = {
"some_parameter": "some_value",
"another_parameter": "another_value",
}
# The line below will "replace" the result of `call_api()` with whatever
# is given in `return_value`. The original function is never executed.
with mock.patch.object(app_module, "call_api", return_value=test_response) as mock_call:
with TestClient(app) as client:
res = client.get("/call-api")
assert res.status_code == status.HTTP_200_OK
assert res.json() == "some_value"
# Make sure the function has been called with the right parameters.
# This could be dynamic based on how the endpoint has been called.
mock_call.assert_called_once_with(endpoint="something", params={})
If app.py and test_app.py are in the same directory you can run the tests simply by running pytest inside that directory.

Web App MSAL error, 200 error on redirect

Errors:
Access to fetch at 'https://login.microsoftonline.com/{bunchofinfo}' (redirected from 'http://localhost:5000/login') from origin 'http://localhost:3000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
GET https://login.microsoftonline.com/{bunchofinfo} net::ERR_FAILED 200
Code:
#app.route("/")
def index():
if not session.get("user"):
return redirect(url_for("login"))
return redirect('http://localhost:3000/')
#app.route("/login")
def login():
session["flow"] = _build_auth_code_flow(scopes=SCOPE_MSAL)
auth_url = session["flow"]["auth_uri"]
print(auth_url)
return redirect(auth_url)
Things Ive tried:
First tried to set {'mode': 'no-cors'} and redirect. But it gives me the HTML for the redirect and doesn't redirect me.
Then tried sending my frontend the auth_url with no redirect. Then window.location.assign(auth_url)
Problem = Once logged on properly, it stays on localhost:5000/login with the auth_url on the page.
Tried to switch code with this but it doesn't move on after login and runs the if not statement.
def login():
if not session.get("user"):
session["flow"] = _build_auth_code_flow(scopes=SCOPE_MSAL)
auth_url = session["flow"]["auth_uri"]
print(auth_url)
return auth_url
return redirect('http://localhost:3000')
Don't know how to proceed. Rest of code from MSAL
#app.route("/getAToken")
def authorized():
try:
cache = _load_cache()
result = _build_msal_app(cache=cache).acquire_token_by_auth_code_flow(
session.get("flow", {}), request.args)
if "error" in result:
print(result)
# add error page
# return render_template("auth_error.html", result=result)
session["user"] = result.get("id_token_claims")
_save_cache(cache)
except ValueError: # Usually caused by CSRF
pass # Simply ignore them
return redirect(url_for("index"))
def _load_cache():
cache = msal.SerializableTokenCache()
if session.get("token_cache"):
cache.deserialize(session["token_cache"])
return cache
def _save_cache(cache):
if cache.has_state_changed:
session["token_cache"] = cache.serialize()
def _build_msal_app(cache=None, authority=None):
return msal.ConfidentialClientApplication(
microsoft_client_id, authority=authority or AUTHORITY,
client_credential=client_secret, token_cache=cache)
def _build_auth_code_flow(authority=None, scopes=None):
return _build_msal_app(authority=authority).initiate_auth_code_flow(
scopes or [],
redirect_uri=url_for("authorized", _external=True))
def _get_token_from_cache(scope=None):
cache = _load_cache() # This web app maintains one cache per session
cca = _build_msal_app(cache=cache)
accounts = cca.get_accounts()
if accounts: # So all account(s) belong to the current signed-in user
result = cca.acquire_token_silent(scope, account=accounts[0])
_save_cache(cache)
return result

Heroku app having trouble with Spotify API token

So I've been using Flask to create an app that uses Spotify API, and the authorization code flow works perfectly fine when running on my localhost server. However, after deploying the app to Heroku, occasionally the app will crash and give me a 'token not provided' error in the logs. Sometimes it'll work flawlessly, but when I play around with it more it'll crash. I try to redo the process on my localhost but it doesn't seem to break. I'm storing the token in sessions, could it be that Heroku is having problem with retrieving session variables?
Here's the flask app code
#Home view
#app.route('/')
#app.route('/home')
def home():
return render_template('home.html', title='Home')
#Login view
#app.route('/login', methods=['GET','POST'])
def login():
AUTH_FIRST = req_auth()
return redirect(AUTH_FIRST)
#Callback view for Spotify API
#app.route('/callback')
def callback():
if request.args.get('code'):
code = request.args.get('code')
token = req_token(code)
session['token'] = token
return redirect(url_for('generate_playlist'))
else:
return redirect(url_for('home'))
#Generate playlist view
#app.route('/generate_playlist', methods=['GET', 'POST'])
def generate_playlist():
if request.method == 'POST':
levels = int(float(request.form['level']))
token = session.get('token')
generate(token, levels)
return redirect(url_for('success'))
else:
return redirect(url_for('generate_playlist'))
This is the backend authorization code (I've commented out some parts to make it simpler, yet it still doesn't work):
client_id = os.environ.get("CLIENT_ID")
client_secret = os.environ.get("CLIENT_SECRET")
redirect_uri = os.environ.get('REDIRECT_URI')
state = ''.join(random.choice(string.ascii_lowercase + string.digits) for n in range(8))
scope = 'user-top-read user-library-read playlist-modify-public'
def req_auth():
show_dialog = "false"
AUTH_FIRST_URL = f'https://accounts.spotify.com/authorize?client_id={client_id}&response_type=code&redirect_uri={quote("https://surprisify-playlist-generator.herokuapp.com/callback")}&show_dialog={show_dialog}&state={state}&scope={scope}'
return AUTH_FIRST_URL
def req_token(code):
#B64 encode variables
client_creds = f"{client_id}:{client_secret}"
client_creds_b64 = base64.b64encode(client_creds.encode())
#Token data
token_data = {
"grant_type": "authorization_code",
"code": code,
"redirect_uri": "https://surprisify-playlist-generator.herokuapp.com/callback"
}
#Token header
token_header = {
"Authorization": f"Basic {client_creds_b64.decode()}"
}
#Make request post for token info
token_json = requests.post('https://accounts.spotify.com/api/token', data=token_data, headers=token_header)
return token_json.json()['access_token']
#Checking if token is still valid, otherwise, refresh
# if token_json.json()['expires_in']:
# now = datetime.datetime.now()
# expires_in = token_json.json()['expires_in']
# expires = now + datetime.timedelta(seconds=expires_in)
# if now > expires:
# refresh_token_data = {
# "grant_type": "refresh_token",
# "refresh_token": token_json.json()['refresh_token']
# }
# refresh_token_json = requests.post('https://accounts.spotify.com/api/token', data=refresh_token_data, headers=token_header)
# token = refresh_token_json.json()['access_token']
# return token
# else:
# token = token_json.json()['access_token']
# return token
# else:
# token = token_json.json()['access_token']
# return token

Could the error happen after you restart the browser? Flask session cookies are stored (by default) until you close the browser. Then, if you don't authenticate again, calling session.get(token) will return None and is likely why you get the error. You could try checking if token is None in generate_playlist() and requiring re-authentication with a redirect.
Here is an implementation of the Spotify authorization code flow using Spotipy that I have used with success in my own project: https://stackoverflow.com/a/57929497/6538328 (method 2).

Flask test_client removes query string parameters

I am using Flask to create a couple of very simple services. From outside testing (using HTTPie) parameters through querystring are getting to the service.
But if I am using something like.
data = {
'param1': 'somevalue1',
'param2': 'somevalue2'}
response = self.client.get(url_for("api.my-service", **data))
I can see the correct URI being created:
http://localhost:5000/api1.0/my-service?param1=somevalue1&param2=somevalue2
when I breakpoint into the service:
request.args
is actually empty.
self.client is created by calling app.test_client() on my configured Flask application.
Anyone has any idea why anything after ? is being thrown away or how to work around it while still using test_client?

I've just found out a workaround.
Make
data = {
'param1': 'somevalue1',
'param2': 'somevalue2'}
response = self.client.get(url_for("api.my-service", **data))
into this:
data = {
'param1': 'somevalue1',
'param2': 'somevalue2'}
response = self.client.get(url_for("api.my-service"), query_string = data)
This works but seems a bit unintuitive, and debugging there is a place where the provided query string in the URI is thrown away ....
But anyway this works for the moment.

I know this is an old post, but I ran into this too. There's an open issue about this in the flask github repository. It appears this is intended behavior. From a response in the issue thread:
mitsuhiko commented on Jul 24, 2013
That's currently intended behavior. The first parameter to the test client should be a relative url. If it's not, then the parameters are removed as it's treated as if it was url joined with the second. This works:
>>> from flask import Flask, request
>>> app = Flask(__name__)
>>> app.testing = True
>>> #app.route('/')
... def index():
... return request.url
...
>>> c = app.test_client()
>>> c.get('/?foo=bar').data
'http://localhost/?foo=bar'
One way to convert your absolute url into a relative url and keep the query string is to use urlparse:
from urlparse import urlparse
absolute_url = "http://someurl.com/path/to/endpoint?q=test"
parsed = urlparse(absolute_url)
path = parsed[2] or "/"
query = parsed[4]
relative_url = "{}?{}".format(path, query)
print relative_url

If you are trying any other HTTP Method other than GET
response = self.client.patch(url_for("api.my-service"), query_string=data,
data="{}")
data="{}" or data=json.dumps({}) should be there, even if there is no content in the body. Otherwise, it results in BAD Request.

For me the solution was to use the client within with statements:
with app.app_context():
with app.test_request_context():
with app.test_client() as client:
client.get(...)
instead of
client = app.test_client()
client.get(...)
I put the creation of the test client in a fixture, so that it is "automatically" created for each test method:
from my_back_end import MyBackEnd
sut = None
app = None
client = None
#pytest.fixture(autouse=True)
def before_each():
global sut, app, client
sut = MyBackEnd()
app = sut.create_application('frontEndPathMock')
with app.app_context():
with app.test_request_context():
with app.test_client() as client:
yield