I am building a python flask API.
The requests are sent from the UI and they include an already authorized JWT token in the header (as expected bearer token).
My API service needs to validate the token and extract the tenant from the token.
I have a few questions about how a solution like that usually works.
does the API just need to decode the token and get the tenant from the payload?
the api cannot validate the signature of the token because I don't have the secret, right?
should the api juat call the auth. Server (in thia case key cloak)? If yes can someone please write an example code for that. Which libraries are good to use?
Thank you
Here are some answers.
From a pure technical point of view, you can indeed decode the JWT token (there exists plenty of libs to do it) to extract the payload claims, and you can do it "serverless"
The secret is used to CREATE the signature, not to verify it. When the signature is created (by the keycloak server) using a private key, the client app can verify it using the public key
Issuing an additional request to the KC server in order to decode the token has yet some added value. The (possibly long-living) token may, for some security reason, have been discarded to prevent if from being still used. In such case, the API will return a negative answer. The web service you need to invoke is the "Token Introspection Endpoint" (https://www.oauth.com/oauth2-servers/token-introspection-endpoint/)
Related
Goal:
I have set up a Google API Gateway. The backend for the API is a cloud function (written in python). The cloud function should query data from Google BigQuery (BQ). To do that, I want to create a BQ Client (google.cloud.bigquery.Client()). The API should be accessed by different applications using different service accounts. The service accounts have permission to access only specific datasets within my project. Therefore, the service accounts/applications should only be able to query the datasets they have the permission for. Therefore, the BQ Client within the cloud function should be initialized with the service account that sends the request to the API.
What I tried:
The API is secured with the following OpenAPI definition so that a JWT signed by the service account SA-EMAIL is required to send a request there:
securityDefinitions:
sec-def1:
authorizationUrl: ""
flow: "implicit"
type: "oauth2"
x-google-issuer: "SA-EMAIL"
x-google-jwks_uri: "https://www.googleapis.com/robot/v1/metadata/x509/SA-EMAIL"
x-google-audiences: "SERVICE"
For the path that uses my cloud function, I use the following backend configuration:
x-google-backend:
address: https://PROJECT-ID.cloudfunctions.net/CLOUD-FUNCTION
path_translation: CONSTANT_ADDRESS
So in the cloud function itself I get the forwarded JWT as X-Forwarded-Authorization and also the already verified base64url encoded JWT payload as X-Apigateway-Api-Userinfo from the API Gateway.
I tried to use the JWT from X-Forwarded-Authorization to obtain credentials:
bearer_token = request.headers.get('X-Forwarded-Authorization')
token = bearer_token.split(" ")[1]
cred = google.auth.credentials.Credentials(token)
At first, this seems to work since cred.valid returns True, but when trying to create the client with google.cloud.bigquery.Client(credentials=cred) it returns the following error in the logs:
google.auth.exceptions.RefreshError: The credentials do not contain
the necessary fields need to refresh the access token. You must
specify refresh_token, token_uri, client_id, and client_secret.
I do not have much experience with auth/oauth at all, but I think I do not have the necessary tokens/attributes the error is saying are missing available in my cloud function.
Also, I am not exactly sure why there is a RefreshError, since I don't want to refresh the token (and don't do so explicitly) and just use it again (might be bad practice?).
Question:
Is it possible to achieve my goal in the way I have tried or in any other way?
Your goal is to catch the credential that called the API Gateway, and to reuse it in your Cloud Functions to call BigQuery.
Sadly, you can't. Why? Because API Gateway prevent you to achieve that (and it's a good news for security reason). The JWT token is correctly forwarded to your Cloud Functions, but the signature part has been removed (you receive only the header and the body of the JWT token).
The security verification has been done by API Gateway and you have to rely on that authentication.
What's the solution?
My solution is the following: In the truncated JWT that you receive, you can get the body and get the Service Account email. From there, you can use the Cloud Functions service account, to impersonate the Service Account email that you receive.
Like that, the Cloud Functions service account only needs the permission to impersonate these service account, and you keep the permission provided on the original service account.
I don't see other solutions to solve your issue.
The JWT that you are receiving from API Gateway is not an OAuth Access Token. Therefore the JWT Payload portion is not a credential that you can use for the BigQuery Client authorization.
As #guillaume blaquiere pointed out, the Payload contains the email address of an identity. If the identity is a service account, you could implement impersonation of that identity. This could be a good solution if you are using multiple identities with API Gateway. If the identity is a user account, then you would need to implement Domain-Wide Delegation.
I recommend simply using the service account assigned to the Cloud Function with proper roles assigned to initialize the BigQuery client. Provided that API Gateway is providing authorization to reach your Cloud Function, there is no need for the extra layer of impersonation.
Another option is to store the matching service account JSON key file in Secret Manager and pull it when required to create the BigQuery Client.
According to the documentation, I can either send the request with authorization (token) in order to get all of my gists, or anonymously and I will get public popular gists.
My Python code is:
url = "https://api.github.com/gists"
with Get(
url,
headers={"Accept": accept},
params={"since": since, "per_page": per_page, "page": page},
auth=("token", token)
) as response:
return response
When token is set to None, I get all public gists (not mine) and when token is set to my OAuth token, I get all of my gists.
However, the issue is that it only gives me my non-secret gists instead of secret and public together.
Initially I was thinking that my token was wrong and therefore I was not getting the secret gists, but turns out that the token is correct (for sure, I can even post new gists) and also has permissions to read/write gists, and that is why it is weird.
The issue is also not related to either params or headers, tested.
Additional Information:
Get is a class which implements a context-manager and sends a get request [link].
After a long research I found out that GitHub's OAuth token from Developer Settings is not enough to perform this action and I need to create a GitHub App in order to extend GitHub.
I used this tool:
https://github.com/defunkt/gist
in order to ask GitHub for such a particular token (which is being used in the GitHub App), and then I started using it, and it worked!
With the new fine grained personal access tokens this can now be done without a GitHub App:
You need to give read-write access to Gists under Account Permissions:
I am evaluating different options for authentication in a python App Engine flex environment, for apps that run within a G Suite domain.
I am trying to put together the OpenID Connect "Server flow" instructions here with how google-auth-library-python implements the general OAuth2 instructions here.
I kind of follow things up until 4. Exchange code for access token and ID token, which looks like flow.fetch_token, except it says "response to this request contains the following fields in a JSON array," and it includes not just the access token but the id token and other things. I did see this patch to the library. Does that mean I could use some flow.fetch_token to create an IDTokenCredentials (how?) and then use this to build an OpenID Connect API client (and where is that API documented)? And what about validating the id token, is there a separate python library to help with that or is that part of the API library?
It is all very confusing. A great deal would be cleared up with some actual "soup to nuts" example code but I haven't found anything anywhere on the internet, which makes me think (a) perhaps this is not a viable way to do authentication, or (b) it is so recent the python libraries have not caught up? I would however much rather do authentication on the server than in the client with Google Sign-In.
Any suggestions or links to code are much appreciated.
It seems Google's python library contains a module for id token validation. This can be found at google.oauth2.id_token module. Once validated, it will return the decoded token which you can use to obtain user information.
from google.oauth2 import id_token
from google.auth.transport import requests
request = requests.Request()
id_info = id_token.verify_oauth2_token(
token, request, 'my-client-id.example.com')
if id_info['iss'] != 'https://accounts.google.com':
raise ValueError('Wrong issuer.')
userid = id_info['sub']
Once you obtain user information, you should follow authentication process as described in Authenticate the user section.
OK, I think I found my answer in the source code now.
google.oauth2.credentials.Credentials exposes id_token:
Depending on the authorization server and the scopes requested, this may be populated when credentials are obtained and updated when refresh is called. This token is a JWT. It can be verified and decoded [as #kavindu-dodanduwa pointed out] using google.oauth2.id_token.verify_oauth2_token.
And several layers down the call stack we can see fetch_token does some minimal validation of the response JSON (checking that an access token was returned, etc.) but basically passes through whatever it gets from the token endpoint, including (i.e. if an OpenID Connect scope is included) the id token as a JWT.
EDIT:
And the final piece of the puzzle is the translation of tokens from the (generic) OAuthSession to (Google-specific) credentials in google_auth_oauthlib.helpers, where the id_token is grabbed, if it exists.
Note that the generic oauthlib library does seem to implement OpenID Connect now, but looks to be very recent and in process (July 2018). Google doesn't seem to use any of this at the moment (this threw me off a bit).
My project is python and using boto3 lib.
I'm using aws cognito Authorization code grant flow with return_type=code instead of return_type=token (implicit flow). Once my user is authorized my redirect url is injected with the queryStringParameter code=4d55a121-8ffc-4058-844b-xxxx.
outlined here
I need to be able to verify this code. Because of course someone can take the redirect url and make a fake code and paste it into the browser. According to this doc I can exchange the code for a token. This works as expected via a rest client. I get the token and can continue to pass the token as the Authorization header. But what I'm asking is there has to be a boto3 method that takes this code and converts it into a token for me. If i have to use the requests lib I will.
I have tried for days. get_user isnt the answer as that requires a token not the code.
For reference on what I'm trying to do heres my repo. The focus is in def edit(). I'm currently using requests to achieve the same thing but would like to use the boto library
https://github.com/knittledan/python-lambda-cognito
Nope, believe you should use an https client to exchange the authorization code for tokens with the token endpoint provided:
https://docs.aws.amazon.com/cognito/latest/developerguide/token-endpoint.html
I've looked around for this but can't seem to find a canonical answer. I'm trying to follow best practices. Maybe I'm thinking of it the wrong way.
I'm thinking of my API users as two different types: developers and end users, with separate django models in separate applications.
Developers will build clients for the API, and have access to certain resources of the API without the need of users login in. To limit their access, I would require them to register and in exchange give them an API key. We would also dogfood this to say, build a site frontend using Angular and iOS app.
Once those developers build their API clients, users of my site, which have already created a user account, will use the API clients created by developers. In the request from those clients I would expect a developer name, api_key as well as username/password (digest, if its our own trusted client and oauth token for thid party developers). This will require to check 1) developers are allowed to use the API by checking their APIKey, and 2) authenticate the end user. Is this possible in tastypie?
Am I going about this the wrong way? How would I do the double authentication?
We run a production site with this exact scheme. Of course you'll have to do your own tunning. But the general idea is good. You could have some OAuth inplace too, but we've found that it's not worth it. OAuth is too complicated for most cases.
I'll explain roughly what we do.
This is the App/developers part:
We identify "apps" (iOS, Android, BB, the site). Each app has an ApiClient instance model. The ApiClient has three attrs: name, public key, and private key.
We exchange the public and private keys through a safe channel with the ApiClient owner (the app).
The app must send every request indicating the public key and a signature generated with the private key (using hmac).
Everytime we get a request, we get the public key from the request, look it up in the DB, see what App it belongs too (the name) and check the signature. If everything is ok, the request is fulfilled.
For the user authentication part:
To authenticate a user we use other model ApiKey (provided by tastypie). Each user has an ApiKey. That model stores a unique (we could say random) string. When the user gets to the app he/she logs in into your API. The app should issue a request similar to this one:
POST /api/v1/login/
{
'username': 'XXX',
'password': 'XXX'
}
(please note that it always need to pass the previous public/private key auth)
If the user provided the right credentials we return an ApiKey unique key.
Every following request made by the app in behave of that user must include the key. That's the way you identify which user is trying to do each action.
An example of this last part:
User Jon logs in in the iOS app. (using regular username and password)
The app sends the request:
POST /api/v1/login/
{
'username': 'jon',
'password': 'snow'
}
We have a login API method. We check if the user exists and if the pass is ok. Suppose it's ok.
We sent the ApiKey info:
200 OK
{
'username': 'jon',
'key': '$123$'
}
The app has authenticated the user. It needs to use those credentials.
The user tries to do something in your app. Suppose he tries to get the datetime from your app. The app will issue this request:
GET /api/v1/date/
Authorization: ApiKey jon:$123$
That's it. It's not super safe. The ApiKeys are not invalidated. But that's because we create our own internal Apps. It's worth to note that we borrow some stuff from Tastypie from this. Check this out: http://django-tastypie.readthedocs.org/en/latest/authentication.html#apikeyauthentication
This is tangental, but you may want to check out drf-keypair-permissions.
It's a Django module that uses asymmetric public/private keypairs to sign and verify the HTTP request using pre-shared keys. It can pair each public key with a user so the authorization doubles as a login, and can be extended to manage API throttling.
It supports a few algorithms, including RSA-SHA and elliptic curve, and keys can be managed in the admin area.
It uses the IETC Cavage-12 draft standard for processing the Authorization signature