I am trying to download some data from a customer's Microsoft Dyanmics 365 CRM API. Below is my code written in Python 3.7, using MSAL (Microsoft Authentication Library) for Python.
import msal
import requests
class Downloader:
AUTHORITY = 'https://login.microsoftonline.com/common'
SCOPES = ["https://admin.services.crm.dynamics.com/user_impersonation"]
def __init__(self, client_id, client_credential, refresh_token):
self.client_id = client_id
self.client_credential = client_credential
self.refresh_token = refresh_token
def download(self):
application = msal.ConfidentialClientApplication(
client_id=self.client_id,
client_credential=self.client_credential,
authority=self.AUTHORITY,
token_cache=None)
response = application.acquire_token_by_refresh_token(self.refresh_token, self.SCOPES)
instances = requests.get(
'https://globaldisco.crm.dynamics.com/api/discovery/v1.0/Instances',
headers={'Authorization': 'Bearer ' + response['access_token']},
)
return response
When I run the download() method I successfully get a response that contains an Access Token but when I use it to GET instances I get back HTTP Status Code 401 Access Denied.
My question is: Why am I getting 401 Access Denied and how can I fix my code to get 200 OK?
More data:
client_id and client_credential identify a multi-tenant Azure App Registration I've setup. This app registration is not verified yet. It has these API Permissions (scopes) configured:
https://admin.services.crm.dynamics.com/user_impersonation
https://graph.microsoft.com/User.Read
refresh_token has been generated based on Quickstart: Add sign-in with Microsoft to a Python web app. The quickstart goes through setting up a small Flask web application where a person can sign in via Microsoft Identity and grant permission for my app registration to impersonate them when later accessing the Dynamics 365 CRM API. I've setup the Flask app to use my App Registration and the two scopes above.
Because my App Registration is not verified yet, I had an admin of the target tenet sign in via the Flask app, approve and give consent for my app registration to be used by the tenet org. I've tried both their refresh token and another non-admin person's refresh token. Both yield 401.
The tenet org (to which my customers' identities belong) is linked to a Dynamics 365 account.
Update:
When I view my app registration in my customer's (i.e. tenet's) Azure Portal under Enterprise Applications I can see the Common Data Service user_impersonation permission under the Admin consent tab but it's missing from the User consent tab.
I'm trying to figure out whether this might explain why users are denied access and how to add the missing permission to User consent.
Has the user been added to CRM? The identity you are using should still have full roles assigned within CRM. Someone should have added that app registration to the list of users in CRM. Settings->Security (change the list view to "Application Users") press "New", change the form from "user" to "Application User".
Set the Name, copy the application Id from the Application registration. The email can exist or not.
I am trying to enable SSO using keycloak as identity broker and Microsoft AD as identity provider(where keycloak will delegate the client's authentication request to AD) in a django python project.
Tech stack of application:
frontend- React,
backend - django python
For this I am using python-keycloak library in django to communicate with keycloak.What I am able to achieve is : setup a connection with keycloak and get access_token and refresh_token when username and password is provided like this:
# Create Keycloak instance
self.keycloak = KeycloakOpenID(server_url=self.server_url,
client_id=self.client_id,
realm_name=self.realm,
client_secret_key=self.client_secret_key)
# Get WellKnow
self.config_well_know = self.keycloak.well_know()
# Get Token
self.token = self.keycloak.token("user","pwd")
# get userinfo
self.userInfo = self.keycloak.userinfo(self.token['access_token'])
# userinfo returned ok
But here i am providing username and password which I should not as I want to enable sso with Microsoft AD(Note: keycloak realm is configured to use Microsoft AD as default IDP) and only username should be sufficient to enable SSO with Microsoft. But it is giving error on passing only username.
Question:
How to authenticate user from Microsoft AD using keycloak broker and what should be the syntax for the same?
Create two clients in your example realm. One for your React app, setup as Public client where you should use Javascript adapter to generate access token where u pass that using headers and your backend Django app setup as a confidential client that could access the access token generated from the fronted react app. Using python-keycloak Library do introspect the token for its correctness.
Try setting up Identity brokering in your example realm using OIDC/SAML use your Azure apps metadata URL to set up the profile along with relevant mappers.
After this setup you should get SSO to work properly.
I am able to authorize service accounts by creating and downloading the credentials.json in the GC console. But this option is not available for actual human user accounts. My user account has certain roles that I'd like to use within by app and need to auth as that user account. How can I go about that? Is there a way to download a similar creds.json file for user accounts?
After some digging and in part thanks to #JohnHanley, I managed to use the google-cloud oauth library to auth my application on behalf of a human user that had the right permissions:
from google.cloud import secretmanager_v1beta1 as secretmanager
from google_auth_oauthlib import flow
SecretManagerAdminCreds = 'credentials.json'
LAUNCH_BROWSER = True
class SecretManagerAdmin:
def __init__(self, project_id: str = "gcp_project_id",
scopes: list = ['https://www.googleapis.com/auth/cloud-platform']):
self._scopes = scopes
self._project_id = project_id
appflow = flow.InstalledAppFlow.from_client_secrets_file(SecretManagerAdminCreds, scopes=self._scopes)
if LAUNCH_BROWSER:
appflow.run_local_server()
else:
appflow.run_console()
authed_creds = appflow.credentials
self.client = secretmanager.SecretManagerServiceClient(credentials=authed_creds)
When I create an instance of my SecretManagerAdmin class, a browser fires up and asks a user to login to google and confirm permissions for the app, and returns a second set of credentials that are then used to instantiate a Secretmanager client, which the application uses.
Please help. I am trying to search for a specific user in Foursquare but for some reason I got Missing credentials error 401.
user_id = '484542633' # user ID with most agree counts and complete profile
url = 'https://api.foursquare.com/v2/users/{}?client_id={}&client_secret={}&v={}'.format(user_id, CLIENT_ID, CLIENT_SECRET, VERSION) # define URL
# send GET request
results = requests.get(url).json()
user_data = results['response']['user']
# display features associated with user
user_data.keys()
As documentation states, this endpoint is meant to be accessed on behalf of the user:
This endpoint requires user authentication.
User calls require a valid OAuth access token in the query string of each request instead of the Client ID and Secret (&oauth_token=XXXX).
For more information about this authentication method and how to obtain an access token, see the Authentication docs.
This means v2/users can only be accessed by your app, after a user (which can be you, using your own Foursquare account) goes through the OAuth login flow and grants necessary permissions. OAuth doesn't mean your app "logs in" as the user, rather that the user has given you permission to do something on their behalf.
To learn more about OAuth you can watch this talk: https://www.youtube.com/watch?v=996OiexHze0
To read more about Foursquare's API visit their documentation site: https://developer.foursquare.com/docs/api/
We are super excited about App Engine's support for Google Cloud Endpoints.
That said we don't use OAuth2 yet and usually authenticate users with username/password
so we can support customers that don't have Google accounts.
We want to migrate our API over to Google Cloud Endpoints because of all the benefits we then get for free (API Console, Client Libraries, robustness, …) but our main question is …
How to add custom authentication to cloud endpoints where we previously check for a valid user session + CSRF token in our existing API.
Is there an elegant way to do this without adding stuff like session information and CSRF tokens to the protoRPC messages?
I'm using webapp2 Authentication system for my entire application. So I tried to reuse this for Google Cloud Authentication and I get it!
webapp2_extras.auth uses webapp2_extras.sessions to store auth information. And it this session could be stored in 3 different formats: securecookie, datastore or memcache.
Securecookie is the default format and which I'm using. I consider it secure enough as webapp2 auth system is used for a lot of GAE application running in production enviroment.
So I decode this securecookie and reuse it from GAE Endpoints. I don't know if this could generate some secure problem (I hope not) but maybe #bossylobster could say if it is ok looking at security side.
My Api:
import Cookie
import logging
import endpoints
import os
from google.appengine.ext import ndb
from protorpc import remote
import time
from webapp2_extras.sessions import SessionDict
from web.frankcrm_api_messages import IdContactMsg, FullContactMsg, ContactList, SimpleResponseMsg
from web.models import Contact, User
from webapp2_extras import sessions, securecookie, auth
import config
__author__ = 'Douglas S. Correa'
TOKEN_CONFIG = {
'token_max_age': 86400 * 7 * 3,
'token_new_age': 86400,
'token_cache_age': 3600,
}
SESSION_ATTRIBUTES = ['user_id', 'remember',
'token', 'token_ts', 'cache_ts']
SESSION_SECRET_KEY = '9C3155EFEEB9D9A66A22EDC16AEDA'
#endpoints.api(name='frank', version='v1',
description='FrankCRM API')
class FrankApi(remote.Service):
user = None
token = None
#classmethod
def get_user_from_cookie(cls):
serializer = securecookie.SecureCookieSerializer(SESSION_SECRET_KEY)
cookie_string = os.environ.get('HTTP_COOKIE')
cookie = Cookie.SimpleCookie()
cookie.load(cookie_string)
session = cookie['session'].value
session_name = cookie['session_name'].value
session_name_data = serializer.deserialize('session_name', session_name)
session_dict = SessionDict(cls, data=session_name_data, new=False)
if session_dict:
session_final = dict(zip(SESSION_ATTRIBUTES, session_dict.get('_user')))
_user, _token = cls.validate_token(session_final.get('user_id'), session_final.get('token'),
token_ts=session_final.get('token_ts'))
cls.user = _user
cls.token = _token
#classmethod
def user_to_dict(cls, user):
"""Returns a dictionary based on a user object.
Extra attributes to be retrieved must be set in this module's
configuration.
:param user:
User object: an instance the custom user model.
:returns:
A dictionary with user data.
"""
if not user:
return None
user_dict = dict((a, getattr(user, a)) for a in [])
user_dict['user_id'] = user.get_id()
return user_dict
#classmethod
def get_user_by_auth_token(cls, user_id, token):
"""Returns a user dict based on user_id and auth token.
:param user_id:
User id.
:param token:
Authentication token.
:returns:
A tuple ``(user_dict, token_timestamp)``. Both values can be None.
The token timestamp will be None if the user is invalid or it
is valid but the token requires renewal.
"""
user, ts = User.get_by_auth_token(user_id, token)
return cls.user_to_dict(user), ts
#classmethod
def validate_token(cls, user_id, token, token_ts=None):
"""Validates a token.
Tokens are random strings used to authenticate temporarily. They are
used to validate sessions or service requests.
:param user_id:
User id.
:param token:
Token to be checked.
:param token_ts:
Optional token timestamp used to pre-validate the token age.
:returns:
A tuple ``(user_dict, token)``.
"""
now = int(time.time())
delete = token_ts and ((now - token_ts) > TOKEN_CONFIG['token_max_age'])
create = False
if not delete:
# Try to fetch the user.
user, ts = cls.get_user_by_auth_token(user_id, token)
if user:
# Now validate the real timestamp.
delete = (now - ts) > TOKEN_CONFIG['token_max_age']
create = (now - ts) > TOKEN_CONFIG['token_new_age']
if delete or create or not user:
if delete or create:
# Delete token from db.
User.delete_auth_token(user_id, token)
if delete:
user = None
token = None
return user, token
#endpoints.method(IdContactMsg, ContactList,
path='contact/list', http_method='GET',
name='contact.list')
def list_contacts(self, request):
self.get_user_from_cookie()
if not self.user:
raise endpoints.UnauthorizedException('Invalid token.')
model_list = Contact.query().fetch(20)
contact_list = []
for contact in model_list:
contact_list.append(contact.to_full_contact_message())
return ContactList(contact_list=contact_list)
#endpoints.method(FullContactMsg, IdContactMsg,
path='contact/add', http_method='POST',
name='contact.add')
def add_contact(self, request):
self.get_user_from_cookie()
if not self.user:
raise endpoints.UnauthorizedException('Invalid token.')
new_contact = Contact.put_from_message(request)
logging.info(new_contact.key.id())
return IdContactMsg(id=new_contact.key.id())
#endpoints.method(FullContactMsg, IdContactMsg,
path='contact/update', http_method='POST',
name='contact.update')
def update_contact(self, request):
self.get_user_from_cookie()
if not self.user:
raise endpoints.UnauthorizedException('Invalid token.')
new_contact = Contact.put_from_message(request)
logging.info(new_contact.key.id())
return IdContactMsg(id=new_contact.key.id())
#endpoints.method(IdContactMsg, SimpleResponseMsg,
path='contact/delete', http_method='POST',
name='contact.delete')
def delete_contact(self, request):
self.get_user_from_cookie()
if not self.user:
raise endpoints.UnauthorizedException('Invalid token.')
if request.id:
contact_to_delete_key = ndb.Key(Contact, request.id)
if contact_to_delete_key.get():
contact_to_delete_key.delete()
return SimpleResponseMsg(success=True)
return SimpleResponseMsg(success=False)
APPLICATION = endpoints.api_server([FrankApi],
restricted=False)
I wrote a custom python authentication library called Authtopus that may be of interest to anyone looking for a solution to this problem: https://github.com/rggibson/Authtopus
Authtopus supports basic username and password registrations and logins, as well as social logins via Facebook or Google (more social providers could probably be added without too much hassle too). User accounts are merged according to verified email addresses, so if a user first registers by username and password, then later uses a social login, and the verified email addresses of the accounts match up, then no separate User account is created.
From my understanding Google Cloud Endpoints provides a way to implement a (RESTful?) API and to generate a mobile client library. Authentication in this case would be OAuth2. OAuth2 provides different 'flows', some of which support mobile clients.
In the case of authentication using a principal and credentials (username and password) this doesn't seem like a good fit. I honestly think you would be better off by using OAuth2.
Implementing a custom OAuth2 flow to support your case is an approach that could work but is very error prone.
I haven't worked with OAuth2 yet but maybe an 'API key' can be created for a user so they can both use the front-end and the back-end through the use of mobile clients.
you can used jwt for authentication. Solutions here
I did not coded it yet, but it imagined next way:
When server receives login request it look up username/password in datastore. In case user not found server responds with some error object that contains appropriate message like "User doesn't exist" or like. In case found it stored in FIFO kind of collection (cache) with limited size like 100 (or 1000 or 10000).
On successful login request server returns to client sessionid like ";LKJLK345345LKJLKJSDF53KL". Can be Base64 encoded username:password.
Client stores it in Cookie named "authString" or "sessionid" (or something less eloquent) with 30 min (any) expiration.
With each request after login client sends Autorization header that it takes from cookie. Each time cookie taken, it renewed -- so it never expires while user active.
On server side we will have AuthFilter that will check presence of Authorization header in each request (exclude login, signup, reset_password). If no such header found, filter returns response to client with status code 401 (client shows login screen to user). If header found filter first checks presence of user in the cache, after in datastore and if user found -- does nothing (request handled by appropriate method), not found -- 401.
Above architecture allows to keep server stateless but still have auto disconnecting sessions.