I have built a python application to access read only Power BI Rest API’s. I am automating the collection of tenant activity. However despite configuring my Azure App and using the service principal to generate an access token, the response I receive from the API request is one of an unauthorised response:
{"error": {"code": "PowerBINotAuthorizedException", "pbi.error": {"code":
"PowerBINotAuthorizedException", "parameters": {}, "details": [], "exceptionCulprit": 1}}}
I have found a number of similar issues posted online, however feel that I have done everything that is suggested but am still not able to get it working. I would appreciate any guidance.
The steps that I have taken are:
Configured an Azure App, adding the Application Permission for Power Bi Service-Tenant.Read.All
Screenshot of App Settings in Azure Portal
Requested my access token based upon the Client Credentials Flow using my app's client_ID and client_Secret as documented in the below link:
https://learn.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-client-creds-grant-flow
I successfully receive a token using the script below:
import requests
azureTenantID = "xxxxxxxxxxxxxxxxx"
azureClientId = "xxxxxxxxxxxxxxxxx"
azureClientSecret = "xxxxxxxxxxxxxxxxxx"
url = f"https://login.microsoftonline.com/{azureTenantID}/oauth2/v2.0/token"
payload = {
"grant_type": "client_credentials",
"client_id": azureClientId,
"client_secret": azureClientSecret,
"scope": "https://analysis.windows.net/powerbi/api/.default"
}
# Header HAS to be x-www-form-urlencoded for MS to accept it.
headers = {'Content-Type': 'application/x-www-form-urlencoded'}
# Return POST content as JSON.
r = requests.post(url, data=payload, headers=headers).json()
# Grab the access token.
response = r.get("access_token")
# Concatenate with Bearer string
access_token = "Bearer {r['access_token']}"
Configured my Power BI Tenant Settings to enable Service Principals to use API's.
Screenshot of Admin API Setting
Screenshot of Developer API Setting
Note that I added the Service Principal as a member of the Security Group for which both of these settings are enabled
Execute my Get request to the API
The followings script returns a good response when I take an access token from the Power BI REST API Documentation's 'Try it out' feature, but not when I generate the token as above.
import requests
# Initialise parameters.
url = "https://api.powerbi.com/v1.0/myorg/admin/groups?$top=1000&$expand=datasets,dataflows,reports,users,dashboards"
headers = {'Authorization': get_access_token2()}
# Get response.
response = requests.get(url, headers=headers)
response = response.json()
Any assistance would be appreciated !
I just went through this exact scenario that you described, and in the end we had to engage Microsoft support to solve it.
Although extremely counter intuitive, if the app that you create for your service principal authentication has any Power BI permissions assigned to it then the access token that is generated (when passed to Power BI REST Admin API) will return an error response that reports PowerBINotAuthorizedException.
To be even more specific, if the access token that you pass to the Power BI API has a roles key/value pair, then you will get a PowerBINotAuthorizedException.
In your case, the issue is easier because you have listed out what permissions you granted. You mentioned that you Configured an Azure App, adding the Application Permission for Power Bi Service-Tenant.Read.All. In order to resolve this issue, you will need to remove that permission.
For future readers, you can troubleshoot this by decoding your access token using a JWT token decoder like one found at jstoolset.com. If your app has permissions allocated to the scope that you have requested (https://analysis.windows.net/powerbi/api/.default is the typical Power BI scope that you request in your authorization) and you decode your JWT token then you will see a roles key/value pair. The presence of this roles is essentially the issue. It does not matter that the values there might match up to the Required Scope in the Power BI REST Admin API documentation. It was described to us as if there is a roles value in your access token then when the token is presented to the Power BI API the roles that are granted are attempted to be used, which ultimately results in a PowerBINotAuthorizedException because service principals are not allowed to use a certain role.
If you have an app that you have removed all permissions from, but still has a value coming through in your access token for the roles key/value pair, then I would suggest starting with a new app with no permissions allocated to it, and simply add the new app to the existing security group that you originally created. This is how we realized that this truly was the issue, and were then able to reconcile from there.
EDIT: Microsoft has now updated their API documentation on the relevant endpoints to reflect this information. For example, in Admin - Groups GetGroupUsersAsAdmin the Required Scope now reads:
Tenant.Read.All or Tenant.ReadWrite.All
Relevant only when authenticating via a standard delegated admin access token. Must not be present when authentication via a service principal is used.
Related
I have an MSAL app that creates authentication tokens for accessing various Microsoft APIs.
I provide the app specific scopes, and it creates a corresponding authentication token bearing those scopes. This app works perfectly fine for all types of endpoint I tried up
def _create_or_get_msal_app_client(
self, client_id: str, tenant_id: str | None = None, is_confidential_client: bool = False
) -> msal.ClientApplication:
"""
Create public or confidential msal app client to generate tokens
:param client_id: the client id (also known as application id)
:param tenant_id: the tenant id to use as authority, if not provided will use common authority
:return: the msal app
"""
if self._msal_app:
return self._msal_app
try:
authority = tenant_id if tenant_id else "common"
authority_url = f"https://login.microsoftonline.com/{authority}"
if is_confidential_client:
self._msal_app = msal.ConfidentialClientApplication(
client_id=[client_id], client_credential=[client_credential], authority=authority_url
)
else:
self._msal_app = msal.PublicClientApplication(client_id=client_id, authority=authority_url)
return self._msal_app
msal_app = self._create_or_get_msal_app_client(
client_id=[client_id], tenant_id=[tenant_id]
)
return msal_app.acquire_token_by_username_password(
username=[username], password=[password], scopes=[some scopes]
)
The tokens produced if inputted into jwt.io, will be marked as invalid, which is not a bad thing in itself, as noted by this qustion.
My problem is, when I try to call APIs with endpoints of type:
https://admin.powerplatform.microsoft.com/api/*
It almost seems like those kinds of endpoints has a different authorization system than the rest of the endpoints; For once, the token this EP uses in the UI I tool it from have a perfectly valid signature when trying to decode it in JTW.io, as opposed to the token issues by MSAL. But, this means that now I get in the response a 401 response when I try to use the MSAL-issues tokens, and the reason for the failed request, is, according to the response header resp.headers._store['www-authenticate'][1] is:
Bearer error="invalid_token", error_description="The signature is invalid"
This doesn't happen in any other Microsoft API I tried to call; for example in EPs of type https://graph.microsoft.com/v1.0/* the token produced by MSAL works perfectly fine.
The prime suspect in these types of authentication errors is the scopes asked. But no matter what scopes I ask, whether I ask for insufficient or sufficient or no scopes at all, I still get the same error.
Except what was suggested here to try to ask for the scope [client_id]/.defualt (where client id is the client id) but when I try to do that I get the error:
Bearer error="invalid_token", error_description="The audience \'[client_id]\' is invalid"
in the response headers.
I have another clue about what might be the problem in this forum, where the one asking the question mentioned that the EP is using OAuth. could it be that this is different from MS Graph in any way?
So my question is, how do I configure my MSAL app to work with https://admin.powerplatform.microsoft.com/api/*? Or alternatively, what EP could I use instead that does work with MSAL, and contains the same functionality as this one?
Note: looking at the headers in the request to get the tokens in the UI, I see they are using msal.js.browser, so this should be possible in theory. (by the way, the requested scope in the UI is [client_id]/.defualt openid profile offline_access) to the EP https://login.microsoftonline.com/common/oauth2/v2.0/token). When trying to decode the UI token in jwt.ms it says that the token is issued by AAD.
Example of a concrete EP I am trying to access: https://admin.powerplatform.microsoft.com/api/Environments/{env_name}/roleassignments/environmentadmin. The API is taken from the Power Platform Admin Center. More info about it here.
I'm creating an Airbyte connector for Docusign's E-signature Rest API.
Part of the process of implementing a connector is to write an authentication routine that extends the AuthBase class from requests.auth.
The issue is that Docusign does not support refresh tokens for JWT grants. According to the docusign documentation:
The access token granted by JWT Grant expires after one hour, and no refresh token is provided. After the token expires, you must generate a new JWT and exchange it for a new access token.
You can reuse most of the old assertion, just modifying the IAT and EXP values and updating the signature, then submit the updated JWT to get a new access token.
Generally, apps that use JWT should get a new access token about 15 minutes before their existing one expires.
However, all of the examples in the "backend application flow" from this part of the requests documentation (which links to this page in the requests-authlib docs) only seem to allow an Auth2 workflow that includes a refresh token.
How can I work around this to make it so that, each time a refresh token expires, a new request is made (with updated IAT EXP, and signature)?
Refresh tokens are a feature of the OAuth Authorization Code grant flow.
The Authorization Code grant flow requires a human to authenticate themself. The result is an 8 hour access token and a 30 day refresh token.
To obtain a new access token, either:
Ask the human to authenticate again
Or the app can use the refresh token to obtain a new access token. This can be done autonomously by the app, without bothering the human.
For the JWT grant flow, there is no human and no refresh token. Instead, the app simply re-runs the JWT grant flow and receive a new 1 hour access token.
When you re-do the JWT flow, create a new JWT (updated IAT, EXP, etc). Sign it with your private key, and send it to DocuSign to obtain a new access token.
The JWT operation is cheap enough to do once per hour per impersonated user. But you must cache the access token and not re-do the JWT grant flow for each API call...
Python authentication libraries
Most authentication libraries for most languages focus on the Authorization Code grant flow since that is the most commonly used OAuth flow.
But as you've pointed out, you're using the JWT flow. This means that you cannot use these libraries. Instead, you will need to roll your own. Good news is that it isn't too hard. Here's my pseudo code:
Send_an_API_request(url, request_details, etc):
access_token = Get_access_token(user_id);
api_results = send_api_request(access_token, url, request_details, etc)
return api_results;
Get_access_token(user_id):
(access_token, expire_time) = database_lookup(user_id);
# if access_token is about to expire or we don't have one,
# create a new access_token and store it
if (
((current_time + 10minutes) > expire_time)
or
(access_token is null)
):
# Make a new JWT request
jwt = make_jwt(user_id);
signed_jwt = sign(jwt, private_key)
(access_token, expire_sec) = send_jwt_request(signed_jwt)
database_store (user_id, access_token, current_time + expire_sec)
return access_token
Added
Re:
[I need to] extend the AuthBase class from requests.auth
If the app's architecture requires you to extend the AuthBase class, then you will need to implement the JWT grant flow within the AuthBase class.
If the AuthBase class doesn't give you access to the data you need for the JWT grant flow, then a hack is to stuff the needed data into an available attribute such as the "refresh token."
Here's my problem:
I have a 365 Family OneDrive subscription with 3 members, my account being the admin.
I am trying to build a python application to read/extract the content of the files I have on this onedrive space based on specific criterias. I want to build it as a command line application, running locally on my PC. I am aware some tools may exist for this but I'd like to code my own solution.
After going through tons of different documentation, I ended up doing the following
Registered my application on the Azure portal
Granted some permission on the Microsoft Graph API (User.read, Files.Read and Files.ReadAll)
Created a secret
Grabbed the sample code provided by Microsoft
Replaces some variables with my Client_Id and Secret
Ran the code
The code returns an access token but the authorization requests fails with 401 - Unauthorized: Access is denied due to invalid credentials.
Here's the Python code I'm using.
import msal
config = {
"authority": "https://login.microsoftonline.com/consumers",
"client_id": "<my client ID>",
"scope": ["https://graph.microsoft.com/.default"],
"secret": "<My secret stuff>",
"endpoint": "https://graph.microsoft.com/v1.0/users"
}
# Create a preferably long-lived app instance which maintains a token cache.
app = msal.ConfidentialClientApplication(
config["client_id"], authority=config["authority"],
client_credential=config["secret"],
)
result = None
result = app.acquire_token_silent(config["scope"], account=None)
if not result:
result = app.acquire_token_for_client(scopes=config["scope"])
if "access_token" in result:
# Calling graph using the access token
graph_data = requests.get( # Use token to call downstream service
config["endpoint"],
headers={'Authorization': 'Bearer ' + result['access_token']}, ).json()
print("Graph API call result: ")
print(json.dumps(graph_data, indent=2))
else:
print(result.get("error"))
print(result.get("error_description"))
print(result.get("correlation_id")) # You may need this when reporting a bug
According to the error message, I'm obviously missing something in the authorization process but can't tell what. I'm not even sure about the Authority and Endpoints I should use. My account being a personal one, I have no tenant.
Do I need to set-up / configure some URI somewhere?
Any help would be welcome.
Thank you in advance.
In your client app you need to store the token that you are getting from the MSAL. and then send the token with an authorized request.
For OneDrive, download the OneDrive for python. You can see the different option for Authentication.
The reason you are getting an access token, ID token, and a refresh token is because of the flow you're using. My suggestion is to review the flows for a better understanding of how the authentication process works and what will be returned accordingly. You can use this MSAL library for python.
I created 2 applications in my Azure directory, 1 for my API Server and one for my API client. I am using the Python ADAL Library and can successfully obtain a token using the following code:
tenant_id = "abc123-abc123-abc123"
context = adal.AuthenticationContext('https://login.microsoftonline.com/' + tenant_id)
token = context.acquire_token_with_username_password(
'https://myapiserver.azurewebsites.net/',
'myuser',
'mypassword',
'my_apiclient_client_id'
)
I then try to send a request to my API app using the following method but keep getting 'unauthorized':
at = token['accessToken']
id_token = "Bearer {0}".format(at)
response = requests.get('https://myapiserver.azurewebsites.net/', headers={"Authorization": id_token})
I am able to successfully login using myuser/mypass from the loginurl. I have also given the client app access to the server app in Azure AD.
Although the question was posted a long time ago, I'll try to provide an answer. I stumbled across the question because we had the exact same problem here. We could successfully obtain a token with the adal library but then we were not able to access the resource I obtained the token for.
To make things worse, we sat up a simple console app in .Net, used the exact same parameters, and it was working. We could also copy the token obtained through the .Net app and use it in our Python request and it worked (this one is kind of obvious, but made us confident that the problem was not related to how I assemble the request).
The source of the problem was in the end in the oauth2_client of the adal python package. When I compared the actual HTTP requests sent by the .Net and the python app, a subtle difference was that the python app sent a POST request explicitly asking for api-version=1.0.
POST https://login.microsoftonline.com/common//oauth2/token?api-version=1.0
Once I changed the following line in oauth2_client.py in the adal library, I could access my resource.
Changed
return urlparse('{}?{}'.format(self._token_endpoint, urlencode(parameters)))
in the method _create_token_url, to
return urlparse(self._token_endpoint)
We are working on a pull request to patch the library in github.
For the current release of Azure Python SDK, it support authentication with a service principal. It does not support authentication using an ADAL library yet. Maybe it will in future releases.
See https://azure-sdk-for-python.readthedocs.io/en/latest/resourcemanagement.html#authentication for details.
See also Azure Active Directory Authentication Libraries for the platforms ADAL is available on.
#Derek,
Could you set your Issue URL on Azure Portal? If I set the wrong Issue URL, I could get the same error with you. It seems that your code is right.
Base on my experience, you need add your application into Azure AD and get a client ID.(I am sure you have done this.) And then you can get the tenant ID and input into Issue URL textbox on Azure portal.
NOTE:
On old portal(manage.windowsazure.com),in the bottom command bar, click View Endpoints, and then copy the Federation Metadata Document URL and download that document or navigate to it in a browser.
Within the root EntityDescriptor element, there should be an entityID attribute of the form https://sts.windows.net/ followed by a GUID specific to your tenant (called a "tenant ID"). Copy this value - it will serve as your Issuer URL. You will configure your application to use this later.
My demo is as following:
import adal
import requests
TenantURL='https://login.microsoftonline.com/*******'
context = adal.AuthenticationContext(TenantURL)
RESOURCE = 'http://wi****.azurewebsites.net'
ClientID='****'
ClientSect='7****'
token_response = context.acquire_token_with_client_credentials(
RESOURCE,
ClientID,
ClientSect
)
access_token = token_response.get('accessToken')
print(access_token)
id_token = "Bearer {0}".format(access_token)
response = requests.get(RESOURCE, headers={"Authorization": id_token})
print(response)
Please try to modified it. Any updates, please let me know.
I am trying to fetch captions from YouTube video using YouTube Data API (v3)
https://developers.google.com/youtube/v3/guides/implementation/captions
So, first I tried to retrieve a captions list using this url:
https://www.googleapis.com/youtube/v3/captions?part=snippet&videoId=KK9bwTlAvgo&key={My API KEY}
I could retrieve the caption id that I'd like to download (jEDP-pmNCIqoB8QGlXWQf4Rh3faalD_l) from the above link.
Then, I followed this instruction to download the caption:
https://developers.google.com/youtube/v3/docs/captions/download
However, even though I input the caption id and my api key correctly, it shows "Login Required" error.
I suppose I need OAuth authentication, but what I am trying to do is not related to my users's account, but simply downloading public caption data automatically.
My question is: Is there any way to process OAuth authentication just once to get an access token of my own YouTube account and then reuse it whenever I need it in my application?
I can't speak to the permissions needed for the captions API in particular, but in general, yes, you can OAuth to your app once using your own account and use the access and refresh tokens to make subsequent OAuth'd requests to the API. You can find the details of generating tokens here:
https://developers.google.com/youtube/v3/guides/auth/server-side-web-apps#Obtaining_Access_Tokens
To perform the steps manually (fortunately, you only need to do this once):
If access has already been granted for an app, it needs to be removed so that new auth credentials can be established. Go to https://security.google.com/settings/security/permissions (while logged into your account) and remove access to the app. If the client ID or secret change (or you need to create one), find them at https://console.developers.google.com under API Manager.
To grant access and receive a temporary code, enter this URL in a browser:
https://accounts.google.com/o/oauth2/auth?
client_id=<client_id>&
redirect_uri=http://www.google.com&
scope=https://www.googleapis.com/auth/youtube.force-ssl&
response_type=code&
access_type=offline&
approval_prompt=force
Follow the prompt to grant access to the app.
This will redirect to google.com with a code parameter (e.g.,
https://www.google.com/?code=4/ux5gNj-_mIu4DOD_gNZdjX9EtOFf&gws_rd=ssl#). Save the code.
Send a POST request (e.g., via Postman Chrome plugin) to https://accounts.google.com/o/oauth2/token with the following in the request body:
code=<code>&
client_id=<client_id>&
client_secret=<client_secret>&
redirect_uri=http://www.google.com&
grant_type=authorization_code
The response will contain both an access token and refresh token. Save both, but particularly the refresh token (because the access token will expire in 1 hour).
You can then use the access token to send an OAuth'd request manually, following one of the options here, essentially:
curl -H "Authorization: Bearer ACCESS_TOKEN" https://www.googleapis.com/youtube/v3/captions/<id>
or
curl https://www.googleapis.com/youtube/v3/captions/<id>?access_token=ACCESS_TOKEN
(When I tried the second option for captions, however, I got the message: "The OAuth token was received in the query string, which this API forbids for response formats other than JSON or XML. If possible, try sending the OAuth token in the Authorization header instead.")
You can also use the refresh token in your code to create the credential needed when building your YouTube object. In Java, this looks like the following:
String clientId = <your client ID>
String clientSecret = <your client secret>
String refreshToken = <refresh token>
HttpTransport transport = new NetHttpTransport();
JsonFactory jsonFactory = new JacksonFactory();
GoogleCredential credential = new GoogleCredential.Builder()
.setTransport(transport)
.setJsonFactory(jsonFactory)
.setClientSecrets(clientId, clientSecret)
.build()
.setRefreshToken(refreshToken);
try {
credential.refreshToken();
} catch (IOException e) {
e.printStackTrace();
}
youtube = new YouTube.Builder(transport, jsonFactory, credential).build();
I imagine you can do something similar in Python with the API Client Libraries, although I haven't tried Python.