Develop Reference

Reading Outlook Calendars [The request is not valid for the application's 'userAudience' configuration.]

Today I'm trying to do read calendar from outlook. I created a new app through Microsoft Azure then setted a secret key and added a api permissions. When I was trying to authenticate via simple script I caught an error
The request is not valid for the application's 'userAudience' configuration.
In order to use /common/ endpoint%2c the application must not be configured with 'Consumer' as the user audience.
The userAudience should be configured with 'All' to use /common/ endpoint
This is my script
from O365 import Account, MSGraphProtocol
CLIENT_ID = 'MY CLIENT ID'
SECRET_ID = 'MY SECRET ID'
credentials = (CLIENT_ID, SECRET_ID)
protocol = MSGraphProtocol()
scopes = ['Calendars.Read.Shared']
account = Account(credentials, protocol=protocol)
if account.authenticate(scopes=scopes):
print('Authenticated!')
Could you tell me a reason of this error and how should i fix it?

It looks like you trying to use the client_credentials flow but your firstly using a Delegate permission which isn't correct. So in your Application registration you need to make sure you have assigned the Application permission for Calendars eg
to use the Client_credentials flow you need to first find your tenantId (if you don't already know it) you can do this in python eg using requests (you need to replace yourdomain.com with the domain your using
requests.get('https://login.windows.net/yourdomain.com/v2.0/.well-known/openid-configuration').json()["token_endpoint"]
Then take the guid part of the response eg
'https://login.windows.net/1c3a18bf-da31-4f6c-xxxx-2c06c9cf5ae4/oauth2/v2.0/token'
Then your code should look like where
from O365 import Account
credentials = ('my_client_id', 'my_client_secret')
# the default protocol will be Microsoft Graph
account = Account(credentials, auth_flow_type='credentials', tenant_id='1c3a18bf-da31-4f6c-xxxx-2c06c9cf5ae4')
if account.authenticate():
print('Authenticated!')

How to use the AWS Python SDK while connecting via SSO credentials

I am attempting to create a python script to connect to and interact with my AWS account. I was reading up on it here https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html
and I see that it reads your credentials from ~/.aws/credentials (on a Linux machine). I however and not connecting with an IAM user but SSO user. Thus, the profile connection data I use is located at ~/.aws/sso/cache directory.
Inside that directory, I see two json files. One has the following keys:
startUrl
region
accessToken
expiresAt
the second has the following keys:
clientId
clientSecret
expiresAt
I don't see anywhere in the docs about how to tell it to use my SSO user.
Thus, when I try to run my script, I get error such as
botocore.exceptions.ClientError: An error occurred (AuthFailure) when calling the DescribeSecurityGroups operation: AWS was not able to validate the provided access credentials
even though I can run the same command fine from the command prompt.

This was fixed in boto3 1.14.
So given you have a profile like this in your ~/.aws/config:
[profile sso_profile]
sso_start_url = <sso-url>
sso_region = <sso-region>
sso_account_id = <account-id>
sso_role_name = <role>
region = <default region>
output = <default output (json or text)>
And then login with
$ aws sso login --profile sso_profile
You will be able to create a session:
import boto3
boto3.setup_default_session(profile_name='sso_profile')
client = boto3.client('<whatever service you want>')

So here's the long and hairy answer tested on boto3==1.21.39:
It's an eight-step process where:
register the client using sso-oidc.register_client
start the device authorization flow using sso-oidc.start_device_authorization
redirect the user to the sso login page using webbrowser.open
poll sso-oidc.create_token until the user completes the signin
list and present the account roles to the user using sso.list_account_roles
get role credentials using sso.get_role_credentials
create a new boto3 session with the session credentials from (6)
eat a cookie
Step 8 is really key and should not be overlooked as part of any successful authorization flow.
In the sample below the account_id should be the account id of the account you are trying to get credentials for. And the start_url should be the url that aws generates for you to start the sso flow (in the AWS SSO management console, under Settings).
from time import time, sleep
import webbrowser
from boto3.session import Session
session = Session()
account_id = '1234567890'
start_url = 'https://d-0987654321.awsapps.com/start'
region = 'us-east-1'
sso_oidc = session.client('sso-oidc')
client_creds = sso_oidc.register_client(
clientName='myapp',
clientType='public',
)
device_authorization = sso_oidc.start_device_authorization(
clientId=client_creds['clientId'],
clientSecret=client_creds['clientSecret'],
startUrl=start_url,
)
url = device_authorization['verificationUriComplete']
device_code = device_authorization['deviceCode']
expires_in = device_authorization['expiresIn']
interval = device_authorization['interval']
webbrowser.open(url, autoraise=True)
for n in range(1, expires_in // interval + 1):
sleep(interval)
try:
token = sso_oidc.create_token(
grantType='urn:ietf:params:oauth:grant-type:device_code',
deviceCode=device_code,
clientId=client_creds['clientId'],
clientSecret=client_creds['clientSecret'],
)
break
except sso_oidc.exceptions.AuthorizationPendingException:
pass
access_token = token['accessToken']
sso = session.client('sso')
account_roles = sso.list_account_roles(
accessToken=access_token,
accountId=account_id,
)
roles = account_roles['roleList']
# simplifying here for illustrative purposes
role = roles[0]
role_creds = sso.get_role_credentials(
roleName=role['roleName'],
accountId=account_id,
accessToken=access_token,
)
session = Session(
region_name=region,
aws_access_key_id=role_creds['accessKeyId'],
aws_secret_access_key=role_creds['secretAccessKey'],
aws_session_token=role_creds['sessionToken'],
)

Your current .aws/sso/cache folder structure looks like this:
$ ls
botocore-client-XXXXXXXX.json cXXXXXXXXXXXXXXXXXXX.json
The 2 json files contain 3 different parameters that are useful.
botocore-client-XXXXXXXX.json -> clientId and clientSecret
cXXXXXXXXXXXXXXXXXXX.json -> accessToken
Using the access token in cXXXXXXXXXXXXXXXXXXX.json you can call get-role-credentials. The output from this command can be used to create a new session.
Your Python file should look something like this:
import json
import os
import boto3
dir = os.path.expanduser('~/.aws/sso/cache')
json_files = [pos_json for pos_json in os.listdir(dir) if pos_json.endswith('.json')]
for json_file in json_files :
path = dir + '/' + json_file
with open(path) as file :
data = json.load(file)
if 'accessToken' in data:
accessToken = data['accessToken']
client = boto3.client('sso',region_name='us-east-1')
response = client.get_role_credentials(
roleName='string',
accountId='string',
accessToken=accessToken
)
session = boto3.Session(aws_access_key_id=response['roleCredentials']['accessKeyId'], aws_secret_access_key=response['roleCredentials']['secretAccessKey'], aws_session_token=response['roleCredentials']['sessionToken'], region_name='us-east-1')

A well-formed boto3-based script should transparently authenticate based on profile name. It is not recommended to handle the cached files or keys or tokens yourself, since the official code methods might change in the future. To see the state of your profile(s), run aws configure list --examples:
$ aws configure list --profile=sso
Name Value Type Location
---- ----- ---- --------
profile sso manual --profile
The SSO session associated with this profile has expired or is otherwise invalid.
To refresh this SSO session run aws sso login with the corresponding profile.
$ aws configure list --profile=old
Name Value Type Location
---- ----- ---- --------
profile old manual --profile
access_key ****************3DSx shared-credentials-file
secret_key ****************sX64 shared-credentials-file
region us-west-1 env ['AWS_REGION', 'AWS_DEFAULT_REGION']

What works for me is the following:
import boto 3
session = boto3.Session(profile_name="sso_profile_name")
session.resource("whatever")
using boto3==1.20.18.
This would work if you had previously configured SSO for aws ie. aws configure sso.
Interestingly enough, I don't have to go through this if I use ipython, I just aws sso login beforehand and then call boto3.Session().
I am trying to figure out whether there is something wrong with my approach - I fully agree with what was said above with respect to transparency and although it is a working solution, I am not in love with it.
EDIT: there was something wrong and here is how I fixed it:
run aws configure sso (as above);
install aws-vault - it basically replaces aws sso login --profile <profile-name>;
run aws-vault exec <profile-name> to create a sub-shell with AWS credentials exported to environment variables.
Doing so, it is possible to run any boto3 command both interactively (eg. iPython) and from a script, as in my case. Therefore, the snippet above simply becomes:
import boto 3
session = boto3.Session()
session.resource("whatever")
Here for further details on AWS vault.

Google Domains API "HttpAccessTokenRefreshError: unauthorized_client: Client is unauthorized to retrieve access tokens using this method."

Using the Google Domains API
I am getting this error "HttpAccessTokenRefreshError: unauthorized_client: Client is unauthorized to retrieve access tokens using this method."
And yet I seem to be associating the right files with the right Service account, with Delegation on:
Furthermore, as shown here, that client does have the permissions I need:
So I am stumped and would appreciate help.
My code is below, based on https://developers.google.com/+/domains/quickstart/python
# Update SERVICE_ACCOUNT_JSON_FILE_PATH with the file path to the private key
# file downloaded from the developer console.
SERVICE_ACCOUNT_JSON_FILE_PATH = 'gsuite1-7b1dc64d67fd.json'
#SERVICE_ACCOUNT_JSON_FILE_PATH = 'file.txt'
# Update USER_EMAIL with the email address of the user within your domain that
# you would like to act on behalf of.
USER_EMAIL = 'jschull#e-nable.org'
# plus.me and plus.stream.write are the scopes required to perform the tasks in
# this quickstart. For a full list of available scopes and their uses, please
# see the documentation.
SCOPES = ['https://www.googleapis.com/auth/plus.me',
'https://www.googleapis.com/auth/plus.stream.write']
def authenticate():
"""Build and return a Plus service object authorized with the service accounts
that act on behalf of the given user.
Returns:
Plus service object.
"""
print( 'Authenticate the domain for %s' % USER_EMAIL)
# Make service account credentials
credentials = ServiceAccountCredentials.from_json_keyfile_name(SERVICE_ACCOUNT_JSON_FILE_PATH, scopes=SCOPES)
# Setting the sub field with USER_EMAIL allows you to make API calls using the
# special keyword 'me' in place of a user id for that user.
delegate_credentials = credentials.create_delegated(USER_EMAIL)
http = httplib2.Http()
http = delegate_credentials.authorize(http)
# Create and return the Plus service object
return build('plusDomains', 'v1', http=http)

DialogFlow detectIntentText

I am using DF with Python API and here is the code and I can't use the detect intent text.
If I use the second line I get the next error:
google.api_core.exceptions.PermissionDenied: 403 IAM permission 'dialogflow.sessions.detectIntent' on 'projects/newagent/agent' denied.
If I use the first one:
google.api_core.exceptions.InvalidArgument: 400 Resource name 'projects/newagent/agent/environments/draft/users//agent/sessions/5276b6d4-a0b6-4e91-84d3-16512d1f3299' does not match 'projects//agent/environments//users//sessions/'.
I have enabled billing on Google Cloud and the user has Owner privileges. What is going wrong?
def detect_intent_texts(project_id, session_id, texts, language_code):
session_client = dialogflow_v2.SessionsClient()
#----------------------------------------------------------Lines that I talk about in the question---------------------------------------------------------------------------------------------------
#session = session_client.session_path(project_id, session_id)
session = "projects/newagent/agent/environments/draft/users/<user id>/sessions/6344a857-9de5-406c-ba0f-c71b7b3ffdba"
#----------------------------------------------------------Lines that I talk about in the question---------------------------------------------------------------------------------------------------
for text in texts:
text_input = dialogflow_v2.types.TextInput(text=text, language_code=language_code)
query_input = dialogflow_v2.types.QueryInput(text=text_input)
response = session_client.detect_intent(session=session, query_input=query_input)
detect_intent_texts("newagent/agent/environments/draft/users/<User Number>",str(uuid.uuid4()),"Que tal?","es-ES")

The Session ID should have the format projects/<Project ID>/agent/sessions/<Session ID> (being <Project ID> the ID of the GCP project where your agent is located and <Session ID> the ID you use for your ongoing session), as can be seen in this documentation page.
In your code I see that you are calling the detect_intent_texts() function like:
project_id = "newagent/agent/environments/draft/users/<User Number>"
session_id = str(uuid.uuid4())
texts = "Que tal?"
language_code = "es-ES"
I see two main errors here:
The Project ID has the wrong format, it should be the ID of your GCP project, which usually has a format like my-first-project or similar, and slashes / are not supported, so you are using a wrong Project ID.
The text should be a Python list of strings, like ["hello"] and not just "hello".
Just as an example, the following minimal code provides the result below:
import dialogflow
def detect_intent_texts(project_id, session_id, texts, language_code):
session_client = dialogflow.SessionsClient()
session = session_client.session_path(project_id, session_id)
print('Session path: {}\n'.format(session))
for text in texts:
text_input = dialogflow.types.TextInput(text=text, language_code=language_code)
query_input = dialogflow.types.QueryInput(text=text_input)
response = session_client.detect_intent(session=session, query_input=query_input)
print('Fulfillment text: {}\n'.format(response.query_result.fulfillment_text))
detect_intent_texts("my-project","abcd",["hello"],"en-US")
Result:
user#my-project:~/dialogflow$ python detect_intent_minimal.py
Session path: projects/my-project/agent/sessions/abcd
Fulfillment text: Hi!
Therefore I suspect that changing the project_id to its correct value and the texts to a list should solve your issues.
EDIT:
I have been able to reproduce the issue that you are seeing with a 403 PermissionDenied message by using a Service Account without the required permissions.
In order to run intents in Dialogflow, you need to use a Service Account with one of the following roles:
Dialogflow API Admin and Dialogflow API Client can query for intents, and therefore, one of those is required in order to make the type of requests you are trying to do with your script.
I see you said that your user has owner privileges over the project. However, the issue may be that you are using a wrong service account. In order to set up authentication correctly, follow the steps detailed in the docs. In summary, you will have to create a Service Account with the right permissions, download its JSON key, and use it as an environment variable by running the command export GOOGLE_APPLICATION_CREDENTIALS="/path/to/your/key.json" wherever you are running the script.

When an identity calls a Google Cloud Platform API, Google Cloud Identity and Access Management(IAM) requires that the identity has the appropriate permissions to use the resource for which you have to create custom roles and then assign to service account. Then you will use that service account to call Google Cloud Platform API. Here you can search Dialogflow and see that DF is supported with custom roles only. That is why you have google.api_core.exceptions.PermissionDenied: 403 IAM permission 'dialogflow.sessions.detectIntent'. Do following steps:
Go to you project in Google Cloud Platform and then select roles as shown here:
Then click on Create Role, insert role name and related fields. Then click on Add Permissions and in the filter, search 'Service: Dialogflow'. Select the permissions you want and then click on create.
Then select this:
Click on Create Service Account and on Select Role option, type and search for the role you created on step 2 and save the account.
Do this: . A list of service account will show. Click on 'Create credentials' button.
Select the service account created in above steps and choose JSON. Then select Create. A JSON file be downloaded.
Add that file in your code as: os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'yourfilename.json'
Create an agent in Dialogflow console with google project for which you did all of the above steps. Enjoy :D

Microsoft Graph Authentication

I’m building an application in Python which can retrieve data from Azure AD. This data can require either Application permissions or Delegated permissions. I had a success retrieving data which needs only Application permissions. However, in order to retrieve data which needs delegated permission, I am trying to use OAuth2. Is it possible to get authenticated with Microsoft Graph using OAuth2 but not having the user sign in using the web page, but instead supplying the user credentials through the Python script itself?
Note: I want to use Microsoft Graph API (v1.0 and beta) and not Azure AD Graph API.

Assuming you have registered and configured (api permissions) your azure app and you have copied the apps "client id" and "client secret" you can define a class that holds your session.
The following code works for my app:
import json
import requests
from requests_oauthlib import OAuth2Session
from oauthlib.oauth2 import BackendApplicationClient
class SharepointSession(object):
""" Base Class without credentials, use real credentials in derived Classes
or instances
"""
api_uri = "https://graph.microsoft.com"
api_version = "v1.0"
scope = ["https://graph.microsoft.com/.default"]
directory_id = "" # - tenant id
token_url = "https://login.microsoftonline.com/{}/oauth2/v2.0/token"
sites_url = "{}/{}/sites".format(api_uri, api_version)
site = document_name = app_name = client_id = client_secret = ""
site_id = None
doc_id = None
def __init__(self):
""" """
def getTokenizedSession(self):
"""
OAuth2 to get access token
First set up a backend client, mind to set grant_type
build a OAuth2 Session with the client
get access token
Mind: python 3.x oauthlib requires scope params on more calls than py 2.x
"""
client = BackendApplicationClient(
client_id=self.client_id, scope=self.scope, grant_type="client_credentials")
session = OAuth2Session(client=client, scope=self.scope)
# fill access token
token = session.fetch_token(token_url=self.token_url.format(self.directory_id),
client_id=self.client_id,
scope=self.scope,
client_secret=self.client_secret)
self.session = session
self.token = token
return session, token
def getSiteId(self):
# get the site id
ae = "{}/myonline.sharepoint.com:/sites/{}:".format(
self.sites_url, self.site)
rt = self.session.get(ae)
response = json.loads(rt.text)
self.site_id = response.get("id")
return self.site_id
def someOtherMethod(self):
""" ... """
Now you can instantiate the session class with the credentials copied from your azure app registration i.e. "directory id" (same as tenant id), "client id" and "client secret"
like this:
mysp_session = SharepointSession()
mysp_session.directory_id = "XXXXXXXX-XXXX-YYYY-ZZZZ-XXXXXXXXX"
mysp_session.site = "MySitename"
mysp_session.document_name = "Testlist"
mysp_session.client_id = r"xxxxxxxxxxxxxxxxxxxxxxx"
mysp_session.client_secret = r"xxxxxxxxxxxxxxxxxxxxxxx"
# connect
session, token = mysp_session.getTokenizedSession()
# do your business logic
mysp_session.getSiteId()
....
mysp_session.someOtherMethod()
hope that helps

Yes, this is possible - but keep in mind that there are two Azure AD endpoints for application registration!
Try registering an application on the AAD V2.0 endpoint (apps.dev.microsoft.com), and then use a 'password' grant_type in your request.
Here are the steps you need:
Register your app on the AAD v2.0 endpoint, and generate a password (take
note of this)
Assign your required permissions (in this case, delegated)
As a callback URL I'd suggest using postman's Oauth2 callback URL first so you can debug what you're doing: https://www.getpostman.com/oauth2/callback
Important! If any of those permissions require admin consent, you MUST consent to them first to make the app available. This requires the admin user to sign in once.
Once consent has been given, here's a what your request needs to get a bearer token as a prototype:
POST https://login.microsoftonline.com/common/oauth2/token
Request body (application/x-www-form-urlencoded):
grant_type=[password]
username=[user email address]
password=[user password]
resource=https://graph.microsoft.com
client_id=[your newly registered application ID]
client_secret=[application password you noted during registration]
If successful, you'll get the bearer & refresh token as a response.
Hope this helps,
Ben

You need an Azure AD application to be able to authenticate with Graph API. A native Azure AD app and the flow and considerations described here work for ADAL.net. I use it to provision Microsoft Teams unattended: http://www.cloudidentity.com/blog/2014/07/08/using-adal-net-to-authenticate-users-via-usernamepassword/
I guess for Python you should have a look at ADAL for Python: https://github.com/introp-software/azure-activedirectory-library-for-python-old/blob/master/README.md
I think that the username/password auth is only possible with a native Azure AD app and not the web/web api types.