In order to round-trip test mail sending code in our GCP backend I am sending an email to a GMail inbox and attempting to verify its arrival. The current mechanism for authentication to the GMail API is fairly standard, pasted from the GMail API documentation and embedded in a function:
def authenticate():
"""Authenticates to the Gmail API using data in credentials.json,
returning the service instance for use in queries etc."""
store = file.Storage('token.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets(CRED_FILE_PATH, SCOPES)
creds = tools.run_flow(flow, store)
service = build('gmail', 'v1', http=creds.authorize(Http()))
return service
CRED_FILE_PATH points to a downloaded credentials file for the service. The absence of the token.json file triggers its re-creation after an authentication interaction via a browser window, as does the token's expiry.
This is an integration test that must run headless (i.e. with no interaction whatsoever). When re-authentication is required the test currently raises an exception when the authentication flow starts to access sys.argv, which means it sees the arguments to pytest!
I've been trying to find out how to authenticate reliably using a mechanism that does not require user interaction (such as an API key). Nothing in the documentation or on Stackoverflow seems to answer this question.
A more recent effort uses the keyfile from a service account with GMail delegation to avoid the interactive Oauth2 flows.
def authenticate():
"""Authenticates to the Gmail API using data in g_suite_access.json,
returning the service instance for use in queries etc."""
main_cred = service_account.Credentials.from_service_account_file(
CRED_FILE_PATH, scopes=SCOPES)
# Establish limited credential to minimise any damage.
credentials = main_cred.with_subject(GMAIL_USER)
service = build('gmail', 'v1', credentials=credentials)
return service
On trying to use this service with
response = service.users().messages().list(userId='me',
q=f'subject:{subject}').execute()
I get:
google.auth.exceptions.RefreshError:
('unauthorized_client: Client is unauthorized to retrieve access tokens using this method.',
'{\n "error": "unauthorized_client",\n "error_description": "Client is unauthorized to retrieve access tokens using this method."\n}')
I get the feeling there's something fundamental I'm not understanding.
The service account needs to be authorized or it cant access the emails for the domain.
"Client is unauthorized to retrieve access tokens using this method"
Means that you have not authorized it properly; check Delegating domain-wide authority to the service account
Source: Client is unauthorized to retrieve access tokens using this method Gmail API C#
Related
I have built a django app, which it includes google Oauth2.0 login.
I want to get google calendar events of every users when they login with Oauth2.0 and I wrote the following code. I saved the access token into UserAuth table and fetched it, then used it to get google calendar.
def get_events_server(request):
user = User.objects.get(username=request.user)
creds = UserAuth.objects.get(user=user).google_id_token
credentials = AccessTokenCredentials(creds, "")
http = httplib2.Http()
http = credentials.authorize(http)
service = build('calendar', 'v3', http=http)
return service
When I run the code, the following error has happened.
HttpError at /calendar/
<HttpError 403 when requesting https://www.googleapis.com/calendar/v3/calendars/primary/events?timeMin=2021-10-28T04%3A33%3A08.956703Z&timeMax=2021-11-04T04%3A33%3A08.956712Z&singleEvents=true&timeZone=GMT%2B9%3A00&orderBy=startTime&alt=json returned "Request had insufficient authentication scopes.". Details: "[{'message': 'Insufficient Permission', 'domain': 'global', 'reason': 'insufficientPermissions'}]">
Is there a solution to skip this issue?
You are a little confused here lets start by looking at the difference between authentication and authorization.
Authentication or Open Id connect is signin your letting a user signin to their google account and you get an id token back and you are able to access their profile information because the user signed in. You are authentication that the user who is behind the machine owns the account. In your code see the id_token you are using Open id connect to authentication the user.
creds = UserAuth.objects.get(user=user).google_id_token
In order to access a users private data your application needs to be authorized to access that data. Authorization is defined by scopes, or the scope of access you need. In order to use the google calendar api you will need an access token with a scope that will give you permission to access the users google calendar events
You should have a look at the Python quickstart for google calendar it will show you how to use Oauth2 to have your application request authorization from the user to access their google calendar data.
def main():
"""Shows basic usage of the Google Calendar API.
Prints the start and name of the next 10 events on the user's calendar.
"""
creds = None
# The file token.json stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.json'):
creds = Credentials.from_authorized_user_file('token.json', SCOPES)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.json', 'w') as token:
token.write(creds.to_json())
service = build('calendar', 'v3', credentials=creds)
# Call the Calendar API
now = datetime.datetime.utcnow().isoformat() + 'Z' # 'Z' indicates UTC time
print('Getting the upcoming 10 events')
events_result = service.events().list(calendarId='primary', timeMin=now,
maxResults=10, singleEvents=True,
orderBy='startTime').execute()
events = events_result.get('items', [])
from comments
Your link consent screen request is returning an error. This is the redirect uri miss match errror and its one of the most common Errors you can get when you are setting up oauth2.
If you check the error it is telling you that there is an issue with this url redirect_uri: http://localhost:61668/ you are sending your request from that url. which means you need to go to google cloud console and add that redirect uri to your list of accepted redirect uris. Remember it must match exactly so the port number and trailing slash must be included.
These are your current redirect uris you need to add http://localhost:61668/
try setting
flow.run_local_server(port=0)
to
flow.run_local_server(port=8000)
then add
http://localhost:8000/
as your redirect uri.
If you don't know how this Video will show you how to fix it. Google OAuth2: How the fix redirect_uri_mismatch error. Part 2 server sided web applications.
Is it possible to access the Google Admin Reports API via server to server Service Account authorization?
I am try to make a server to server call to the Google Admin API, following the tutorial here.
When setting domain-wide delegation, I added these scopes: https://www.googleapis.com/auth/admin.reports.usage.readonly, https://www.googleapis.com/auth/admin.reports.audit.readonly, as defined here.
I try making the API call like this, using the relevant PyPI packages:
creds = service_account.Credentials.from_service_account_file('credentials.json', scopes=SCOPES)
with build('admin', 'reports_v1', credentials=creds) as service:
response = service.activities().list(userKey='all', applicationName='login', maxResults=10).execute()
Which results in the following error:
googleapiclient.errors.HttpError: <HttpError 401 when requesting https://admin.googleapis.com/admin/reports/v1/activity/users/all/applications/login?maxResults=10&alt=json returned "Access denied. You are not authorized to read activity records.". Details: "[{'message': 'Access denied. You are not authorized to read activity records.', 'domain': 'global', 'reason': 'authError', 'location': 'Authorization', 'locationType': 'header'}]">
When I make the API call using a different credentials method, such as Desktop Application, the call works as expected. However, the first time I run it, I have to interact with it via browser to approve/authenticate the call. Because this code will be running on a server without user interaction, that is not desirable behavior.
As a note, the docs for the Admin API say
Your application must use OAuth 2.0 to authorize requests. No other authorization protocols are supported.
Based on the documentation for sever to server calls, I believe service accounts still qualify as OAuth 2.0, but I could be wrong in that assumption.
In Google Workspace domains, the domain administrator can grant third-party applications with domain-wide access to its users' data — this is referred as domain-wide delegation of authority. Perform Google Workspace Domain-Wide Delegation of Authority
The documentation even has a python example.
from googleapiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
"""Email of the Service Account"""
SERVICE_ACCOUNT_EMAIL = '<some-id>#developer.gserviceaccount.com'
"""Path to the Service Account's Private Key file"""
SERVICE_ACCOUNT_PKCS12_FILE_PATH = '/path/to/<public_key_fingerprint>-privatekey.p12'
def create_reports_service(user_email):
"""Build and returns an Admin SDK Reports service object authorized with the service accounts
that act on behalf of the given user.
Args:
user_email: The email of the user. Needs permissions to access the Admin APIs.
Returns:
Admin SDK reports service object.
"""
credentials = ServiceAccountCredentials.from_p12_keyfile(
SERVICE_ACCOUNT_EMAIL,
SERVICE_ACCOUNT_PKCS12_FILE_PATH,
'notasecret',
scopes=['https://www.googleapis.com/auth/admin.reports.audit.readonly'])
credentials = credentials.create_delegated(user_email)
return build('admin', 'reports_v1', http=http)
Notice that user_email is the user you are acting on behalf of, not the service account email.
The error message Access denied. You are not authorized to read activity records. means that you have not properly set up delegation for the user. Contact your workspace admin and have them look into it.
I'm following this tutorial Using OAuth 2.0 for Server to Server Applications. I am trying to connect to the Gmail API using a service account.
The code I end up with looks like this:
from oauth2client.service_account import ServiceAccountCredentials
from httplib2 import Http
from apiclient.discovery import build
import json
scopes = ['https://www.googleapis.com/auth/gmail.readonly']
credentials = ServiceAccountCredentials.from_json_keyfile_name('******.json', scopes)
http_auth = credentials.authorize(Http())
service = build('gmail', 'v1', http=http_auth)
request = service.users().messages().list(userId='me')
response = request.execute()
print json.dumps(response, sort_keys=True, indent=2)
However, when I run this code, I get the following error:
googleapiclient.errors.HttpError:https://www.googleapis.com/gmail/v1/users/me/messages?alt=json returned "Bad Request">
Can someone help me understand where this error is coming from?
Think of a service account as a dummy user. It has a Google Drive account a google calendar account. What it doesn't to my knowlage is have a Gmail account.
Normally when you request data using a service account you have to grant the service account access to that data manually. In the case of google drive you can share a folder with the service account enabling it to access google drive. (you can also upload to its drive account but that's out of scope for this question)
There is no way to grant another user access to your Gmail account so there is no way to use a service account with a normal user Gmail account.
Note: If this is Not a normal user Gmail account and is in fact one based on google domains then you can grant the service account access to all the emails of the other users on the domain via the admin section.
Other wise you need to look into using Oauth2 to access gmail.
I have struggled to make this work but did half the job.
Actually I can only read messages from Gmail API, If I try to use the gmail.modify Scope I get an error:
HttpAccessTokenRefreshError: unauthorized_client: Unauthorized client or scope in request.
Here is my code:
# init gmail api
credentials_path = os.path.join(settings.PROJECT_DIR, 'settings/gmail_credential.json')
scopes = ['https://www.googleapis.com/auth/gmail.readonly',
'https://www.googleapis.com/auth/gmail.modify']
credentials = ServiceAccountCredentials.from_json_keyfile_name(credentials_path, scopes=scopes)
delegated_credentials = credentials.create_delegated('my_account#gmail.com')
http_auth = delegated_credentials.authorize(Http())
gmail = build('gmail', 'v1', http=http_auth)
In my service account:
I have set all possibles rôles to my service account "......iam.gserviceaccount.com"
I activated DWD: DwD: Google Apps Domain-wide Delegation is enabled.
I have read somewhere that I need a google work account to give permission to my service account to use gmail.Modify on my my_account#gmail email account. Seems very hard way to just modify a message in an email.
I don't know what to do next.
Based from this documentation, you need to use the client ID from your "Developers Console" as the Client Name in the "Manage API client access" when you're setting your API scopes. Google API does not work as expected with your personal account #gmail.com. You should have organization domain account in Google in format you#your_organisation_domain.
Check these threads:
Google API Python unauthorized_client: Unauthorized client or scope in request
Google API OAuth2, Service Account, "error" : "invalid_grant"
I have bunch of gmail.storage files containing a JSON blob with access info for users' accounts. I got these credential sets using PHP to prompt the user to authenticate/authorize my app. I'm able to call the Gmail API using these credentials. The JSON has the following fields:
{"access_token": xxxxxx,
"token_type":"Bearer",
"expires_in":3599,
"refresh_token":"xxxxxx",
"created":1421545394}
However, when I use the Python SDK for Gmail to authenticate a call using these credentials file like so:
credentials = storage.get()
It says that it needs the fields _module, _class and a few others. I'm able to get these when I use Python to fetch the information but not with PHP. Any idea how I can use the above code to get credentials without these extraneous fields?
Just call the refresh method before using the creds again.
storage = file.Storage("auth.dat")
credentials = storage.get()
http = httplib2.Http()
credentials.refresh(http) # <----------
http = credentials.authorize(http)
service = build('whatever_service', 'v2', http=http)