Develop Reference

GMail Python API returns: 400 Precondition check failed when running on Cloud Composer with Workload Identity

I am trying to build an Airflow DAG (on Cloud Composer) that reads emails from Gmail, using the Google API Python client.
I would like to avoid the use of JSON files for Service Accounts, and therefore I am trying to take advantage of Workload Identity. Therefore, I performed the following steps:
Created a Service Account (my-service-account#my-project.iam.gserviceaccount.com) that will then be used to impersonate the Google mail my-email#my-domain.com
Granted Cloud Composer Service account the roles/iam.serviceAccountTokenCreator to the Google mail Service Account
Delegated domain-wide authority to the service account with the scopes 'https://www.googleapis.com/auth/gmail.readonly' such that the service account my-service-account#my-project.iam.gserviceaccount.com is authorized to access the emails of my-email#my-domain.com.
Now I'm trying to use the Google API Python client, in order to instantiate a Gmail service and use it to search the inbox of my-email#my-domain.com. Here's the code:
import google.auth
import google.auth.impersonated_credentials
SERVICE_ACCOUNT = 'my-service-account#my-project.iam.gserviceaccount.com'
SCOPES = ['https://www.googleapis.com/auth/gmail.readonly']
credentials, project_id = google.auth.default()
logging.info(f'Obtained application default credentials for project {project_id}.')
impersonated_credentials = google.auth.impersonated_credentials.Credentials(
source_credentials=credentials,
target_principal=SERVICE_ACCOUNT,
target_scopes=SCOPES,
)
logging.info(f'Obtained impersonated credentials for {SERVICE_ACCOUNT}')
service = build(
serviceName='gmail',
version='v1',
credentials=impersonated_credentials,
cache_discovery=False,
)
So initially, the code infers the Application Default Credentials (Cloud Composer), and then impersonates Cloud composer to act like the my-service-account#my-project.iam.gserviceaccount.com Service Account). Finally, it uses the returned credentials to build the gmail service.
When attempting to run a query:
results = service.users().messages().list(userId='me', q='from: someEmail#outlook.com').execute()
I get the following error:
[2022-11-14, 18:23:47 UTC] {standard_task_runner.py:93} ERROR - Failed to execute job 604219 for task test (<HttpError 400 when requesting https://gmail.googleapis.com/gmail/v1/users/me/messages?q=from%3A+someEmail%40outlook.com&alt=json returned "Precondition check failed.". Details: "Precondition check failed.">; 30352)
Any clue what I might be missing here? I've found a few similar questions but apparently they all use Service Account JSON files, which is clearly not the case here.

Google Calendar api permissions - Service account

I have a script which modifies my work supplied GCal. For authentication I use an access/refresh token (like this: https://developers.google.com/people/quickstart/python).
I want to run the script in Docker now. For authentication I have decided to use a service account.
I have created the service account, shared my calendar with it and accepted the calendar. In the Google Console where you create the service account, I set the permission to "owner".
When I try to run the script using the service account (not in Docker yet) it returns only a subset of attributes for each calendar event. I can see that accessRole = freeBusyReader.
How do I grant write access to this service account? I have tried:
rule = service.acl().get(calendarId="myId", ruleId='user:service#myApp-351310.iam.gserviceaccount.com').execute() # Get this from acl_items
rule["role"] = "owner"
service.acl().update(calendarId="myId", ruleId="user:service#myApp-351310.iam.gserviceaccount.com", body=rule).execute()
I have read about firmwide delegation and impersonation of users. I'm not sure if this is requred or not. Does anyone know how to do this?

The code to authenticate a service account is slightly different then the sample you were using for an installed application it is as follows.
credentials = ServiceAccountCredentials.from_json_keyfile_name(
key_file_location, scopes=scopes)
credentials = credentials.create_delegated(user_email)
This video shows How to create Google Oauth2 Service account credentials. just make sure to enable the google calendar api.
Remember service accounts are only supported via domain wide delegation to users on your google workspace domain. You cant use a standard google gmail user.
I recommend following the Delegate domain-wide authority to your service account sample it shows how to set up the delegation to a service account from your workspace domain. Just change out the section about admin sdk to that of google calendar as this is the api you are trying to connect to.
You add the user_Email being the user on your domain you want the service account to impersonate.

How to use a service account to authorize google sheets?

I am trying to open a private google sheet using python. The end goal here is to read that private sheet data into a json object. I have made sure to create a google cloud project, enable the API's, and service account. The service account email has been shared and added as an editor. I also created OAuth keys for a desktop application. This is required since the file is private.
I know I need to somehow request a token to use for access to the sheets API, but I am at a loss for how to create a request, and utilize the client_secret file generated from OAuth keys. I figured the googleAPI would have a function where you can pass this file directly, but I am lost in documentation.
Any insight would be appreciated!

All you need to do is supply the library with the location of the clientSecret.json file you should have downloaded from Google cloud console. This method should build the service for you and you can make the requests to the api. It will handle all the authorization.
from apiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
def get_service(api_name, api_version, scopes, key_file_location):
"""Get a service that communicates to a Google API.
Args:
api_name: The name of the api to connect to.
api_version: The api version to connect to.
scopes: A list auth scopes to authorize for the application.
key_file_location: The path to a valid service account JSON key file.
Returns:
A service that is connected to the specified API.
"""
credentials = ServiceAccountCredentials.from_json_keyfile_name(
key_file_location, scopes=scopes)
# Build the service object.
service = build(api_name, api_version, credentials=credentials)
return service
The best example I know of for service account authentication with python is the Google analytics quickstart If you have any issues altering it for google sheets let me know i can try and help.
Calling it should be something like this.
def main():
# Define the auth scopes to request.
scope = 'https://www.googleapis.com/auth/spreadsheets'
key_file_location = '<REPLACE_WITH_JSON_FILE>'
# Authenticate and construct service.
service = get_service(
api_name='sheets',
api_version='v4',
scopes=[scope],
key_file_location=key_file_location)
data = your_method_to_call_sheets(service)
How to create clientSecret.json remember to enable the google sheets api under libary

Could not establish an API connection in Google Cloud Platform

For retrieving monitoring metrics from my project, I used below Python code:
from google.cloud import monitoring_v3
from google.oauth2 import service_account
from googleapiclient import discovery
credentials = service_account.Credentials.from_service_account_file(
r'D:\GCP\credentials\blahblah-04e8fd0245b8.json')
service = discovery.build('compute', 'v1', credentials=credentials)
client = monitoring_v3.MetricServiceClient()
project_name = f"projects/{blahblah-300807}"
resource_descriptors = client.list_monitored_resource_descriptors(
name=project_name)
for descriptor in resource_descriptors:
print(descriptor.type)
I did everything well. I gave the file path for credentials information correctly, but I received this error message:
raise exceptions.DefaultCredentialsError(_HELP_MESSAGE)
google.auth.exceptions.DefaultCredentialsError: \
Could not automatically determine credentials. \
Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create \
credentials and re-run the application. \
For more information, please see \
https://cloud.google.com/docs/authentication/getting-started
I even checked that link and tried the alternative method, but still, it didn't work. How can I rectify this? Am I making a mistake?

You don't use the credential when you create the client
client = monitoring_v3.MetricServiceClient()
You can change it like this
client = monitoring_v3.MetricServiceClient(credentials=credentials)
Personally, I prefer to not explicitly provide the credential in the code, and I prefer to use the environment variable GOOGLE_APPLICATION_CREDENTIALS for this.
Create an environment variable in your OS with the name GOOGLE_APPLICATION_CREDENTIALS and the value that point to the service account key file D:\GCP\credentials\blahblah-04e8fd0245b8.json.
But, if it's on your own computer, you can even don't use service account key file (which is not really secure, I explain why in this article), you can use your own credential. For this, simply create an application default credential (ADC) like this gcloud auth application-default login

Using Google Admin to view Drive files Domain-wide

I'm trying to list all Google Drive files Domain-wide, both users that still work here, and those that have moved on. With that, we can grep the output for certain terms (former customers) to delete customer-related files.
I believe I have a successful way to list all users using the Admin SDK Quickstart, since we have only about 200 total users (max is 500). I also have a way to list all files for a user using the Drive REST API's files.list() method. What I need to know is how to impersonate each user iteratively, in order to run the file listing script.
I have found the blurb .setServiceAccountUser(someone#domain.com) but I'm not really sure where to implement this, either in the service account authorization step, or in a separate middle-man script.

Have a look at https://github.com/pinoyyid/googleDriveTransferOwnership/blob/master/src/couk/cleverthinking/tof/Main.java
Specifically lines 285-299 which deal with generating a credential for an impersonated user.
GoogleCredential.Builder builder = new GoogleCredential.Builder()
.setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(serviceAccountEmailAddress)
.setServiceAccountPrivateKeyFromP12File(f)
.setServiceAccountScopes(Collections.singleton(SCOPE));
// if requested, impresonate a domain user
if (!"ServiceAccount".equals(impersonatedAccountEmailAddress)) {
builder.setServiceAccountUser(impersonatedAccountEmailAddress);
}
// build the Drive service
Drive service = new Drive.Builder(HTTP_TRANSPORT, JSON_FACTORY, null)
.setApplicationName("TOF")
.setHttpRequestInitializer(builder.build()).build();
This is Java, but should at least tell you what the steps are.

You need to implement the authorization flow for Service Accounts.
Once you create a service account in a GCP project (console.developers.google.com), enable DWD (domain-wide delegation), then authorize that service account in your G Suite admin console, that key can then be used to "impersonate" any account in the G Suite instance:
Create the credentials object from the json file
from oauth2client.service_account import ServiceAccountCredentials
scopes = ['https://www.googleapis.com/auth/gmail.readonly']
credentials = ServiceAccountCredentials.from_json_keyfile_name(
'/path/to/keyfile.json', scopes=scopes)
Create a credential that can impersonate user#example.org (could be any user in the domain though)
delegated_credentials = credentials.create_delegated('user#example.org')
Authorize the credential object (i.e. get an access_token)
from httplib2 import Http
http_auth = credentials.authorize(Http())
Call the Gmail API:
from apiclient import discovery
service = discovery.build('gmail', 'v1', http=http)
response = service.users().messages().list(userId='user#example.org').execute()