I have a script which modifies my work supplied GCal. For authentication I use an access/refresh token (like this: https://developers.google.com/people/quickstart/python).
I want to run the script in Docker now. For authentication I have decided to use a service account.
I have created the service account, shared my calendar with it and accepted the calendar. In the Google Console where you create the service account, I set the permission to "owner".
When I try to run the script using the service account (not in Docker yet) it returns only a subset of attributes for each calendar event. I can see that accessRole = freeBusyReader.
How do I grant write access to this service account? I have tried:
rule = service.acl().get(calendarId="myId", ruleId='user:service#myApp-351310.iam.gserviceaccount.com').execute() # Get this from acl_items
rule["role"] = "owner"
service.acl().update(calendarId="myId", ruleId="user:service#myApp-351310.iam.gserviceaccount.com", body=rule).execute()
I have read about firmwide delegation and impersonation of users. I'm not sure if this is requred or not. Does anyone know how to do this?
The code to authenticate a service account is slightly different then the sample you were using for an installed application it is as follows.
credentials = ServiceAccountCredentials.from_json_keyfile_name(
key_file_location, scopes=scopes)
credentials = credentials.create_delegated(user_email)
This video shows How to create Google Oauth2 Service account credentials. just make sure to enable the google calendar api.
Remember service accounts are only supported via domain wide delegation to users on your google workspace domain. You cant use a standard google gmail user.
I recommend following the Delegate domain-wide authority to your service account sample it shows how to set up the delegation to a service account from your workspace domain. Just change out the section about admin sdk to that of google calendar as this is the api you are trying to connect to.
You add the user_Email being the user on your domain you want the service account to impersonate.
Related
I am trying to build an Airflow DAG (on Cloud Composer) that reads emails from Gmail, using the Google API Python client.
I would like to avoid the use of JSON files for Service Accounts, and therefore I am trying to take advantage of Workload Identity. Therefore, I performed the following steps:
Created a Service Account (my-service-account#my-project.iam.gserviceaccount.com) that will then be used to impersonate the Google mail my-email#my-domain.com
Granted Cloud Composer Service account the roles/iam.serviceAccountTokenCreator to the Google mail Service Account
Delegated domain-wide authority to the service account with the scopes 'https://www.googleapis.com/auth/gmail.readonly' such that the service account my-service-account#my-project.iam.gserviceaccount.com is authorized to access the emails of my-email#my-domain.com.
Now I'm trying to use the Google API Python client, in order to instantiate a Gmail service and use it to search the inbox of my-email#my-domain.com. Here's the code:
import google.auth
import google.auth.impersonated_credentials
SERVICE_ACCOUNT = 'my-service-account#my-project.iam.gserviceaccount.com'
SCOPES = ['https://www.googleapis.com/auth/gmail.readonly']
credentials, project_id = google.auth.default()
logging.info(f'Obtained application default credentials for project {project_id}.')
impersonated_credentials = google.auth.impersonated_credentials.Credentials(
source_credentials=credentials,
target_principal=SERVICE_ACCOUNT,
target_scopes=SCOPES,
)
logging.info(f'Obtained impersonated credentials for {SERVICE_ACCOUNT}')
service = build(
serviceName='gmail',
version='v1',
credentials=impersonated_credentials,
cache_discovery=False,
)
So initially, the code infers the Application Default Credentials (Cloud Composer), and then impersonates Cloud composer to act like the my-service-account#my-project.iam.gserviceaccount.com Service Account). Finally, it uses the returned credentials to build the gmail service.
When attempting to run a query:
results = service.users().messages().list(userId='me', q='from: someEmail#outlook.com').execute()
I get the following error:
[2022-11-14, 18:23:47 UTC] {standard_task_runner.py:93} ERROR - Failed to execute job 604219 for task test (<HttpError 400 when requesting https://gmail.googleapis.com/gmail/v1/users/me/messages?q=from%3A+someEmail%40outlook.com&alt=json returned "Precondition check failed.". Details: "Precondition check failed.">; 30352)
Any clue what I might be missing here? I've found a few similar questions but apparently they all use Service Account JSON files, which is clearly not the case here.
Example code
from google.oauth2 import service_account
import pygsheets
creds = service_account.Credentials.from_service_account_file(
'my/path/to/credentials.json',
scopes=('https://www.googleapis.com/auth/spreadsheets', 'https://www.googleapis.com/auth/drive'),
subject='account#mydomain.com'
)
pg = pygsheets.authorize(custom_credentials=creds)
pg.open_by_url('https://docs.google.com/spreadsheets/d/my_spreadsheet_id/edit#gid=my_sheet_id')
Problematic behaviour
Fails on the last line with Client is unauthorized to retrieve access tokens using this method, or client not authorized for any of the scopes requested.
Expected behaviour
The last line provides an object for Google Sheets access.
Additional info
Domain-wide delegation is enabled for the service account, subject account is on the domain
Sheet is shared with the subject account
When I don't provide the subject and share the sheet with the service account directly, it works
Environment
python==3.6.9
pygsheets==2.0.3.1
google-auth==1.6.3
To consider when using domain-wide delegation
The domain-wide delegation is not enabled by default. To allow it you need to follow the steps described in the documentation.
Step: in the GCP console:
You need to activate the checkbox Enable G Suite Domain-wide Delegation for each service account you want to use for such purpose
To you use the service account to impersonate a user, you need to give the necessary permissions in the Admin console
In the Admin console:
Any scopes that the service account needs when impersonating a user have to be authorized in the admin console
For this go to Main menu menu> Security > API controls.
Add (if not already done) the service account of interest by its Client ID, provide it all the scopes it needs and authorize
You can modify the scopes anytime at a later stage if needed
I'm using a Google Service Account to upload videos using Resumable Method to Google Drive. The python code works well but I'm running into Google Service Account storage issue.
It seems like Google Service Account can only have 15 GB of storage. Even though I upload the video to a regular Google Drive folder, the video is still owned by the Service Account. Therefore, I tried to transfer the owner of the videos to a different account but it didn't work, the error is bad request. User message: \"You can't yet change the owner of this item. (We're working on it.)
Below is my python code that generate an access token from the service account and perform the Resumable Upload
credentials = ServiceAccountCredentials.from_json_keyfile_name(
'creds.json',
scopes='https://www.googleapis.com/auth/drive'
)
delegated_credentials = credentials.create_delegated('service_account_email')
access_token = delegated_credentials.get_access_token().access_token
filesize = os.path.getsize(file_location)
# Retrieve session for resumable upload.
headers1 = {"Authorization": "Bearer " + access_token, "Content-Type": "application/json"}
params = {
"name": file_name,
"mimeType": "video/mp4",
"parents": [folder_id]
}
r = requests.post(
"https://www.googleapis.com/upload/drive/v3/files?uploadType=resumable",
headers=headers1,
data=json.dumps(params)
)
location = r.headers['Location']
# Upload the file.
headers2 = {"Content-Range": "bytes 0-" + str(filesize - 1) + "/" + str(filesize)}
r = requests.put(
location,
headers=headers2,
data=open(file_location, 'rb')
)
Is there a workaround or increase the storage limit of the Google Service Accounts?
Any advice would be very appreciated. Thank you!
You want to use the Service Account to make a Resumable Upload to Drive.
You want the owner of the video not to be the Service Account, but a regular account which has enough Drive storage capacity.
If that's correct, then you can just have to delegate domain-wide authority to the Service Account, so that it can act on behalf of any user in the domain and, when uploading the file, impersonate the account you want to be the owner of the file.
Delegating domain-wide authority:
The process of granting domain-wide authority is explained here:
On the Service accounts page, select your Service Account, and while editing the SA, click SHOW DOMAIN-WIDE DELEGATION and, on the content that was just displayed, check the option Enable G Suite Domain-wide Delegation.
Once you've done this, go to the Admin console and then go to Main menu > Security > API Controls.
Select Manage Domain Wide Delegation in the Domain wide delegation pane, and click Add new.
Fill up the corresponding fields: (1) in Client ID, enter the SA's Client ID, which you can find both in the credentials JSON file and on the Service Account page, and (2) in OAuth scopes, add the scopes corresponding to the resources you want the SA to access on behalf of users in the domain. In this case, I guess that's just https://www.googleapis.com/auth/drive.
After clicking Authorize, you have conferred the Service Account the ability to access resources on behalf of any user in the domain.
Impersonating another user:
Now the Service Account impersonate any user in the domain, but you have to specify which user you want it to impersonate. In order to do that, you just have to do a small change in your code. Right now, you're setting the service_account_email when delegating credentials via create_delegated:
delegated_credentials = credentials.create_delegated('service_account_email')
That is, the Service Account is acting on behalf of the Service Account. If you didn't want to impersonate another account, there would be no real need for this line of code (it doesn't have any effect, since credentials and delegated_credentials both refer to the same account (the Service Account).
But since you want to use the Service Account to act on behalf of another account, you have to specify this other account's email address on this line:
delegated_credentials = credentials.create_delegated('user_account_email')
That's the only change you need to do in your code. If you have granted domain-wide delegation, the Service Account will act as if it was this other user. It will be like it was this other user who uploaded the file, so this user will be the owner of the file.
Note:
You are using a deprecated library (oauth2client). Since this is still working, there is no real need to do it now, but please consider changing your code to google-auth.
Reference:
Delegating domain-wide authority to the service account
I want to create an automatic deployment of GCP for clients.
In order to do that, I have opened a page for them to login with google, and then enabled the IAM API and the Service Usage API.
Then I have created a service account that I want to use from this point forward in order to enable other required APIs on demand and not all at once.
When I try to enable the cloudkms API, I get
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://serviceusage.googleapis.com/v1/projects/x-y-z/services/cloudkms.googleapis.com?alt=json returned "The caller does not have permission"
I tried using the service account credentials (google.auth.jwt.Credentials) that I have created from the response of creating the service account, and I have added all the required permissions. I don't want to grant the role owner to the service account, because I want the account to have as less permissions as possible.
When I try to get the status of cloudkms API using the user's permissions, it works.
I have seen some solutions addressing me needing to create credentials for the service account here : https://console.developers.google.com/apis/credentials but I really need to do this programatically as well.
My code:
credentials = jwt.Credentials.from_service_account_file(service_account_info['email'] + '.json', audience="https://www.googleapis.com/auth/cloud-platform")
# credentials = GoogleCredentials.get_application_default() - it works with this
service_usage = googleapiclient.discovery.build('serviceusage', 'v1', credentials=credentials)
service_usage.services().get(name="projects/<project_id>/services/cloudkms.googleapis.com").execute()
The error was mentioned above.
You need the Cloud IAM permission serviceusage.services.enable to enable services. Depending on what features your require, such as listing services, you need serviceusage.services.list.
Typically you add the role roles/serviceusage.serviceUsageAdmin which includes the following permissions:
serviceusage.services.get
serviceusage.services.list
serviceusage.services.enable
serviceusage.services.disable
Goto IAM
Edit user selected
Add new rol
Type Service Usage Admin
Save
In a django application, I try to have RW access to a google calendar which I own myself.
Tried several ways with a service account & client secrets, but all resulting in authentication errors.
The API explorer works, but it requests consent in a popup window, which is obviously not acceptable.
Documentation on google OAuth2 describes several scenarios. Probably "web server application" applies here? It says:
"The authorization sequence begins when your application redirects a
browser to a Google URL; the URL includes query parameters that
indicate the type of access being requested. Google handles the user
authentication, session selection, and user consent. The result is an
authorization code, which the application can exchange for an access
token and a refresh token."
Again, we do not want a browser redirection, we want direct access to the google calendar.
So question is: how can a django server access a google calendar, on which I have full rights, view events and add events using a simple server stored key or similar mechanism?
With help of DalmTo and this great article, I got RW access to a google calendar working from python code. I will summarize the solution here.
Here are the steps:
First of all register for a google service account: Service accounts are pre-authorized accounts that avoid you need to get consent or refresh keys every time:
https://developers.google.com/identity/protocols/OAuth2ServiceAccount
(The part on G-suite can be ignored)
Download the service account credentials and store them safely. Your python code will need access to this file.
Go to your google calendar you want to get access to.
e.g. https://calendar.google.com/calendar/r/month
On the right side you see your calendars. Create an additional one for testing (since we'll write to it soon). Then point to this new calendar: click the 3 dots next to it and edit the sharing settings. Add the service account email address to the share under "share with specific people". (you can find the service account email address in the file downloaded previously under "client_email")
In the same screen, note the "calendar ID", you'll need it in below code.
Now you service account has the RW rights to the calendar.
Add at least one event to the calendar using the web UI (https://calendar.google.com/calendar/r/month) so we can read and change it from below code.
Then use following python code to read the calendar and change an event.
from google.oauth2 import service_account
import googleapiclient.discovery
SCOPES = ['https://www.googleapis.com/auth/calendar']
SERVICE_ACCOUNT_FILE = '<path to your service account file>'
CAL_ID = '<your calendar ID>'
credentials = service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE, scopes=SCOPES)
service = googleapiclient.discovery.build('calendar', 'v3', credentials=credentials)
events_result = service.events().list(calendarId=CAL_ID).execute()
events = events_result.get('items', [])
event_id = events[0]['id']
event = events[0]
service.events().update(calendarId=CAL_ID, eventId=event_id, body={"end":{"date":"2018-03-25"},"start":{"date":"2018-03-25"},"summary":"Kilroy was here"}).execute()
And there you go... read an event and updated the event.