I'm trying to get a simple script (python) to work against the google calendar API and don't seem to be able to get the authentication to work properly. Its for a basic raspberry pi based thing I'm trying to make at home, and I need to just grab all the events from my google calendar.
I started with the code examples on the website, but because my script is totally server side, There was no way I could get the auth challenge step to work properly (it tried with lynx, but didn't work)
I then found examples with a service account - which seemed like the right thing as it didn't need the challenge flow. Furthermore, getting the creds_with_subject, seemed like the right approach to actually get a token related to the google account that owns the calendar. I think I got it all strung together, but then I don't actually get anything listed (as a start, I was using the calendar.list() endpoint to list the calendars out). The authentication step seemed to work (no errors) - but the ensuing list was empty. I'm guessing because my service account token wasn't associated/allowed to access my google account calendar. I've enabled the calendar API in the project to which the service account belongs.
Many of the guides I found seem to mention GSuite, which its not clear if I need
So summary questions:
For this type of usage - is the service account the right approach?
If so, do I need to associate this service account with my google account (that actually owns the calendar)?
It looks like the simple API key work doesn't work any more, but thought I'd check in case there is a simpler way?
do I need GSuite?
from __future__ import print_function
import datetime
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google.oauth2 import service_account
SCOPES = ['https://www.googleapis.com/auth/calendar']
def main():
"""Shows basic usage of the Google Calendar API.
Prints the start and name of the next 10 events on the user's calendar.
"""
creds = service_account.Credentials.from_service_account_file("credentials.json", scopes=SCOPES);
creds_with_subjects = creds.with_subject("<my real account>#gmail.com");
print(creds);
print(creds_with_subjects);
service = build('calendar', 'v3', credentials=creds_with_subjects)
print(creds);
print("Getting Calendar list");
page_token = None
while True:
calendar_list = service.calendarList().list(pageToken=page_token).execute()
print(calendar_list);
page_token = calendar_list.get('nextPageToken')
if not page_token:
break
if __name__ == '__main__':
main()
Creating a service account is not enough. You also have to perform domain wide delegation and impersonate your user.
The main purpose of granting domain-wide authority to a service account is for these accounts to be able to access data on behalf of a user in your domain as otherwise the service account acts like just another account and it is trying to access its own data from Calendar, hence the empty response you are receiving.
To impersonate your user, you will have to follow the steps below:
Create a Credentials object from the service account's credentials and the scopes your application needs access to:
from google.oauth2 import service_account
SCOPES = ['SCOPE1', 'SCOPE2', ...]
SERVICE_ACCOUNT_FILE = '/path/to/service.json'
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
Delegate domain-wide authority:
delegated_credentials = credentials.with_subject('YOURADDRESS#DOMAIN.COM')
However, in order to do this, you need to have a Google Workspace domain.
Reference
Using OAuth 2.0 for Server to Server Applications;
Authorizing Requests to the Google Calendar API.
Related
I get the next code used to pull youtube information form the youtube analytics api (not youtube data API V3)
import os
import google_auth_oauthlib.flow
import googleapiclient.discovery
import googleapiclient.errors
scopes = ["https://www.googleapis.com/auth/youtube.readonly"]
def main():
# Disable OAuthlib's HTTPS verification when running locally.
# *DO NOT* leave this option enabled in production.
os.environ["OAUTHLIB_INSECURE_TRANSPORT"] = "1"
api_service_name = "youtubeAnalytics"
api_version = "v2"
client_secrets_file = "YOUR_CLIENT_SECRET_FILE.json"
# Get credentials and create an API client
flow = google_auth_oauthlib.flow.InstalledAppFlow.from_client_secrets_file(
client_secrets_file, scopes)
credentials = flow.run_console()
youtube_analytics = googleapiclient.discovery.build(
api_service_name, api_version, credentials=credentials)
request = youtube_analytics.reports().query(
dimensions="video",
endDate="2014-06-30",
ids="channel==MINE",
maxResults=10,
metrics="estimatedMinutesWatched,views,likes,subscribersGained",
sort="-estimatedMinutesWatched",
startDate="2014-05-01"
)
response = request.execute()
print(response)
I get the client secret and the code runs fine, but i'm running the code on cloud(deepnote specifically) so at some point the code requires to enter manually the token, is there a way to avoid this, or to pull the token in someway? because I don´t see possible to retrive the token when running on cloud this code.
Thanks in advance.
The OAuth flow is designed to be interactive. It is usually used in cases where you (an app/web developer) want a user to click a link to log in with their own account and on their own. This usually requires a redirect, or a popup window, or an input field (like in Colab or Jupyter) where they do this kind of interaction.
The right way to do this kind of automatic requests is via a different kind of API, or using a different kind of authentication. In many cases of GCP APIs you'd use service account, but this particular API does not support service account authentication.
If you really want to use OAuth client secret flow, you can do it in Deepnote, when you run the flow.run_console() command, you'll get an input where you have to enter your approved token by your (user) google account. However, for this to work, you have to create a Desktop app client secret, and allow your user to access the testing mode, and click through the OAuth warnings. Or you have to publish the OAuth app, which requires approval from Google.
Here, I published an example Deepnote notebook which shows this is possible: https://deepnote.com/#jz/YouTube-Analytics-API-vuA9wCgGRKiGN9Mjaoi54Q
But this requires the manual interaction which you probably don't want. In other GCP APIs you could use your service account like this:
credentials = service_account.Credentials.from_service_account_file(service_account_file, scopes=scopes)
So if you really need to be automated, you need to use a different YouTube API.
You can save your credentials, which will automate your further process. use this thread link I have explained it.
I would like to use python to add/update external contacts that are shared to all users in a g suite domain and I'm having issue trying to figure out how. Looking at documents in https://developers.google.com/contacts/v3, looks like the process involves Google sign-in, however in the app I'm working on I don't want user to provide creds/sign in to Google, all authorization process will be done on server side.
This is my current code I'm using to get users' contact, and I'm wondering if it is possible to do anything similar to get/add/update external contacts using service account/p12 keys:
from googleapiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
credentials = ServiceAccountCredentials.from_p12_keyfile(
SERVICE_ACCOUNT_EMAIL,
P12_FILE_PATH,
KEY_SECRET,
scopes=[
'https://www.googleapis.com/auth/admin.directory.user'
]
)
credentials = credentials.create_delegated(SUPER_ADMIN_EMAIL)
admin = build('admin', 'directory_v1', credentials=credentials)
admin.users().get(userKey=user_email).execute()
Thanks
In a django application, I try to have RW access to a google calendar which I own myself.
Tried several ways with a service account & client secrets, but all resulting in authentication errors.
The API explorer works, but it requests consent in a popup window, which is obviously not acceptable.
Documentation on google OAuth2 describes several scenarios. Probably "web server application" applies here? It says:
"The authorization sequence begins when your application redirects a
browser to a Google URL; the URL includes query parameters that
indicate the type of access being requested. Google handles the user
authentication, session selection, and user consent. The result is an
authorization code, which the application can exchange for an access
token and a refresh token."
Again, we do not want a browser redirection, we want direct access to the google calendar.
So question is: how can a django server access a google calendar, on which I have full rights, view events and add events using a simple server stored key or similar mechanism?
With help of DalmTo and this great article, I got RW access to a google calendar working from python code. I will summarize the solution here.
Here are the steps:
First of all register for a google service account: Service accounts are pre-authorized accounts that avoid you need to get consent or refresh keys every time:
https://developers.google.com/identity/protocols/OAuth2ServiceAccount
(The part on G-suite can be ignored)
Download the service account credentials and store them safely. Your python code will need access to this file.
Go to your google calendar you want to get access to.
e.g. https://calendar.google.com/calendar/r/month
On the right side you see your calendars. Create an additional one for testing (since we'll write to it soon). Then point to this new calendar: click the 3 dots next to it and edit the sharing settings. Add the service account email address to the share under "share with specific people". (you can find the service account email address in the file downloaded previously under "client_email")
In the same screen, note the "calendar ID", you'll need it in below code.
Now you service account has the RW rights to the calendar.
Add at least one event to the calendar using the web UI (https://calendar.google.com/calendar/r/month) so we can read and change it from below code.
Then use following python code to read the calendar and change an event.
from google.oauth2 import service_account
import googleapiclient.discovery
SCOPES = ['https://www.googleapis.com/auth/calendar']
SERVICE_ACCOUNT_FILE = '<path to your service account file>'
CAL_ID = '<your calendar ID>'
credentials = service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE, scopes=SCOPES)
service = googleapiclient.discovery.build('calendar', 'v3', credentials=credentials)
events_result = service.events().list(calendarId=CAL_ID).execute()
events = events_result.get('items', [])
event_id = events[0]['id']
event = events[0]
service.events().update(calendarId=CAL_ID, eventId=event_id, body={"end":{"date":"2018-03-25"},"start":{"date":"2018-03-25"},"summary":"Kilroy was here"}).execute()
And there you go... read an event and updated the event.
I'm trying to list all Google Drive files Domain-wide, both users that still work here, and those that have moved on. With that, we can grep the output for certain terms (former customers) to delete customer-related files.
I believe I have a successful way to list all users using the Admin SDK Quickstart, since we have only about 200 total users (max is 500). I also have a way to list all files for a user using the Drive REST API's files.list() method. What I need to know is how to impersonate each user iteratively, in order to run the file listing script.
I have found the blurb .setServiceAccountUser(someone#domain.com) but I'm not really sure where to implement this, either in the service account authorization step, or in a separate middle-man script.
Have a look at https://github.com/pinoyyid/googleDriveTransferOwnership/blob/master/src/couk/cleverthinking/tof/Main.java
Specifically lines 285-299 which deal with generating a credential for an impersonated user.
GoogleCredential.Builder builder = new GoogleCredential.Builder()
.setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(serviceAccountEmailAddress)
.setServiceAccountPrivateKeyFromP12File(f)
.setServiceAccountScopes(Collections.singleton(SCOPE));
// if requested, impresonate a domain user
if (!"ServiceAccount".equals(impersonatedAccountEmailAddress)) {
builder.setServiceAccountUser(impersonatedAccountEmailAddress);
}
// build the Drive service
Drive service = new Drive.Builder(HTTP_TRANSPORT, JSON_FACTORY, null)
.setApplicationName("TOF")
.setHttpRequestInitializer(builder.build()).build();
This is Java, but should at least tell you what the steps are.
You need to implement the authorization flow for Service Accounts.
Once you create a service account in a GCP project (console.developers.google.com), enable DWD (domain-wide delegation), then authorize that service account in your G Suite admin console, that key can then be used to "impersonate" any account in the G Suite instance:
Create the credentials object from the json file
from oauth2client.service_account import ServiceAccountCredentials
scopes = ['https://www.googleapis.com/auth/gmail.readonly']
credentials = ServiceAccountCredentials.from_json_keyfile_name(
'/path/to/keyfile.json', scopes=scopes)
Create a credential that can impersonate user#example.org (could be any user in the domain though)
delegated_credentials = credentials.create_delegated('user#example.org')
Authorize the credential object (i.e. get an access_token)
from httplib2 import Http
http_auth = credentials.authorize(Http())
Call the Gmail API:
from apiclient import discovery
service = discovery.build('gmail', 'v1', http=http)
response = service.users().messages().list(userId='user#example.org').execute()
So I'm trying to create a new calendar, but i want to be able to specify what google account to create it in, assuming i have the credentials for said account, which i do. The code bellow creates it on the currently signed in user, or requires user interaction to allow access. Is there a way to specify an user and run the command on the background. I essentially just want to add a calendar to my account when the program runs, but i cant guarantee that my account will be logged in at the time.
I believe this was possible with the version 2 of the google api through ClientLogin, but i'm trying to use version 3.
import gflags
import httplib2
from apiclient.discovery import build
from oauth2client.file import Storage
from oauth2client.client import OAuth2WebServerFlow
from oauth2client.tools import run
FLAGS = gflags.FLAGS
FLOW = OAuth2WebServerFlow(
client_id='MY_CLIENT_KEY',
client_secret='MY_CLIENT_SECRET',
scope='https://www.googleapis.com/auth/calendar',
user_agent='MyApp/v1')
storage = Storage('calendar.dat')
credentials = storage.get()
if credentials is None or credentials.invalid == True:
credentials = run(FLOW, storage)
http = httplib2.Http()
http = credentials.authorize(http)
service = build(serviceName='calendar', version='v3', http=http)
calendar = {
'summary': 'Test3',
'timeZone': 'America/New_York'
}
created_calendar = service.calendars().insert(body=calendar).execute()
With V3, you'll need to use a Service Account in order to act as the user. The process is described in the Google Drive documentation, you just need to use Calendar API v3 scopes and references instead of Google Drive API.
Another option would be to store the OAuth2 refresh token and use it to grab valid access tokens even if the user is not logged in. See my reply to google Calendar api v3 Auth only for first time