I am trying to get emails of all users of specific domain. I am able to get emails of user who created the service account (admin account), but during getting emails of other users of that domain I am facing HttpError 403 "Delegation denied for xxx#example.com"
I did the following steps:
Created a project, fetched a credentials.json file and using this file created a token.json file to authenticate requests,
Created service account and enabled domain wide delegation,
Collected the client id from that service account
and then delegate domain-wide authority to that service account using this guide link https://developers.google.com/identity/protocols/oauth2/service-account#delegatingauthority
Here is example code of my request to get emails data of other users (using python):
from googleapiclient.discovery import build
from oauth2client import file
from httplib2 import Http
TOKEN_FILEPATH = "/path-to-token-file/"
def get_auth():
store = file.Storage(TOKEN_FILEPATH)
creds = store.get()
service = build('gmail', 'v1', http=creds.authorize(Http()))
return service
def get_emails():
user = 'xxx#example.com'
service = get_auth()
response = service.users().messages().list(userId=user).execute()
Thanks!
Related
I have a script to export text from a GDrive file using an OAuth client, which works perfectly well -
import googleapiclient.discovery as google
from apiclient.http import MediaIoBaseDownload
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
import datetime, io, os, pickle
Scopes=" ".join(['https://www.googleapis.com/auth/drive.file',
'https://www.googleapis.com/auth/drive.metadata',
'https://www.googleapis.com/auth/drive.readonly'])
TokenFile="token.pickle"
def init_creds(clientfile,
scopes,
tokenfile=TokenFile):
token=None
if os.path.exists(tokenfile):
with open(tokenfile, 'rb') as f:
token=pickle.load(f)
if (not token or
not token.valid or
token.expiry < datetime.datetime.utcnow()):
if (token and
token.expired and
token.refresh_token):
token.refresh(Request())
else:
flow=InstalledAppFlow.from_client_secrets_file(clientfile, scopes)
token=flow.run_local_server(port=0)
with open(tokenfile, 'wb') as f:
pickle.dump(token, f)
return token
def export_text(id,
clientfile,
scopes=Scopes):
creds=init_creds(clientfile=clientfile,
scopes=scopes)
service=google.build('drive', 'v3', credentials=creds)
request=service.files().export_media(fileId=id,
mimeType='text/plain')
buf=io.BytesIO()
downloader, done = MediaIoBaseDownload(buf, request), False
while done is False:
status, done = downloader.next_chunk()
destfilename="tmp/%s.txt" % id
return buf.getvalue().decode("utf-8")
if __name__=='__main__':
print (export_text(id="#{redacted}"
clientfile="/path/to/oath/client.json"))
But it's a pain to have to go through the OAuth flow every time, and since it's only me using the script I want to simplify things and use a Service Account instead, following on from this post -
Google Drive API Python Service Account Example
My new Service Account script, doing exactly the same thing, is as follows -
import googleapiclient.discovery as google
from oauth2client.service_account import ServiceAccountCredentials
from apiclient.http import MediaIoBaseDownload
import io
Scopes=" ".join(['https://www.googleapis.com/auth/drive.file',
'https://www.googleapis.com/auth/drive.metadata',
'https://www.googleapis.com/auth/drive.readonly'])
def export_text(id,
clientfile,
scopes=Scopes):
creds=ServiceAccountCredentials.from_json_keyfile_name(clientfile,
scopes)
service=google.build('drive', 'v3', credentials=creds)
request=service.files().export_media(fileId=id,
mimeType='text/plain')
buf=io.BytesIO()
downloader, done = MediaIoBaseDownload(buf, request), False
while done is False:
status, done = downloader.next_chunk()
destfilename="tmp/%s.txt" % id
return buf.getvalue().decode("utf-8")
if __name__=='__main__':
print (export_text(id="#{redacted}",
clientfile="path/to/service/account.json"))
but when I run it for the same id, I get the following -
googleapiclient.errors.HttpError: <HttpError 404 when requesting https://www.googleapis.com/drive/v3/files/#{redacted}/export?mimeType=text%2Fplain&alt=media returned "File not found: #{redacted}.">
It feels like the Service Account script is passing the authentication step (ie Service Account creds are okay) but then failing when trying to fetch the file - weird as I can fetch it fine using the OAuth version :/
Any thoughts on what might be causing this 404 error in the Service Account version, given the OAuth client version clearly works for the same id?
Answer:
You need to share your file with the service account.
More Information:
As you would with any file, you need to give a user explicit permissions to be able to see it. As a service account is a separate entitiy to you, this goes for them as well.
Using the file sharing settings (you can just do this in the Drive UI by right-clicking the file and hitting Share), give the email address of the service account the correct permission (read/write). The email address of the service account is in the form:
service-account-name#project-id.iam.gserviceaccount.com
Before making your call do a File.list to see which files the service account has access to. Doing a file.get on a file that the service account doesn't have access to will result in a file not found error. Remember that the service account is not you, it has its own google drive account. Any files you want to access need to be uploaded to its account or shared with the service account.
If the file.list fails then it would suggest to me that there is something wrong with the authorization and you should ensure that the service account has access to client file maybe its that file it cant find.
Granting service account acccess
Create a directory on your personal google drive account. Take the service account email address, it can be found in the key file you downloaded it has a # in it. Then share that directory on your drive account with the service account like you would share with any other user.
Adding files to that directory may or may not give the service account access to them automatically permissions is a pain you may need to also share the file with the service account.
Remember to have the service account grant your personal account permissions to the file when it uploads it its going to be the owner.
I'm trying to list all Google Drive files Domain-wide, both users that still work here, and those that have moved on. With that, we can grep the output for certain terms (former customers) to delete customer-related files.
I believe I have a successful way to list all users using the Admin SDK Quickstart, since we have only about 200 total users (max is 500). I also have a way to list all files for a user using the Drive REST API's files.list() method. What I need to know is how to impersonate each user iteratively, in order to run the file listing script.
I have found the blurb .setServiceAccountUser(someone#domain.com) but I'm not really sure where to implement this, either in the service account authorization step, or in a separate middle-man script.
Have a look at https://github.com/pinoyyid/googleDriveTransferOwnership/blob/master/src/couk/cleverthinking/tof/Main.java
Specifically lines 285-299 which deal with generating a credential for an impersonated user.
GoogleCredential.Builder builder = new GoogleCredential.Builder()
.setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(serviceAccountEmailAddress)
.setServiceAccountPrivateKeyFromP12File(f)
.setServiceAccountScopes(Collections.singleton(SCOPE));
// if requested, impresonate a domain user
if (!"ServiceAccount".equals(impersonatedAccountEmailAddress)) {
builder.setServiceAccountUser(impersonatedAccountEmailAddress);
}
// build the Drive service
Drive service = new Drive.Builder(HTTP_TRANSPORT, JSON_FACTORY, null)
.setApplicationName("TOF")
.setHttpRequestInitializer(builder.build()).build();
This is Java, but should at least tell you what the steps are.
You need to implement the authorization flow for Service Accounts.
Once you create a service account in a GCP project (console.developers.google.com), enable DWD (domain-wide delegation), then authorize that service account in your G Suite admin console, that key can then be used to "impersonate" any account in the G Suite instance:
Create the credentials object from the json file
from oauth2client.service_account import ServiceAccountCredentials
scopes = ['https://www.googleapis.com/auth/gmail.readonly']
credentials = ServiceAccountCredentials.from_json_keyfile_name(
'/path/to/keyfile.json', scopes=scopes)
Create a credential that can impersonate user#example.org (could be any user in the domain though)
delegated_credentials = credentials.create_delegated('user#example.org')
Authorize the credential object (i.e. get an access_token)
from httplib2 import Http
http_auth = credentials.authorize(Http())
Call the Gmail API:
from apiclient import discovery
service = discovery.build('gmail', 'v1', http=http)
response = service.users().messages().list(userId='user#example.org').execute()
I'm following this tutorial Using OAuth 2.0 for Server to Server Applications. I am trying to connect to the Gmail API using a service account.
The code I end up with looks like this:
from oauth2client.service_account import ServiceAccountCredentials
from httplib2 import Http
from apiclient.discovery import build
import json
scopes = ['https://www.googleapis.com/auth/gmail.readonly']
credentials = ServiceAccountCredentials.from_json_keyfile_name('******.json', scopes)
http_auth = credentials.authorize(Http())
service = build('gmail', 'v1', http=http_auth)
request = service.users().messages().list(userId='me')
response = request.execute()
print json.dumps(response, sort_keys=True, indent=2)
However, when I run this code, I get the following error:
googleapiclient.errors.HttpError:https://www.googleapis.com/gmail/v1/users/me/messages?alt=json returned "Bad Request">
Can someone help me understand where this error is coming from?
Think of a service account as a dummy user. It has a Google Drive account a google calendar account. What it doesn't to my knowlage is have a Gmail account.
Normally when you request data using a service account you have to grant the service account access to that data manually. In the case of google drive you can share a folder with the service account enabling it to access google drive. (you can also upload to its drive account but that's out of scope for this question)
There is no way to grant another user access to your Gmail account so there is no way to use a service account with a normal user Gmail account.
Note: If this is Not a normal user Gmail account and is in fact one based on google domains then you can grant the service account access to all the emails of the other users on the domain via the admin section.
Other wise you need to look into using Oauth2 to access gmail.
I created a service account, instead of using the default one, under my google cloud project and I enabled dwd.
I then enabled Google Apps Marketplace API from the google cloud console.
Then I published a marketplace app as available only to users of my GSuite domain with api_console_project_id in manifest.json set to my project's id.
The status in the developer dashboard is
Published
GAM: Pending
I then had my domain admin install the app.
I believe I am now all set to use the appsmarket v2 Python client to verify license of my domain for the app now.
Below is how I am doing it
from httplib2 import Http
from oauth2client.service_account import ServiceAccountCredentials
from apiclient.discovery import build
path_to_key = 'path/to/my/service_account_key.json'
scopes = ['https://www.googleapis.com/auth/appsmarketplace.license']
APPLICATION_ID = 'xxxxxxxxxxxx'
domain = 'xxxxxx.com'
credentials = ServiceAccountCredentials.from_json_keyfile_name(
path_to_key, scopes)
authorized_http = credentials.authorize(Http())
appsmarket_api_client = build(
'appsmarket', 'v2', http=authorized_http)
appsmarket_license_info = appsmarket_api_client.customerLicense().get(
applicationId=APPLICATION_ID, customerId=domain).execute()
domain_is_licensed = (appsmarket_license_info['state'] == 'ACTIVE')
But my code breaks at
appsmarket_license_info = appsmarket_api_client.customerLicense().get(
applicationId=APPLICATION_ID, customerId=domain).execute()
with HttpError 403 when requesting https://www.googleapis.com/appsmarket/v2/customerLicense/xxxxxxxxxxxx/xxxxxx.com?alt=json returned "Not authorized to access the application ID
I am unable to debug why this is happening.
What I missed was that I didn't add the service account as a member of the project and give it edit access to the project. Now it's working.
I have tried to follow this tutorial to connect to Google Analytics API. I have followed the instructions, step by step. I have Python 2.7 installed on my computer. I have installed Google client library. When I run the program, i get the following error in the terminal:
Traceback (most recent call last):
File "HelloAnalytics.py", line 6, in <module>
from oauth2client.client import SignedJwtAssertionCredential
ImportError: cannot import name SignedJwtAssertionCredentials
The line 6 that it referring to is:
from oauth2client.client import SignedJwtAssertionCredentials
I am at a complete lost. I looked at others who have had the same error, here, here, and here, but the solutions did not work. I have some programming knowledge, but compared to many of you, I am a noob.
The full code is here:
"""A simple example of how to access the Google Analytics API."""
import argparse
from apiclient.discovery import build
from oauth2client.client import SignedJwtAssertionCredentials
import httplib2
from oauth2client import client
from oauth2client import file
from oauth2client import tools
def get_service(api_name, api_version, scope, key_file_location,
service_account_email):
"""Get a service that communicates to a Google API.
Args:
api_name: The name of the api to connect to.
api_version: The api version to connect to.
scope: A list auth scopes to authorize for the application.
key_file_location: The path to a valid service account p12 key file.
service_account_email: The service account email address.
Returns:
A service that is connected to the specified API.
"""
f = open(key_file_location, 'rb')
key = f.read()
f.close()
credentials = SignedJwtAssertionCredentials(service_account_email, key,
scope=scope)
http = credentials.authorize(httplib2.Http())
# Build the service object.
service = build(api_name, api_version, http=http)
return service
def get_first_profile_id(service):
# Use the Analytics service object to get the first profile id.
# Get a list of all Google Analytics accounts for this user
accounts = service.management().accounts().list().execute()
if accounts.get('items'):
# Get the first Google Analytics account.
account = accounts.get('items')[0].get('id')
# Get a list of all the properties for the first account.
properties = service.management().webproperties().list(
accountId=account).execute()
if properties.get('items'):
# Get the first property id.
property = properties.get('items')[0].get('id')
# Get a list of all views (profiles) for the first property.
profiles = service.management().profiles().list(
accountId=account,
webPropertyId=property).execute()
if profiles.get('items'):
# return the first view (profile) id.
return profiles.get('items')[0].get('id')
return None
def get_results(service, profile_id):
# Use the Analytics Service Object to query the Core Reporting API
# for the number of sessions within the past seven days.
return service.data().ga().get(
ids='ga:' + profile_id,
start_date='7daysAgo',
end_date='today',
metrics='ga:sessions').execute()
def print_results(results):
# Print data nicely for the user.
if results:
print 'View (Profile): %s' % results.get('profileInfo').get('profileName')
print 'Total Sessions: %s' % results.get('rows')[0][0]
else:
print 'No results found'
def main():
# Define the auth scopes to request.
scope = ['https://www.googleapis.com/auth/analytics.readonly']
# Use the developer console and replace the values with your
# service account email and relative location of your key file.
service_account_email = '<Replace with your service account email address.>'
key_file_location = '<Replace with /path/to/generated/client_secrets.p12>'
# Authenticate and construct service.
service = get_service('analytics', 'v3', scope, key_file_location,
service_account_email)
profile = get_first_profile_id(service)
print_results(get_results(service, profile))
if __name__ == '__main__':
main()
Any help or direction would be appreciated.
The source repository was recently updated, and the Hello Analytics Guides have since also been updated to make use of the new code:
from apiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
...
credentials = ServiceAccountCredentials.from_p12_keyfile(
service_account_email, key_file_location, scopes=scope)
Where service_account_email is the email address of the service account you get from the dev console, and the key_file_locaiton is /the/path/to/key.p12, and scopes are the scopes you need to grant to the service account.
Please remember to add the service account email address as an authorized user of the Google Analytics view (profile) to which you wish it to have access.