Develop Reference

Using Google Admin to view Drive files Domain-wide

I'm trying to list all Google Drive files Domain-wide, both users that still work here, and those that have moved on. With that, we can grep the output for certain terms (former customers) to delete customer-related files.
I believe I have a successful way to list all users using the Admin SDK Quickstart, since we have only about 200 total users (max is 500). I also have a way to list all files for a user using the Drive REST API's files.list() method. What I need to know is how to impersonate each user iteratively, in order to run the file listing script.
I have found the blurb .setServiceAccountUser(someone#domain.com) but I'm not really sure where to implement this, either in the service account authorization step, or in a separate middle-man script.

Have a look at https://github.com/pinoyyid/googleDriveTransferOwnership/blob/master/src/couk/cleverthinking/tof/Main.java
Specifically lines 285-299 which deal with generating a credential for an impersonated user.
GoogleCredential.Builder builder = new GoogleCredential.Builder()
.setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(serviceAccountEmailAddress)
.setServiceAccountPrivateKeyFromP12File(f)
.setServiceAccountScopes(Collections.singleton(SCOPE));
// if requested, impresonate a domain user
if (!"ServiceAccount".equals(impersonatedAccountEmailAddress)) {
builder.setServiceAccountUser(impersonatedAccountEmailAddress);
}
// build the Drive service
Drive service = new Drive.Builder(HTTP_TRANSPORT, JSON_FACTORY, null)
.setApplicationName("TOF")
.setHttpRequestInitializer(builder.build()).build();
This is Java, but should at least tell you what the steps are.

You need to implement the authorization flow for Service Accounts.
Once you create a service account in a GCP project (console.developers.google.com), enable DWD (domain-wide delegation), then authorize that service account in your G Suite admin console, that key can then be used to "impersonate" any account in the G Suite instance:
Create the credentials object from the json file
from oauth2client.service_account import ServiceAccountCredentials
scopes = ['https://www.googleapis.com/auth/gmail.readonly']
credentials = ServiceAccountCredentials.from_json_keyfile_name(
'/path/to/keyfile.json', scopes=scopes)
Create a credential that can impersonate user#example.org (could be any user in the domain though)
delegated_credentials = credentials.create_delegated('user#example.org')
Authorize the credential object (i.e. get an access_token)
from httplib2 import Http
http_auth = credentials.authorize(Http())
Call the Gmail API:
from apiclient import discovery
service = discovery.build('gmail', 'v1', http=http)
response = service.users().messages().list(userId='user#example.org').execute()

Gmail API server to server application (Bad Request error)

I'm following this tutorial Using OAuth 2.0 for Server to Server Applications. I am trying to connect to the Gmail API using a service account.
The code I end up with looks like this:
from oauth2client.service_account import ServiceAccountCredentials
from httplib2 import Http
from apiclient.discovery import build
import json
scopes = ['https://www.googleapis.com/auth/gmail.readonly']
credentials = ServiceAccountCredentials.from_json_keyfile_name('******.json', scopes)
http_auth = credentials.authorize(Http())
service = build('gmail', 'v1', http=http_auth)
request = service.users().messages().list(userId='me')
response = request.execute()
print json.dumps(response, sort_keys=True, indent=2)
However, when I run this code, I get the following error:
googleapiclient.errors.HttpError:https://www.googleapis.com/gmail/v1/users/me/messages?alt=json returned "Bad Request">
Can someone help me understand where this error is coming from?

Think of a service account as a dummy user. It has a Google Drive account a google calendar account. What it doesn't to my knowlage is have a Gmail account.
Normally when you request data using a service account you have to grant the service account access to that data manually. In the case of google drive you can share a folder with the service account enabling it to access google drive. (you can also upload to its drive account but that's out of scope for this question)
There is no way to grant another user access to your Gmail account so there is no way to use a service account with a normal user Gmail account.
Note: If this is Not a normal user Gmail account and is in fact one based on google domains then you can grant the service account access to all the emails of the other users on the domain via the admin section.
Other wise you need to look into using Oauth2 to access gmail.

Gmail Api return Unauthorized client or scope in request

I have struggled to make this work but did half the job.
Actually I can only read messages from Gmail API, If I try to use the gmail.modify Scope I get an error:
HttpAccessTokenRefreshError: unauthorized_client: Unauthorized client or scope in request.
Here is my code:
# init gmail api
credentials_path = os.path.join(settings.PROJECT_DIR, 'settings/gmail_credential.json')
scopes = ['https://www.googleapis.com/auth/gmail.readonly',
'https://www.googleapis.com/auth/gmail.modify']
credentials = ServiceAccountCredentials.from_json_keyfile_name(credentials_path, scopes=scopes)
delegated_credentials = credentials.create_delegated('my_account#gmail.com')
http_auth = delegated_credentials.authorize(Http())
gmail = build('gmail', 'v1', http=http_auth)
In my service account:
I have set all possibles rôles to my service account "......iam.gserviceaccount.com"
I activated DWD: DwD: Google Apps Domain-wide Delegation is enabled.
I have read somewhere that I need a google work account to give permission to my service account to use gmail.Modify on my my_account#gmail email account. Seems very hard way to just modify a message in an email.
I don't know what to do next.

Based from this documentation, you need to use the client ID from your "Developers Console" as the Client Name in the "Manage API client access" when you're setting your API scopes. Google API does not work as expected with your personal account #gmail.com. You should have organization domain account in Google in format you#your_organisation_domain.
Check these threads:
Google API Python unauthorized_client: Unauthorized client or scope in request
Google API OAuth2, Service Account, "error" : "invalid_grant"

Working with Google Drive API on the google cloud

I have a google app engine site, and what I want to do, is get access to the files on my drive and publish them. Note that, my account owns both the drive and the app engine page.
I have tried looking at the google drive api, and the problem is that I don't know where to start with the following boilerplate code located in their documentation.
If you take a look at this function:
def get_credentials(authorization_code, state):
"""Retrieve credentials using the provided authorization code.
This function exchanges the authorization code for an access token and queries
the UserInfo API to retrieve the user's e-mail address.
If a refresh token has been retrieved along with an access token, it is stored
in the application database using the user's e-mail address as key.
If no refresh token has been retrieved, the function checks in the application
database for one and returns it if found or raises a NoRefreshTokenException
with the authorization URL to redirect the user to.
Args:
authorization_code: Authorization code to use to retrieve an access token.
state: State to set to the authorization URL in case of error.
Returns:
oauth2client.client.OAuth2Credentials instance containing an access and
refresh token.
Raises:
CodeExchangeError: Could not exchange the authorization code.
NoRefreshTokenException: No refresh token could be retrieved from the
available sources.
"""
email_address = ''
try:
credentials = exchange_code(authorization_code)
user_info = get_user_info(credentials)
email_address = user_info.get('email')
user_id = user_info.get('id')
if credentials.refresh_token is not None:
store_credentials(user_id, credentials)
return credentials
else:
credentials = get_stored_credentials(user_id)
if credentials and credentials.refresh_token is not None:
return credentials
except CodeExchangeException, error:
logging.error('An error occurred during code exchange.')
# Drive apps should try to retrieve the user and credentials for the current
# session.
# If none is available, redirect the user to the authorization URL.
error.authorization_url = get_authorization_url(email_address, state)
raise error
except NoUserIdException:
logging.error('No user ID could be retrieved.')
# No refresh token has been retrieved.
authorization_url = get_authorization_url(email_address, state)
raise NoRefreshTokenException(authorization_url)
This is a part of the boilerplate code. However, where am I supposed to get authorisation_code from?

I recently had to implement something similar, and it is quite tricky to find the relevant pieces of documentation.
This is what worked for me.
One-time setup to enable Google Drive for your Google App Engine project
Go to the Google APIs Console and select your App Engine project. If you don't see your App Engine project listed, you need to enable the cloud integration in the App Engine admin tool first (Administration > Application Settings > Cloud Integration > Create project)
In Google APIs Console, now go to Services and look for the "Drive API" in that long list. Turn it on.
Go to the API Access section on Google APIs Console, and find back the "Simple API Access" API Key. (see screenshot below)
Getting and installing the Python Drive API Client
Download the Python Drive API Client: https://developers.google.com/api-client-library/python/start/installation#appengine
Documentation on this Python API: https://google-api-client-libraries.appspot.com/documentation/drive/v2/python/latest/
Using the Python Drive API Client
To create the Drive service object, I use this:
import httplib2
def createDriveService():
"""Builds and returns a Drive service object authorized with the
application's service account.
Returns:
Drive service object.
"""
from oauth2client.appengine import AppAssertionCredentials
from apiclient.discovery import build
credentials = AppAssertionCredentials(scope='https://www.googleapis.com/auth/drive')
http = httplib2.Http()
http = credentials.authorize(http)
return build('drive', 'v2', http=http, developerKey=API_KEY)
You can then use this service object to execute Google Drive API calls, for example, to create a folder:
service = createDriveService()
res = {'title': foldername,
'mimeType': "application/vnd.google-apps.folder"}
service.files().insert(body=res).execute()
Caveats
I was not able to get the Drive API to work in unittesting, nor on the dev_appserver. I always get an error that my credentials are not valid. However, it works fine on the real app engine server.

Error calling Reports API v1

I'm getting an error while calling the Reports API:
<HttpError 403 when requesting https://www.googleapis.com/admin/reports/v1/usage/users/all/dates/2013-08-01?alt=json&maxResults=1 returned "Caller does not have access to the customers reporting data.">
Have anyone seen this error before? What am I missing?
I just can't see why this is showing or what I should be checking.
Regards.
EDIT:
Auth:
credentials = SignedJwtAssertionCredentials(
service_account_name='5163XXXXX#developer.gserviceaccount.com',
private_key=oauth2_private_key,
scope='https://www.googleapis.com/auth/admin.reports.usage.readonly')
# https://developers.google.com/api-client-library/python/guide/thread_safety
http = credentials.authorize(httplib2.Http())
service = apiclient.discovery.build('admin', 'reports_v1', http=http)
The actual call:
result = service.userUsageReport().get(
userKey='all',
date='2013-08-01',
maxResults=1).execute()
Other APIs just work fine with that service account.
'https://www.googleapis.com/auth/admin.reports.usage.readonly' has been properly added to OAuth2 domain config page.

Try:
credentials = SignedJwtAssertionCredentials(
service_account_name='5163XXXXX#developer.gserviceaccount.com',
private_key=oauth2_private_key,
scope='https://www.googleapis.com/auth/admin.reports.usage.readonly',
sub='super-admin#yourdomain.com')
when using Service Accounts with Admin SDK, you still need to act on behalf of an admin within the Google Apps instance.
Also, make sure that the Service Account Client ID has been granted rights to use the Reports API scopes in the Google Apps Control Panel. The Google Drive docs describe this process well, just sub in the Reports scopes instead.