I'm trying to list all Google Drive files Domain-wide, both users that still work here, and those that have moved on. With that, we can grep the output for certain terms (former customers) to delete customer-related files.
I believe I have a successful way to list all users using the Admin SDK Quickstart, since we have only about 200 total users (max is 500). I also have a way to list all files for a user using the Drive REST API's files.list() method. What I need to know is how to impersonate each user iteratively, in order to run the file listing script.
I have found the blurb .setServiceAccountUser(someone#domain.com) but I'm not really sure where to implement this, either in the service account authorization step, or in a separate middle-man script.
Have a look at https://github.com/pinoyyid/googleDriveTransferOwnership/blob/master/src/couk/cleverthinking/tof/Main.java
Specifically lines 285-299 which deal with generating a credential for an impersonated user.
GoogleCredential.Builder builder = new GoogleCredential.Builder()
.setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(serviceAccountEmailAddress)
.setServiceAccountPrivateKeyFromP12File(f)
.setServiceAccountScopes(Collections.singleton(SCOPE));
// if requested, impresonate a domain user
if (!"ServiceAccount".equals(impersonatedAccountEmailAddress)) {
builder.setServiceAccountUser(impersonatedAccountEmailAddress);
}
// build the Drive service
Drive service = new Drive.Builder(HTTP_TRANSPORT, JSON_FACTORY, null)
.setApplicationName("TOF")
.setHttpRequestInitializer(builder.build()).build();
This is Java, but should at least tell you what the steps are.
You need to implement the authorization flow for Service Accounts.
Once you create a service account in a GCP project (console.developers.google.com), enable DWD (domain-wide delegation), then authorize that service account in your G Suite admin console, that key can then be used to "impersonate" any account in the G Suite instance:
Create the credentials object from the json file
from oauth2client.service_account import ServiceAccountCredentials
scopes = ['https://www.googleapis.com/auth/gmail.readonly']
credentials = ServiceAccountCredentials.from_json_keyfile_name(
'/path/to/keyfile.json', scopes=scopes)
Create a credential that can impersonate user#example.org (could be any user in the domain though)
delegated_credentials = credentials.create_delegated('user#example.org')
Authorize the credential object (i.e. get an access_token)
from httplib2 import Http
http_auth = credentials.authorize(Http())
Call the Gmail API:
from apiclient import discovery
service = discovery.build('gmail', 'v1', http=http)
response = service.users().messages().list(userId='user#example.org').execute()
Related
I am trying to open a private google sheet using python. The end goal here is to read that private sheet data into a json object. I have made sure to create a google cloud project, enable the API's, and service account. The service account email has been shared and added as an editor. I also created OAuth keys for a desktop application. This is required since the file is private.
I know I need to somehow request a token to use for access to the sheets API, but I am at a loss for how to create a request, and utilize the client_secret file generated from OAuth keys. I figured the googleAPI would have a function where you can pass this file directly, but I am lost in documentation.
Any insight would be appreciated!
All you need to do is supply the library with the location of the clientSecret.json file you should have downloaded from Google cloud console. This method should build the service for you and you can make the requests to the api. It will handle all the authorization.
from apiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
def get_service(api_name, api_version, scopes, key_file_location):
"""Get a service that communicates to a Google API.
Args:
api_name: The name of the api to connect to.
api_version: The api version to connect to.
scopes: A list auth scopes to authorize for the application.
key_file_location: The path to a valid service account JSON key file.
Returns:
A service that is connected to the specified API.
"""
credentials = ServiceAccountCredentials.from_json_keyfile_name(
key_file_location, scopes=scopes)
# Build the service object.
service = build(api_name, api_version, credentials=credentials)
return service
The best example I know of for service account authentication with python is the Google analytics quickstart If you have any issues altering it for google sheets let me know i can try and help.
Calling it should be something like this.
def main():
# Define the auth scopes to request.
scope = 'https://www.googleapis.com/auth/spreadsheets'
key_file_location = '<REPLACE_WITH_JSON_FILE>'
# Authenticate and construct service.
service = get_service(
api_name='sheets',
api_version='v4',
scopes=[scope],
key_file_location=key_file_location)
data = your_method_to_call_sheets(service)
How to create clientSecret.json remember to enable the google sheets api under libary
I'm following this tutorial Using OAuth 2.0 for Server to Server Applications. I am trying to connect to the Gmail API using a service account.
The code I end up with looks like this:
from oauth2client.service_account import ServiceAccountCredentials
from httplib2 import Http
from apiclient.discovery import build
import json
scopes = ['https://www.googleapis.com/auth/gmail.readonly']
credentials = ServiceAccountCredentials.from_json_keyfile_name('******.json', scopes)
http_auth = credentials.authorize(Http())
service = build('gmail', 'v1', http=http_auth)
request = service.users().messages().list(userId='me')
response = request.execute()
print json.dumps(response, sort_keys=True, indent=2)
However, when I run this code, I get the following error:
googleapiclient.errors.HttpError:https://www.googleapis.com/gmail/v1/users/me/messages?alt=json returned "Bad Request">
Can someone help me understand where this error is coming from?
Think of a service account as a dummy user. It has a Google Drive account a google calendar account. What it doesn't to my knowlage is have a Gmail account.
Normally when you request data using a service account you have to grant the service account access to that data manually. In the case of google drive you can share a folder with the service account enabling it to access google drive. (you can also upload to its drive account but that's out of scope for this question)
There is no way to grant another user access to your Gmail account so there is no way to use a service account with a normal user Gmail account.
Note: If this is Not a normal user Gmail account and is in fact one based on google domains then you can grant the service account access to all the emails of the other users on the domain via the admin section.
Other wise you need to look into using Oauth2 to access gmail.
I am try to access dbm api , I am authenticating the url using service account please find the sample code below
from oauth2client.service_account import ServiceAccountCredentials
from apiclient.discovery import build
from httplib2 import Http
scopes =['https://www.googleapis.com/auth/doubleclickbidmanager']
credentials = ServiceAccountCredentials.from_json_keyfile_name(
'path/to/key/.jsonfile', scopes=scopes)
http_auth = credentials.authorize(Http())
body={}
dbm = build('doubleclickbidmanager', 'v1', http=http_auth)
print dbm
request = dbm.lineitems().downloadlineitems(body=body).execute()
print request
If I use oauth mechanism to authenticate the url the code is running properly, since I don't want user interaction, I need server to server mechanism so I used service account
Steps which I tried:
I have created the service account and downloaded the json key file and used in the code but when I try to run my code it throws the following error:
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/doubleclickbidmanager/v1/lineitems/downloadlineitems?alt=json returned "You are not authorized to use DoubleClick Bid Manager API. Please contact dbm-support#google.com.">
Please help , thanks in advance.
As others have said here, you want to log in to the DBM site and add your service account as a user:
Then, per this documentation, you can set up service account credentials using your client secrets json file. If you want that service account to be able to access reports you've created in DBM under your user account (what you log in with) you need to delegate domain-wide authority:
delegated_credentials = credentials.create_delegated('user#example.org')
http_auth = delegated_credentials.authorize(Http())
dbm = build('doubleclickbidmanager', 'v1', http=http_auth)
queries = dbm.queries().listqueries().execute()['queries']
A service account isn't you its a dummy user it has its on Google drive account for example, and by default it doesn't have access to any DoubleClick Bid Manager APIs. Service accounts need to be pre authorized to be able to access private data. So for it to be able to access your double click data you are going to have to grant it access.
Normally with any other API I would say you take the service account email address and add it as a user. I don't have access to double click so I am not even sure if you can add other users manually. They don't have anything in the documentation about service accounts kind of makes me think its not supported. Let us know if you manage to get it to work.
I am trying to list all the files in my drive (about 10) but the following will only list 1 (and that isn't even a real file of mine)....
the code:
from httplib2 import Http
from oauth2client.client import SignedJwtAssertionCredentials
client='my_client_id'
client_email = 'my_client_email'
with open("/path/to/file.p12") as f:
private_key = f.read()
credentials = SignedJwtAssertionCredentials(client_email, private_key, 'https://www.googleapis.com/auth/drive')
http_auth = credentials.authorize(Http())
drive_service = build('drive', 'v2', http=http_auth)
r = drive_service.files().list().execute()
files = r['items']
for f in files:
print f['id'], f['title']
result:
"<file_id> How to get started with Drive"
EDIT:
This question is similar but the answer is to have the correct oauth scope, which I have above.
EDIT #2:
I thought it might be a timing issue so I gave it a few hours and still no goose.
EDIT #3:
If I try to copy a file from another user then list my files then I'll get 2 files:
" How to get started with Drive"
" My New File"
So, this is just listing files created by that app? How do I get the rest of my files???
You use a service account to authenticate. A service account does not have by default the right to access your Drive data, but only files that it owns by itself.
You have three options to work around this :
Create a folder in your Drive account, and share it (read/write) with the service account. Any file you place in that folder will be readable and writable both by you and your service account.
If you use Google Apps For Business, setup domain wide delegation to allow your service account to impersonate all users in your domain. That way you will be able to get your service account to behave as if it were your actual Google Apps account.
Whether you use or not Google Apps For Business : do not use a service account but rather 3-legged OAuth. With 3-legged OAuth you will be able to generate an access token and a refresh token that will allow your application to act in Drive on behalf of your actual Google account. Note that this last options does not use service accounts at all.
The simplest is obviously option (1). If it is not acceptable then I would go with option (3), unless you actually want to be able to impersonate all the users in your domain.
I'm having issues with the Directory API + Service Accounts (Google APIs). This is my current setup:
A web page has an OAuth2 login link like this: https://accounts.google.com/o/oauth2/auth?access_type=offline&state=%2Fprofile&redirect_uri=##REDIR##&response_type=code&client_id=##CLIENTID##&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fadmin.directory.user.readonly
Users log in there, authorizing the app to access the Directory API in read-only mode on their behalf.
I then try to retrieve the users of the domain of a given user (by knowing its email address), using the Directory API.
Python code:
from apiclient.discovery import build
from oauth2client.client import SignedJwtAssertionCredentials
import httplib2
CLIENT_ID = "xzxzxzxzxzxz.apps.googleusercontent.com"
APP_EMAIL = "xzxzxzxzxzxz#developer.gserviceaccount.com"
SCOPES = ('https://www.googleapis.com/auth/admin.directory.user.readonly')
f = file('key.p12', 'rb')
key = f.read()
f.close()
credentials = SignedJwtAssertionCredentials(APP_EMAIL, key, SCOPES, sub="user#example.com")
http = httplib2.Http()
http = credentials.authorize(http)
directory_service = build('admin', 'directory_v1', http=http)
users = directory_service.users().list(domain="example.com").execute()
print users
I have also tried setting sub="user#example.com" to the app owner like this sub="appowner#company.com", to no avail.
Another thing I have tried is not using impersonation at all (ie. removing the sub=xx part), which leads me to this error:
apiclient.errors.HttpError: https://www.googleapis.com/admin/directory/v1/users?domain=example.com&alt=json returned "Not Authorized to access this resource/api">
Using impersonation always yields me this. I have verified it has to do with the scopes and the api which I try to call:
oauth2client.client.AccessTokenRefreshError: access_denied
Now, the actual questions:
Should I be using service accounts? For me, it is the most convenient way as I don't have to be storing tokens which can be outdated altogether.
If service accounts are the way to go, what am I doing wrong in the way I use them? Impersonation with either the Google Apps administrator account (which logs in via OAuth web) or the app owner account does not seem to work.