Develop Reference

Google Drive API Service Account authentication to upload and insert an image file to into copied doc using Python

Google Drive API docs are not super great at helping determine best way to authenticate using a service account that I can then upload a .png file to the Drive. My end goal it so upload a .png file, copy a template doc, batch update that doc using text replace, and insert the newly uploaded .png image into that doc.
Sample code below:
from dotenv import load_dotenv
from google.oauth2 import service_account
from googleapiclient.discovery import build
from googleapiclient.http import MediaFileUpload
def credentials_from_file():
credentials = service_account.Credentials.from_service_account_file(
os.getenv('SERVICE_ACCOUNT_FILE'),
scopes=os.getenv('SCOPES')
)
drive_service = build('drive', 'v3', credentials=credentials)
return drive_service
def google_upload(drive_service, metadata_name, parents, permissions, file_path, mime_type):
file_metadata = {'kind':'drive#file', 'name':metadata_name, 'parents':parents, 'permissions':permissions}
media = MediaFileUpload(file_path, mimetype=mime_type)
file = drive_service.files().create(body=file_metadata, media_body=media, fields='id', supportsAllDrives=True).execute()
print('File ID: %s' % file.get('id'))
Implementation of Code
credentials = credentials_from_file()
drive_service = build('drive', 'v3', credentials=credentials)
metadata_name = custom_variables_png_table_img
parents = ['xxxx']
permissions = [{'kind':'drive#permission', 'emailAddress':os.getenv('EMAIL_ACCOUNT'), 'role':'owner'}]
file_path = custom_variables_png_table_img
mime_type = 'image/png'
google_upload(drive_service, metadata_name, parents, permissions, file_path, mime_type)
EDIT:
Looks like I forgot to actually write was the problem is. It's two fold.
I keep getting 2 errors when trying to run the google_upload() function which looks like an authentication error with the service account.
Error #1: jwt_grant access_token = response_data["access_token"] KeyError: 'access_token'
The above exception was the direct cause of the following exception:
Error #2: google.auth.exceptions.RefreshError: ('No access token in response.', {'id_token': 'xxx'})
Permissions being properly set on the recently uploaded image file.

The code you are using currently seams to be the same as what I have seen before.
from apiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
SCOPES = ['https://www.googleapis.com/auth/drive.readonly']
KEY_FILE_LOCATION = '<REPLACE_WITH_JSON_FILE_PATH_TO_FILE>'
def initialize_drive():
"""Initializes an Google Drive API V3 service object.
Returns:
An authorized Google Drive API V3 service object.
"""
credentials = ServiceAccountCredentials.from_json_keyfile_name(
KEY_FILE_LOCATION, SCOPES)
# Build the service object.
driveService = build('drive', 'v4', credentials=credentials)
return driveService
You haven't mentioned what is wrong with your code however i can make a few guesses.
The thing is that you mention you want to upload an image and the insert it into a document. You need to remember that the Google drive api is just a file storage api it can do more then that store files.
When you upload the file using the service account you need to remember that the service account is not you. So when you are uploading this file to this directory parents = ['xxxx'] where ever that directory is, either on the service accounts drive account or if this directory is one of your persona directories which you have shared with the service account. You may not have permissions to see this file.
By calling permissions create after uploading your file you can grant your own personal account permissions to access the file as well.
As for adding the image to a document. well the only way google can help you with that is if it is a Google doc type document. Then you would need to go though the Google docs api which would then give you access to add things programmaticlly to a document. I haven't used this API much so im not sure if it has the ability to add images to a document.
You should be able to use the google docs api with your service account you will just need to create a docs service using the same creds you already have from google drive.
service = build('docs', 'v1', credentials=creds)

How to move a file in a shared google drive to trash using a service account

I use a service account that is content-manager. I have no problem uploading files to the shared drive using the drive-api from python. Using
service.files().list(q="name='file_name'", fields="files(id)").execute()
I obtain the file_id from my code. This file_id is the right one based on the link to the file.
When I perform the following statement:
response = service.files().update(fileId=file_id, body={'trashed': True}).execute()
I get a
404: file not found.
How to resolve this ? With my personal account (also as content-manager) I have no problems trashing the file.

Requirements
If you clearly understand how to impersonate an account you can skip to the Solution step.
First of all you will need a service account
Delegate domain-wide authority
Make sure that you are correctly calling the API
Solution
By default Python Google Drive API client V3 doesn't include shared drive files, that's why you have to explicitly pass a parameter supportsAllDrives and set it to True and before that you should list your files in order to know the fileId parameter by using includeItemsFromAllDrives and supportsAllDrives. Here's an example to list all the files in all your drives and how to trash a file in a Shared Drive using a service account:
from googleapiclient.discovery import build
from google.oauth2 import service_account
SCOPES = ['https://www.googleapis.com/auth/drive']
SERVICE_ACCOUNT_FILE = './service_account_key.json'
credentials = service_account.Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE, scopes=SCOPES)
# Impersonate user#example.com account in my example.com domain
delegated_credentials = credentials.with_subject('user#example.com')
# Use the delegated credentials to impersonate the user
service = build('drive', 'v3', credentials=delegated_credentials)
# List all the files in your Drives (Shared Drives included)
results = service.files().list(fields="nextPageToken, files(id, name, trashed)", includeItemsFromAllDrives=True, supportsAllDrives=True).execute()
items = results.get('files', [])
if not items:
print('No files found.')
else:
print('Files:')
for item in items:
print(u'{0} ({1}) - Trashed? {2}'.format(item['name'], item['id'], item['trashed']))
# Use the filedId in order to trash your shared file
response = service.files().update(fileId=fileId, body={'trashed': True}, supportsAllDrives=True).execute()
print(response)
Otherwise if you already know the fileId just use the update part.
Reference
Python Google Drive API client V3 > Update a file
Google Identity Platform > Impersonate a user by using a service account

google service account credentials not working

I am trying to create a simple web application that automatically uploads my database & media backup to a designated google drive. I have followed the official document and created a service account credential, gave it the owner role, and extracted a key(json file) from Google cloud platform. I enabled the Google Drive API on my account and wrote this code, but the credentials.valid returns False and my file would not upload to my drive.
from google.oauth2 import service_account
import googleapiclient as google
from googleapiclient.http import MediaFileUpload, HttpRequest
from googleapiclient.discovery import build
SCOPES = ['https://www.googleapis.com/auth/drive']
credentials = service_account.Credentials.from_service_account_file('./service-credentials.json', scopes=SCOPES)
print(credentials.valid)
service = build('drive', 'v3', credentials=credentials)
file_metadata = {'name' : 'python.png'}
media = MediaFileUpload('./python.png', mimetype='image/png')
file_up = service.files().create(body=file_metadata, media_body=media, fields='id').execute()
file_back = service.files().get(fileId=file_up['id']).execute()
print(file_back.get('WebContentLink'))

How about this modification?
Modification points:
I think that in your script, service of service = build('drive', 'v3', credentials=credentials) can be used for uploading the file.
In my environment, I could confirm that the file can be uploaded using your script.
From my file would not upload to my drive., I thought that you might misunderstand about the service account. The file uploaded with the service account is created to the Drive of the service account. This Drive is different from your Google Drive of your account. I thought that this might be the reason of my file would not upload to my drive..
If you want to see the file uploaded with the service account at your Google Drive, it is required to share the uploaded file with your Google account. Or, it is required to upload the file to the folder in your Google Drive shared with the service account.
And also, in your script, file_back.get('WebContentLink') is used. In this case, None is always returned because WebContentLink is required to be WebContentLink. And also, in Drive API v3, the default returned values don't include webContentLink. So it is required to set fields.
When above points are reflected to your script, your script becomes as follows.
Modified script:
from google.oauth2 import service_account
import googleapiclient as google
from googleapiclient.http import MediaFileUpload, HttpRequest
from googleapiclient.discovery import build
SCOPES = ['https://www.googleapis.com/auth/drive']
credentials = service_account.Credentials.from_service_account_file('./service-credentials.json', scopes=SCOPES)
service = build('drive', 'v3', credentials=credentials)
file_metadata = {'name': 'python.png'}
media = MediaFileUpload('./python.png', mimetype='image/png')
file_up = service.files().create(body=file_metadata, media_body=media, fields='id').execute()
# Create a permission. Here, your Google account is shared with the uploaded file.
yourEmailOfGoogleAccount = '###' # <--- Please set your Email address of Google account.
permission = {
'type': 'user',
'role': 'writer',
'emailAddress': yourEmailOfGoogleAccount,
}
service.permissions().create(fileId=file_up['id'], body=permission).execute()
file_back = service.files().get(fileId=file_up['id'], fields='webContentLink').execute() # or fields='*'
print(file_back.get('webContentLink'))
When you run above script, the uploaded file can be seen at "Shared with me" in your Google Drive.
If you want to put the specific folder of your Google Drive, please use the following script. In this case, before you run the script, please share the folder with the email of the service account. Please be careful this.
from google.oauth2 import service_account
import googleapiclient as google
from googleapiclient.http import MediaFileUpload, HttpRequest
from googleapiclient.discovery import build
SCOPES = ['https://www.googleapis.com/auth/drive']
credentials = service_account.Credentials.from_service_account_file('./service-credentials.json', scopes=SCOPES)
service = build('drive', 'v3', credentials=credentials)
file_metadata = {'name': 'python.png', 'parents': ['###']} # <--- Please set the folder ID shared with the service account.
media = MediaFileUpload('./python.png', mimetype='image/png')
file_up = service.files().create(body=file_metadata, media_body=media, fields='id').execute()
file_back = service.files().get(fileId=file_up['id'], fields='webContentLink').execute() # or fields='*'
print(file_back.get('webContentLink'))
Note:
In the current stage, when the owner of file uploaded with the service account is changed, an error like You can't yet change the owner of this item. (We're working on it.). So I proposed above modified script.
References:
Files: create
Files: get
Permissions: create

How to use the Google API without checking a client_secret.json into version control?

I'm working on a project in which, in accordance with https://12factor.net/config, we don't things like credentials in our code, but rather in environment variables.
I'm looking into using the Google Sheets API to collate some data from our database and put it into a Google sheet. Here is the partial example script from https://developers.google.com/sheets/api/quickstart/python:
from __future__ import print_function
from apiclient.discovery import build
from httplib2 import Http
from oauth2client import file as oauth_file, client, tools
# Setup the Sheets API
SCOPES = 'https://www.googleapis.com/auth/spreadsheets.readonly'
store = oauth_file.Storage('token.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets('credentials.json', SCOPES)
creds = tools.run_flow(flow, store)
service = build('sheets', 'v4', http=creds.authorize(Http()))
Firstly, it is not clear to me from the documentation what 'token.json' and 'credentials.json' should be in this example. From the API console, in the Credentials tab, I downloaded a client_secret_<long suffix>.json which looks like this:
{"installed":{"client_id":"[our_client_id]","project_id":"nps-survey-1532981793379","auth_uri":"https://accounts.google.com/o/oauth2/auth","token_uri":"https://accounts.google.com/o/oauth2/token","auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs","client_secret":"[our_client_secret]","redirect_uris":["urn:ietf:wg:oauth:2.0:oob","http://localhost"]}}
Should this JSON file be the 'token.json' in this example, or the 'credentials.json'? Also, is there a way to instantiate valid creds by specifying the client secret and client ID directly, and not using this JSON file?

I ended up going through the OAuth 2.0 setup for a web application instead of an installed application, and using google_auth_oauthlib. The Flow object has a class method from_client_config() which can be used like so (cf. https://developers.google.com/identity/protocols/OAuth2WebServer):
from django.conf import settings
from django.shortcuts import redirect
import google.oauth2.credentials
import google_auth_oauthlib.flow
# Client configuration for an OAuth 2.0 web server application
# (cf. https://developers.google.com/identity/protocols/OAuth2WebServer)
CLIENT_CONFIG = {'web': {
'client_id': settings.GOOGLE_CLIENT_ID,
'project_id': settings.GOOGLE_PROJECT_ID,
'auth_uri': 'https://accounts.google.com/o/oauth2/auth',
'token_uri': 'https://www.googleapis.com/oauth2/v3/token',
'auth_provider_x509_cert_url': 'https://www.googleapis.com/oauth2/v1/certs',
'client_secret': settings.GOOGLE_CLIENT_SECRET,
'redirect_uris': settings.GOOGLE_REDIRECT_URIS,
'javascript_origins': settings.GOOGLE_JAVASCRIPT_ORIGINS}}
# This scope will allow the application to manage your calendars
SCOPES = ['https://www.googleapis.com/auth/calendar']
def get_authorization_url():
# Use the information in the client_secret.json to identify
# the application requesting authorization.
flow = google_auth_oauthlib.flow.Flow.from_client_config(
client_config=CLIENT_CONFIG,
scopes=SCOPES)
# Indicate where the API server will redirect the user after the user completes
# the authorization flow. The redirect URI is required.
flow.redirect_uri = 'http://localhost:8000'
# Generate URL for request to Google's OAuth 2.0 server.
# Use kwargs to set optional request parameters.
authorization_url, state = flow.authorization_url(
# Enable offline access so that you can refresh an access token without
# re-prompting the user for permission. Recommended for web server apps.
access_type='offline',
# Enable incremental authorization. Recommended as a best practice.
include_granted_scopes='true')
return authorization_url, state
The settings attributes are, in turn, generated by calling os.getenv() for each corresponding attribute. In this way, the configuration can be obtained from environment variables instead of a local file.

Using Google Admin to view Drive files Domain-wide

I'm trying to list all Google Drive files Domain-wide, both users that still work here, and those that have moved on. With that, we can grep the output for certain terms (former customers) to delete customer-related files.
I believe I have a successful way to list all users using the Admin SDK Quickstart, since we have only about 200 total users (max is 500). I also have a way to list all files for a user using the Drive REST API's files.list() method. What I need to know is how to impersonate each user iteratively, in order to run the file listing script.
I have found the blurb .setServiceAccountUser(someone#domain.com) but I'm not really sure where to implement this, either in the service account authorization step, or in a separate middle-man script.

Have a look at https://github.com/pinoyyid/googleDriveTransferOwnership/blob/master/src/couk/cleverthinking/tof/Main.java
Specifically lines 285-299 which deal with generating a credential for an impersonated user.
GoogleCredential.Builder builder = new GoogleCredential.Builder()
.setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(serviceAccountEmailAddress)
.setServiceAccountPrivateKeyFromP12File(f)
.setServiceAccountScopes(Collections.singleton(SCOPE));
// if requested, impresonate a domain user
if (!"ServiceAccount".equals(impersonatedAccountEmailAddress)) {
builder.setServiceAccountUser(impersonatedAccountEmailAddress);
}
// build the Drive service
Drive service = new Drive.Builder(HTTP_TRANSPORT, JSON_FACTORY, null)
.setApplicationName("TOF")
.setHttpRequestInitializer(builder.build()).build();
This is Java, but should at least tell you what the steps are.

You need to implement the authorization flow for Service Accounts.
Once you create a service account in a GCP project (console.developers.google.com), enable DWD (domain-wide delegation), then authorize that service account in your G Suite admin console, that key can then be used to "impersonate" any account in the G Suite instance:
Create the credentials object from the json file
from oauth2client.service_account import ServiceAccountCredentials
scopes = ['https://www.googleapis.com/auth/gmail.readonly']
credentials = ServiceAccountCredentials.from_json_keyfile_name(
'/path/to/keyfile.json', scopes=scopes)
Create a credential that can impersonate user#example.org (could be any user in the domain though)
delegated_credentials = credentials.create_delegated('user#example.org')
Authorize the credential object (i.e. get an access_token)
from httplib2 import Http
http_auth = credentials.authorize(Http())
Call the Gmail API:
from apiclient import discovery
service = discovery.build('gmail', 'v1', http=http)
response = service.users().messages().list(userId='user#example.org').execute()