Develop Reference

How to migrate from OOB OAuth?

I am running a python script on the device with no screen/no browser (ssh access only). The script used offline Google's OAuth:
def get_authenticated_service( self , args):
flow = flow_from_clientsecrets(os.path.dirname(os.path.abspath(__file__)) + "/" + CLIENT_SECRET_FILE,
scope='https://www.googleapis.com/auth/photoslibrary',
message='client_secrets files is missing')
credentials = None
storage = Storage(os.path.dirname(os.path.abspath(__file__)) + "/" + "token-oauth2.json")
credentials = storage.get()
if credentials is None or credentials.invalid:
credentials = run_flow(flow, storage, args)
self.token = credentials.access_token
return build('photoslibrary', 'v1',
http=credentials.authorize(httplib2.Http()))
If token is expired, it says:
Go to the following link in your browser:
https://accounts.google.com/o/oauth2/auth?scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fphotoslibrary&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&client_id=12345678-abcdefg.apps.googleusercontent.com&access_type=offline
But now Google says that redirect_uri=ietf:wg:oauth:2.0:oob is invalid.
I tried to manually replace it with 'http://localhost' (taken from CLIENT_SECRET_FILE), but when I take the code from the URL:
http://localhost/?code=4/12345_67899-t_vgPpDmLMGwD75F_w&scope=https://www.googleapis.com/auth/photoslibrary
the script says Authentication has failed: redirect_uri_mismatchBad Request.
I use google api client library 1.6.2.
How should I fix it?

I am running a python script on the device with no screen/no browser
(ssh access only).
The issue you are going to have is that the code needs to be authorized at least once. Run it on your local machine. in order to create the token the first time. Then copy it over to your server. with the file created by storage . there should be a refresh token in there.
Your refresh token should not be expiring. However if it expiring after a week then make sure that you have set your project to production in Google cloud console this way your refresh token will stop expiring.
There is no way around showing that consent screen to the user someone must authorize the code. That is done by showing the consent screen in a web browser.
oob issue
open your credentials.json file and make sure it says this
"redirect_uris":["http://localhost"]
If the oob end point is still there remove it.
The reason for your error is that the old credeintils.json file contained the following
"redirect_uris":["urn:ietf:wg:oauth:2.0:oob","http://localhost"]}
When the python client library ran it would always pick up the first one. That being "urn:ietf:wg:oauth:2.0:oob" which no longer works.
Now when you run your code it will probalby display a 404 error ignore this for now and look at the top bar url bar the code should be there in the response.
404
if you are getting a 404 and want to remove that read Loopback IP address (macOS, Linux, Windows desktop)
Google states.
To receive the authorization code using this URL, your application must be listening on the local web server. That is possible on many, but not all, platforms. However, if your platform supports it, this is the recommended mechanism for obtaining the authorization code.
So you need to figure out how to set up a local web app for an installed application in order to catch the response.
Google has not at this time told us how this is possible with all languages.
Sample
The following example works. It stores the user credentials in tokenPhotos.json. It will reload them as needed. Im not seeing any issues with OOB in this.
# To install the Google client library for Python, run the following command:
# pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
from __future__ import print_function
import os.path
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
# If modifying these scopes, delete the file token.json.
SCOPES = ['https://www.googleapis.com/auth/photoslibrary']
CREDS = 'C:\YouTube\dev\credentials.json';
def main():
"""Shows basic usage of the Photos v1 API.
Prints the names and ids of the first 20 albums the user has access to.
"""
creds = None
# The file token.json stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('tokenPhotos.json'):
creds = Credentials.from_authorized_user_file('tokenPhotos.json', SCOPES)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
CREDS, SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('tokenPhotos.json', 'w') as token:
token.write(creds.to_json())
try:
service = build('photoslibrary', 'v1', credentials=creds, static_discovery=False)
# Call the Photos v1 API
results = service.albums().list(
pageSize=10, fields="albums(id,title),nextPageToken").execute()
albums = results.get('albums', [])
if not albums:
print('No albums found.')
return
print('albums:')
for album in albums:
print(u'{0} ({1})'.format(album['title'], album['id']))
except HttpError as error:
# TODO(developer) - Handle errors from Photos API.
print(f'An error occurred: {error}')
if __name__ == '__main__':
main()

I can't access the blank google spreadsheet i created using python

So i have been trying to create a new blank google spreadsheet using python. I am using a python script i found online (also added a few modifications of my own) and i just can't get it to work the way i want. I've never actually used python before, neither google spreadsheets so i am a little confused!! The current issue is that whenever i run the code it seems to be working, but when i copy/paste the URL of the newly generated google spreadsheet, i don't even have permission to view it. Here is my code....Thank you in advance!!
"""
BEFORE RUNNING:
---------------
1. If not already done, enable the Google Sheets API
and check the quota for your project at
https://console.developers.google.com/apis/api/sheets
2. Install the Python client library for Google APIs by running
`pip install --upgrade google-api-python-client`
"""
from pprint import pprint
from googleapiclient import discovery
import gspread
from oauth2client.service_account import ServiceAccountCredentials
from google.oauth2 import service_account
# TODO: Change placeholder below to generate authentication credentials. See
# https://developers.google.com/sheets/quickstart/python#step_3_set_up_the_sample
#
# Authorize using one of the following scopes:
# 'https://www.googleapis.com/auth/drive'
# 'https://www.googleapis.com/auth/drive.file'
# 'https://www.googleapis.com/auth/spreadsheets'
SERVICE_ACCOUNT_FILE = 'client_secret.json'
f = open('client_secret.json','r')
print(f.readline())
f.close()
SCOPES = ["https://www.googleapis.com/auth/spreadsheets", "https://www.googleapis.com/auth/drive"]
credentials = None
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
service = discovery.build('sheets', 'v4', credentials=credentials)
spreadsheet_body = {
# TODO: Add desired entries to the request body.
}
request = service.spreadsheets().create(body=spreadsheet_body)
response = request.execute()
# TODO: Change code below to process the `response` dict:
pprint(response)

The issue is that the sheet is created with the serviceaccount as the owner, not under your personal Gmail account.
There's 2 options:
The bad way would be to give your personal account access to the generated GSheet. Issue with this is that the serviceaccount will still be the owner. I'm not going to tell you how to do this as this is absolutely the wrong way.
The right way would be to use proper credentials when creating the API client. This is explained in detail in this article.
Pay special attention to the piece of code that creates the credentials object.
creds = None
# The file token.json stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.json'):
creds = Credentials.from_authorized_user_file('token.json', SCOPES)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.json', 'w') as token:
token.write(creds.to_json())
service = build('sheets', 'v4', credentials=creds)
As you can see this code doesn't directly create a credentials object from the serviceacocunt. Instead it asks for permission to use your personal Gmail account to call the API.
Note that if you have a GSuite/Workspace account, you can use impersonation instead. This is actually the preferred way, but only works with said GSuite/Workspace accounts.

Google App Script - "message": "Requested entity was not found." with devMode = false

I am trying to connect my Python script with my project in Google App Script. I have followed all the insctructions in this guide.
I have of course deployed it as an executable API and have tested it with access to only myself, my organization and anyone options.
When I pass the request with devMode as true, it all works fine. I understand that in this case, it is running the latest saved version. However when I set it to false then I get back the error "message": "Requested entity was not found." which as I understand is trying to run the latest deployed version.
I have also tried going through these questions 1 and 2 but apparently, the problem they had was the opposite where the script wouldn't run with devMode set to true.
Everything else seems to be executing correctly but I cannot find the reason why it wouldn't run the script without being on devMode
This is my script:
"""
Shows basic usage of the Apps Script API.
Call the Apps Script API to create a new script project, upload a file to the
project, and log the script's URL to the user.
"""
from __future__ import print_function
import pickle
import os.path
from googleapiclient import errors
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
# If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/script.projects',
'https://www.googleapis.com/auth/spreadsheets.currentonly'
]
SCRIPT_ID = 'someid'
def main():
"""Calls the Apps Script API.
"""
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
service = build('script', 'v1', credentials=creds)
# Call the Apps Script API
try:
# Create a new project
request = {'function': 'setData'}
response = service.scripts().run(body=request,
scriptId=SCRIPT_ID).execute()
print(response)
except errors.HttpError as error:
# The API encountered a problem.
print(error.content)
if __name__ == '__main__':
main()

I have been able to face the same issue and I have found out that changing the script Id to the Deployment Id worked.
The deployment Id can be found on the Apps Script script:
Open the script
Go to Deploy -> Manage Deployments
Get the Deployment ID from the active deployment
Once you have the deployment, go to the python script and modify the SCRIPT ID with the Deployment Id

Google Apps Script API - Permissions for my Python script to execute an Apps Script function

this is my first contribution here.
I'm trying to access Gmail through a python script. To do so, I've created a Google Apps Script function and used the Apps Script API between the 2.
(This doc displays what I'm trying to do)
So the python script correctly accesses the API, but fails to execute the function.
While it works in the Script Editor, in Python it raises a permissions issue:
'errorMessage': 'Exception: The script does not have permission to perform that action.
Required permissions: (
https://www.googleapis.com/auth/gmail.labels ||
https://www.googleapis.com/auth/gmail.metadata ||
https://www.googleapis.com/auth/gmail.readonly ||
https://www.googleapis.com/auth/gmail.modify ||
https://mail.google.com/
)',
'errorType': 'ScriptError'
I guess it is related to the Client ID OAuth, since I was not able to find where to grant it permissions. I've just :
created the credentials in Google Cloud Platform,
exported it as creds.json in my python script folder.
Here is my code, almost copy pasted from a tutorial:
import pickle
import os.path
from googleapiclient import errors
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
# Here I've edited the scopes of authorizations required
SCOPES = [
"https://www.googleapis.com/auth/gmail.labels",
"https://www.googleapis.com/auth/gmail.metadata",
"https://www.googleapis.com/auth/gmail.readonly",
"https://www.googleapis.com/auth/gmail.modify",
"https://mail.google.com/"
]
def get_scripts_service():
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
# Here I've placed the downloaded credentials .json file
flow = InstalledAppFlow.from_client_secrets_file(
'creds.json', SCOPES)
creds = flow.run_local_server(port=0)
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
return build('script', 'v1', credentials=creds)
service = get_scripts_service()
API_ID = # Here i've pasted my API_ID
request = {"function": # Here i've pasted my functionName}
try:
response = service.scripts().run(body=request, scriptId=API_ID).execute()
print (response)
except errors.HttpError as error:
# The API encountered a problem.
print(error.content)
How do I grant permissions to my script?

Simple as Aerials said! Thanks.
It was because the Client ID was created before I edited the scopes. I've deleted the token and created a new one.

Google Sheets API works locally but timeout on connection when running from AWS Lambda

I have some code that works perfectly locally and doesn't work at all from AWS Lambda. It's almost like the API is blocked and I'm not sure what to look for next.
I can hit other things "out on the net" so it's not a generic routing issue and I get a socket timeout error from the AWS Run.
I've tried a few different libraries including an older version of the main library. Everyone of them works locally, doesn't work from AWS.
#!/usr/bin/env python3
# replace
creds_file = "/path/to/creds.json"
import pickle
import os.path
from googleapiclient.discovery import build
from google.oauth2 import service_account
scopes = ['https://www.googleapis.com/auth/spreadsheets.readonly']
SAMPLE_SPREADSHEET_ID = "<spreadsheetid>"
# Sample Range
SAMPLE_RANGE_NAME = "Sheet1!A1:D"
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('/tmp/token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
# Customized
creds = service_account.Credentials.from_service_account_file(creds_file)
creds_w_scopes = creds.with_scopes(SCOPES)
# Save the credentials for the next run
with open('/tmp/token.pickle', 'wb') as token:
pickle.dump(creds_w_scopes, token)
# Timeout is Here in the Cloud
service = build('sheets', 'v4', credentials=creds_w_scopes)
# Call the Sheets API
sheet = service.spreadsheets()
result = sheet.values().get(spreadsheetId=SAMPLE_SPREADSHEET_ID,
range=SAMPLE_RANGE_NAME).execute()
values = result.get('values', [])
print(values)
Locally I get the results of the sheet (just some smample data) in the cloud though it hangs on this call
service = build('sheets', 'v4', credentials=creds)
And then times out with a socket.timeout error.

I had the exact same issue today and after quite some time I figured out that for me the problem was lack of memory size for the lambdas.
If you look into CloudWatch you can see how much RAM (memory) your Lambda has available and how much it is using. If you see the usage being equal to the maximum RAM available, you should probably increase the available RAM (or make your code more efficient).
In our case, simply increasing RAM from 128MB to 384MB resolved the 60 second timeouts and made the lambda run in a few seconds instead.

This happens because you have ipv4 stack locally and ipv6 stack at AWS.
I found one solution that helps, and it's socket patching:
import socket
old_getaddrinfo = socket.getaddrinfo
def new_getaddrinfo(*args, **kwargs):
responses = old_getaddrinfo(*args, **kwargs)
return [response
for response in responses
if response[0] == socket.AF_INET]
socket.getaddrinfo = new_getaddrinfo
Note that it should be called before importing googleapiclient.discovery and google.oauth2 and can be troublemaker for other libraries in your code. I ended up with moving these calls to python subprocess with patched socket library.