Develop Reference

Python connection to BigQuery not working

So I was following a tutorial to create the credentials using Oauth2 and connect python to GCP, I've made the following code that successfully calls the auth window in google.
import os
import pickle
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
from google.cloud import bigquery
credentials = None
#token.pickle stores user credentials from prev successful login
if os.path.exists("token.pickle"):
print("Loading Credentials from file...")
with open("token.pickle","rb") as token:
credentials = pickle.load(token)
# if theres not a valid token either refresh or login
if not credentials or not credentials.valid:
if credentials and credentials.expired and credentials.refresh_token:
print("Refreshing Access Token...")
credentials.refresh(Request())
else:
print("Fetching New Tokens...")
flow = InstalledAppFlow.from_client_secrets_file(
"client_secrets.json",
scopes=["https://www.googleapis.com/auth/cloud-platform"],
)
flow.run_local_server(
port=8080, prompt="consent", authorization_prompt_message=""
)
credentials = flow.credentials
# saving credentials for future runs
with open("token.pickle","rb") as f:
print("Saving Credentials for future...")
pickle.dump(credentials,f)
However when I try to connect to Bigquery using this:
client = bigquery.Client(credentials=credentials)
Its telling me the following, the 3 dots have all the data from the token however I deleted it due to security reason:
google.auth.exceptions.DefaultCredentialsError: File {...} was not found.

How to migrate from OOB OAuth?

I am running a python script on the device with no screen/no browser (ssh access only). The script used offline Google's OAuth:
def get_authenticated_service( self , args):
flow = flow_from_clientsecrets(os.path.dirname(os.path.abspath(__file__)) + "/" + CLIENT_SECRET_FILE,
scope='https://www.googleapis.com/auth/photoslibrary',
message='client_secrets files is missing')
credentials = None
storage = Storage(os.path.dirname(os.path.abspath(__file__)) + "/" + "token-oauth2.json")
credentials = storage.get()
if credentials is None or credentials.invalid:
credentials = run_flow(flow, storage, args)
self.token = credentials.access_token
return build('photoslibrary', 'v1',
http=credentials.authorize(httplib2.Http()))
If token is expired, it says:
Go to the following link in your browser:
https://accounts.google.com/o/oauth2/auth?scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fphotoslibrary&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&client_id=12345678-abcdefg.apps.googleusercontent.com&access_type=offline
But now Google says that redirect_uri=ietf:wg:oauth:2.0:oob is invalid.
I tried to manually replace it with 'http://localhost' (taken from CLIENT_SECRET_FILE), but when I take the code from the URL:
http://localhost/?code=4/12345_67899-t_vgPpDmLMGwD75F_w&scope=https://www.googleapis.com/auth/photoslibrary
the script says Authentication has failed: redirect_uri_mismatchBad Request.
I use google api client library 1.6.2.
How should I fix it?

I am running a python script on the device with no screen/no browser
(ssh access only).
The issue you are going to have is that the code needs to be authorized at least once. Run it on your local machine. in order to create the token the first time. Then copy it over to your server. with the file created by storage . there should be a refresh token in there.
Your refresh token should not be expiring. However if it expiring after a week then make sure that you have set your project to production in Google cloud console this way your refresh token will stop expiring.
There is no way around showing that consent screen to the user someone must authorize the code. That is done by showing the consent screen in a web browser.
oob issue
open your credentials.json file and make sure it says this
"redirect_uris":["http://localhost"]
If the oob end point is still there remove it.
The reason for your error is that the old credeintils.json file contained the following
"redirect_uris":["urn:ietf:wg:oauth:2.0:oob","http://localhost"]}
When the python client library ran it would always pick up the first one. That being "urn:ietf:wg:oauth:2.0:oob" which no longer works.
Now when you run your code it will probalby display a 404 error ignore this for now and look at the top bar url bar the code should be there in the response.
404
if you are getting a 404 and want to remove that read Loopback IP address (macOS, Linux, Windows desktop)
Google states.
To receive the authorization code using this URL, your application must be listening on the local web server. That is possible on many, but not all, platforms. However, if your platform supports it, this is the recommended mechanism for obtaining the authorization code.
So you need to figure out how to set up a local web app for an installed application in order to catch the response.
Google has not at this time told us how this is possible with all languages.
Sample
The following example works. It stores the user credentials in tokenPhotos.json. It will reload them as needed. Im not seeing any issues with OOB in this.
# To install the Google client library for Python, run the following command:
# pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
from __future__ import print_function
import os.path
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
# If modifying these scopes, delete the file token.json.
SCOPES = ['https://www.googleapis.com/auth/photoslibrary']
CREDS = 'C:\YouTube\dev\credentials.json';
def main():
"""Shows basic usage of the Photos v1 API.
Prints the names and ids of the first 20 albums the user has access to.
"""
creds = None
# The file token.json stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('tokenPhotos.json'):
creds = Credentials.from_authorized_user_file('tokenPhotos.json', SCOPES)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
CREDS, SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('tokenPhotos.json', 'w') as token:
token.write(creds.to_json())
try:
service = build('photoslibrary', 'v1', credentials=creds, static_discovery=False)
# Call the Photos v1 API
results = service.albums().list(
pageSize=10, fields="albums(id,title),nextPageToken").execute()
albums = results.get('albums', [])
if not albums:
print('No albums found.')
return
print('albums:')
for album in albums:
print(u'{0} ({1})'.format(album['title'], album['id']))
except HttpError as error:
# TODO(developer) - Handle errors from Photos API.
print(f'An error occurred: {error}')
if __name__ == '__main__':
main()

Google App Script - "message": "Requested entity was not found." with devMode = false

I am trying to connect my Python script with my project in Google App Script. I have followed all the insctructions in this guide.
I have of course deployed it as an executable API and have tested it with access to only myself, my organization and anyone options.
When I pass the request with devMode as true, it all works fine. I understand that in this case, it is running the latest saved version. However when I set it to false then I get back the error "message": "Requested entity was not found." which as I understand is trying to run the latest deployed version.
I have also tried going through these questions 1 and 2 but apparently, the problem they had was the opposite where the script wouldn't run with devMode set to true.
Everything else seems to be executing correctly but I cannot find the reason why it wouldn't run the script without being on devMode
This is my script:
"""
Shows basic usage of the Apps Script API.
Call the Apps Script API to create a new script project, upload a file to the
project, and log the script's URL to the user.
"""
from __future__ import print_function
import pickle
import os.path
from googleapiclient import errors
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
# If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/script.projects',
'https://www.googleapis.com/auth/spreadsheets.currentonly'
]
SCRIPT_ID = 'someid'
def main():
"""Calls the Apps Script API.
"""
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
service = build('script', 'v1', credentials=creds)
# Call the Apps Script API
try:
# Create a new project
request = {'function': 'setData'}
response = service.scripts().run(body=request,
scriptId=SCRIPT_ID).execute()
print(response)
except errors.HttpError as error:
# The API encountered a problem.
print(error.content)
if __name__ == '__main__':
main()

I have been able to face the same issue and I have found out that changing the script Id to the Deployment Id worked.
The deployment Id can be found on the Apps Script script:
Open the script
Go to Deploy -> Manage Deployments
Get the Deployment ID from the active deployment
Once you have the deployment, go to the python script and modify the SCRIPT ID with the Deployment Id

Google Apps Script API - Permissions for my Python script to execute an Apps Script function

this is my first contribution here.
I'm trying to access Gmail through a python script. To do so, I've created a Google Apps Script function and used the Apps Script API between the 2.
(This doc displays what I'm trying to do)
So the python script correctly accesses the API, but fails to execute the function.
While it works in the Script Editor, in Python it raises a permissions issue:
'errorMessage': 'Exception: The script does not have permission to perform that action.
Required permissions: (
https://www.googleapis.com/auth/gmail.labels ||
https://www.googleapis.com/auth/gmail.metadata ||
https://www.googleapis.com/auth/gmail.readonly ||
https://www.googleapis.com/auth/gmail.modify ||
https://mail.google.com/
)',
'errorType': 'ScriptError'
I guess it is related to the Client ID OAuth, since I was not able to find where to grant it permissions. I've just :
created the credentials in Google Cloud Platform,
exported it as creds.json in my python script folder.
Here is my code, almost copy pasted from a tutorial:
import pickle
import os.path
from googleapiclient import errors
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
# Here I've edited the scopes of authorizations required
SCOPES = [
"https://www.googleapis.com/auth/gmail.labels",
"https://www.googleapis.com/auth/gmail.metadata",
"https://www.googleapis.com/auth/gmail.readonly",
"https://www.googleapis.com/auth/gmail.modify",
"https://mail.google.com/"
]
def get_scripts_service():
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
# Here I've placed the downloaded credentials .json file
flow = InstalledAppFlow.from_client_secrets_file(
'creds.json', SCOPES)
creds = flow.run_local_server(port=0)
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
return build('script', 'v1', credentials=creds)
service = get_scripts_service()
API_ID = # Here i've pasted my API_ID
request = {"function": # Here i've pasted my functionName}
try:
response = service.scripts().run(body=request, scriptId=API_ID).execute()
print (response)
except errors.HttpError as error:
# The API encountered a problem.
print(error.content)
How do I grant permissions to my script?

Simple as Aerials said! Thanks.
It was because the Client ID was created before I edited the scopes. I've deleted the token and created a new one.

Connecting to Google Analytics API using Python

I have tried to follow this tutorial to connect to Google Analytics API. I have followed the instructions, step by step. I have Python 2.7 installed on my computer. I have installed Google client library. When I run the program, i get the following error in the terminal:
Traceback (most recent call last):
File "HelloAnalytics.py", line 6, in <module>
from oauth2client.client import SignedJwtAssertionCredential
ImportError: cannot import name SignedJwtAssertionCredentials
The line 6 that it referring to is:
from oauth2client.client import SignedJwtAssertionCredentials
I am at a complete lost. I looked at others who have had the same error, here, here, and here, but the solutions did not work. I have some programming knowledge, but compared to many of you, I am a noob.
The full code is here:
"""A simple example of how to access the Google Analytics API."""
import argparse
from apiclient.discovery import build
from oauth2client.client import SignedJwtAssertionCredentials
import httplib2
from oauth2client import client
from oauth2client import file
from oauth2client import tools
def get_service(api_name, api_version, scope, key_file_location,
service_account_email):
"""Get a service that communicates to a Google API.
Args:
api_name: The name of the api to connect to.
api_version: The api version to connect to.
scope: A list auth scopes to authorize for the application.
key_file_location: The path to a valid service account p12 key file.
service_account_email: The service account email address.
Returns:
A service that is connected to the specified API.
"""
f = open(key_file_location, 'rb')
key = f.read()
f.close()
credentials = SignedJwtAssertionCredentials(service_account_email, key,
scope=scope)
http = credentials.authorize(httplib2.Http())
# Build the service object.
service = build(api_name, api_version, http=http)
return service
def get_first_profile_id(service):
# Use the Analytics service object to get the first profile id.
# Get a list of all Google Analytics accounts for this user
accounts = service.management().accounts().list().execute()
if accounts.get('items'):
# Get the first Google Analytics account.
account = accounts.get('items')[0].get('id')
# Get a list of all the properties for the first account.
properties = service.management().webproperties().list(
accountId=account).execute()
if properties.get('items'):
# Get the first property id.
property = properties.get('items')[0].get('id')
# Get a list of all views (profiles) for the first property.
profiles = service.management().profiles().list(
accountId=account,
webPropertyId=property).execute()
if profiles.get('items'):
# return the first view (profile) id.
return profiles.get('items')[0].get('id')
return None
def get_results(service, profile_id):
# Use the Analytics Service Object to query the Core Reporting API
# for the number of sessions within the past seven days.
return service.data().ga().get(
ids='ga:' + profile_id,
start_date='7daysAgo',
end_date='today',
metrics='ga:sessions').execute()
def print_results(results):
# Print data nicely for the user.
if results:
print 'View (Profile): %s' % results.get('profileInfo').get('profileName')
print 'Total Sessions: %s' % results.get('rows')[0][0]
else:
print 'No results found'
def main():
# Define the auth scopes to request.
scope = ['https://www.googleapis.com/auth/analytics.readonly']
# Use the developer console and replace the values with your
# service account email and relative location of your key file.
service_account_email = '<Replace with your service account email address.>'
key_file_location = '<Replace with /path/to/generated/client_secrets.p12>'
# Authenticate and construct service.
service = get_service('analytics', 'v3', scope, key_file_location,
service_account_email)
profile = get_first_profile_id(service)
print_results(get_results(service, profile))
if __name__ == '__main__':
main()
Any help or direction would be appreciated.

The source repository was recently updated, and the Hello Analytics Guides have since also been updated to make use of the new code:
from apiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
...
credentials = ServiceAccountCredentials.from_p12_keyfile(
service_account_email, key_file_location, scopes=scope)
Where service_account_email is the email address of the service account you get from the dev console, and the key_file_locaiton is /the/path/to/key.p12, and scopes are the scopes you need to grant to the service account.
Please remember to add the service account email address as an authorized user of the Google Analytics view (profile) to which you wish it to have access.