Develop Reference

Google Cloud API 403 - Insufficient Permission

I would like to extract the reporting data of our mobile apps from Google Play Console.
According to this documentation, we have to use SignedJwtAssertionCredentials to authenticate. After some errors and research it seems that the code snippet in the doc is kind of outdated (since SignedJWTAssertionCredentials is not available anymore). Instead we tried the following to automatically download the reports:
from oauth2client.service_account import ServiceAccountCredentials
from apiclient.discovery import build
scopes = ['https://www.googleapis.com/auth/analytics.readonly']
key_file_location = 'files/access_token/mykeyfile.json'
credentials = ServiceAccountCredentials.from_json_keyfile_name(key_file_location, scopes)
cloud_storage_bucket = 'pubsite_prod_rev_123456789'
report_to_download = 'installs/installs_com.someapp.etc.etc_2021*'
storage = build('storage', 'v1', credentials=credentials)
print( storage.objects().get(bucket = cloud_storage_bucket,object = report_to_download).execute())
Now the problem is, that we receive the following error message:
Traceback (most recent call last):
File "C:\Users\dev\dev\z_10_ext_google_play_store.py", line 30, in <module>
print( storage.objects().get(bucket = cloud_storage_bucket,object = report_to_download).execute())
File "D:\Software\Python\lib\site-packages\googleapiclient\_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "D:\Software\Python\lib\site-packages\googleapiclient\http.py", line 938, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://storage.googleapis.com/storage/v1/b/pubsite_prod_rev_123456789/o/installs%2Finstalls_com.someapp.etc.etc_2021%2A?alt=json returned "Insufficient Permission". Details: "[{'message': 'Insufficient Permission', 'domain': 'global', 'reason': 'insufficientPermissions'}]">
Does anyone have an idea? Our service account even has admin rights in play console (which normally would be to much..)
UPDATE
As DaImTo pointed out, I was missing some scopes which lead me to a different line of scope definition:
scopes = ['https://www.googleapis.com/auth/analytics.readonly', 'https://www.googleapis.com/auth/devstorage.read_only'
,'https://www.googleapis.com/auth/cloud-platform.read_only']
But I still receive some errors, now because of the missing access token which is weird because the .json file contains everything necessary:
Traceback (most recent call last):
File "C:\Users\andre\Desktop\DAWSE\z_10_ext_google_play_store.py", line 33, in <module>
print( storage.objects().get(bucket = cloud_storage_bucket,object = report_to_download).execute())
File "D:\Sonstige Programme\Python\lib\site-packages\googleapiclient\_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "D:\Sonstige Programme\Python\lib\site-packages\googleapiclient\http.py", line 923, in execute
resp, content = _retry_request(
File "D:\Sonstige Programme\Python\lib\site-packages\googleapiclient\http.py", line 191, in _retry_request
resp, content = http.request(uri, method, *args, **kwargs)
File "D:\Sonstige Programme\Python\lib\site-packages\oauth2client\transport.py", line 159, in new_request
credentials._refresh(orig_request_method)
File "D:\Sonstige Programme\Python\lib\site-packages\oauth2client\client.py", line 749, in _refresh
self._do_refresh_request(http)
File "D:\Sonstige Programme\Python\lib\site-packages\oauth2client\client.py", line 785, in _do_refresh_request
self.access_token = d['access_token']
KeyError: 'access_token'

You appear to be using the objects.get method
The authenticated user must have sufficient permission to use this method. To start with i would ensure your service account has access.
After that i would check your scopes you are currently only loading the scope for google analytics api.
scopes = ['https://www.googleapis.com/auth/analytics.readonly']
This scope will not grant you access to the google cloud storage objects get method. For that you need to use one of the google cloud storage scopes

How to read data from Google Sheets in Python

I have a Google Sheet https://docs.google.com/spreadsheets/d/1Ycg7zTxds9DZnDvTrFcyNNKuTUxg6Yy6WF0a8Wc02WQ/edit#gid=0 I can read only. I need to read it in Python. I uploaded a video of what I am doing https://youtu.be/4cK0g9Bm_7w
import gspread
from oauth2client.service_account import ServiceAccountCredentials
scope = ["https://docs.google.com/spreadsheets/d/1Ycg7zTxds9DZnDvTrFcyNNKuTUxg6Yy6WF0a8Wc02WQ/edit#gid=0",
"https://docs.google.com/spreadsheets/d/1Ycg7zTxds9DZnDvTrFcyNNKuTUxg6Yy6WF0a8Wc02WQ/edit?usp=sharing"]
creds = ServiceAccountCredentials.from_json_keyfile_name("creds.json", scope)
client = gspread.authorize(creds)
sheet = client.open("[XO] developer test data").sheet1
I have an error:
Traceback (most recent call last):
File "C:\Python379\lib\site-packages\google\oauth2\_client.py", line 196, in jwt_grant
access_token = response_data["access_token"]
KeyError: 'access_token'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "E:/Прогрммирование/Последние алгоритмы/Git/GoogleSheets/sheets.py", line 11, in <module>
sheet = client.open("[XO] developer test data").sheet1
File "C:\Python379\lib\site-packages\gspread\client.py", line 121, in open
self.list_spreadsheet_files(title),
File "C:\Python379\lib\site-packages\gspread\client.py", line 98, in list_spreadsheet_files
res = self.request('get', url, params=params).json()
File "C:\Python379\lib\site-packages\gspread\client.py", line 70, in request
headers=headers,
File "C:\Python379\lib\site-packages\requests\sessions.py", line 546, in get
return self.request('GET', url, **kwargs)
File "C:\Python379\lib\site-packages\google\auth\transport\requests.py", line 478, in request
self.credentials.before_request(auth_request, method, url, request_headers)
File "C:\Python379\lib\site-packages\google\auth\credentials.py", line 133, in before_request
self.refresh(request)
File "C:\Python379\lib\site-packages\google\oauth2\service_account.py", line 377, in refresh
request, self._token_uri, assertion
File "C:\Python379\lib\site-packages\google\oauth2\_client.py", line 199, in jwt_grant
six.raise_from(new_exc, caught_exc)
File "<string>", line 3, in raise_from
google.auth.exceptions.RefreshError: ('No access token in response.', {'id_token': 'eyJhbGciOiJSUzI1NiIsImtpZCI6IjE5ZmUyYTdiNjc5NTIzOTYwNmNhMGE3NTA3OTRhN2JkOWZkOTU5NjEiLCJ0eXAiOiJKV1QifQ.eyJhdWQiOiJodHRwczovL2RvY3MuZ29vZ2xlLmNvbS9zcHJlYWRzaGVldHMvZC8xWWNnN3pUeGRzOURabkR2VHJGY3lOTkt1VFV4ZzZZeTZXRjBhOFdjMDJXUS9lZGl0I2dpZD0wLGh0dHBzOi8vZG9jcy5nb29nbGUuY29tL3NwcmVhZHNoZWV0cy9kLzFZY2c3elR4ZHM5RFpuRHZUckZjeU5OS3VUVXhnNll5NldGMGE4V2MwMldRL2VkaXQ_dXNwPXNoYXJpbmciLCJhenAiOiJhY2MxMC0yMjhAcHJvamVjdC0xMC0zMTc0MDIuaWFtLmdzZXJ2aWNlYWNjb3VudC5jb20iLCJlbWFpbCI6ImFjYzEwLTIyOEBwcm9qZWN0LTEwLTMxNzQwMi5pYW0uZ3NlcnZpY2VhY2NvdW50LmNvbSIsImVtYWlsX3ZlcmlmaWVkIjp0cnVlLCJleHAiOjE2MjQxNjA4NTIsImlhdCI6MTYyNDE1NzI1MiwiaXNzIjoiaHR0cHM6Ly9hY2NvdW50cy5nb29nbGUuY29tIiwic3ViIjoiMTA4OTUzMjg5ODAwOTQ4MzEyOTgxIn0.RYEt9eTvaCXSRPH-ddjB4s5wCE60tt2w-ZgL2aaeBT8ZTyKtlDbpMKV1zG8XRkxqCLs_xde-TEwTZoRQ8OKuJGQIZr6U8mSmqjW7xlhz5cvDLL03o-muks06FbLsgAFE2yEow5EdFcEL8hKt8LGuFoQWuiNWNT-sd-Z-vJMG0-XC6twrYgYp59tdmxCCG3MEprKbtgHpwAWHPoWjybKkETGXVnv17DSV99LIXY9FIzQVV4FV4reBn_4sgcT4y6IcbHqe6hpt4joEDYBsVxImzKk7IJjJfsRCg-kGpfc1yOC831qbupMlF45H75XMq9WWmnrn3Su4YFIBTugUxrMpdA'})

Issues:
You are not providing valid scopes. A scope is not the URL of a file you want to access. A scope determines what actions an API client can make, like reading user's spreadsheets, or reading and writing user's spreadsheets; it doesn't refer to specific files. Here you can see a list of the existing scopes in Sheets API: Authorizing requests with OAuth 2.0.
oauth2client is deprecated. Consider using google-auth instead.
Code sample:
import gspread
from google.oauth2.service_account import Credentials
scopes = [
'https://www.googleapis.com/auth/spreadsheets' # Change according to your preferences
]
credentials = Credentials.from_service_account_file(
'creds.json',
scopes=scopes
)
client = gspread.authorize(creds)
sheet = client.open("[XO] developer test data").sheet1
Notes:
gspread is not officially supported by Google. Here is the official library. And here is the corresponding quickstart for Sheets API.
I am assuming that creds.json refers to valid service account credentials.
If you store creds.json to ~/.config/gspread/creds.json, you should be able to do this, according to the gspread documentation:
import gspread
client = gspread.service_account()
sheet = client.open("[XO] developer test data").sheet1
Reference:
google.oauth2.service_account module

SecretClient class in azure-keyvault-secrets python package throws unexpected errir

I've been trying to get my python script to retrieve a secret from an Azure key vault. For the authentication I've been using the ServicePrincipal credential and the Azure-keyvault-secrets package which comes with a SecretClient class. But it's throwing an error which might be because of any updates in the python package.
Could anyone point out what's going wrong?
from azure.common.credentials import ServicePrincipalCredentials
from azure.keyvault.secrets import SecretClient
CLIENT_ID '...' #app_id
TENANT_ID = '...'
KEY = '...'
credentials = ServicePrincipalCredentials(
client_id=CLIENT_ID,
secret=KEY,
tenant=TENANT_ID
)
vault_url = '...'
secret_name = "..."
secret_version = '...'
client = SecretClient(vault_url=vault_url, credential=credentials)
secret = client.get_secret(secret_name)
It gives the following error :
Traceback (most recent call last):
File "key-vault.py", line 23, in <module>
secret = client.get_secret("cromaprivate128")
File "/Users/rishavputatunda/Library/Python/3.7/lib/python/site-packages/azure/core/tracing/decorator.py", line 71, in wrapper_use_tracer
return func(*args, **kwargs)
File "/Users/rishavputatunda/Library/Python/3.7/lib/python/site-packages/azure/keyvault/secrets/_client.py", line 71, in get_secret
**kwargs
File "/Users/rishavputatunda/Library/Python/3.7/lib/python/site-packages/azure/keyvault/secrets/_shared/_generated/v7_0/operations/_key_vault_client_operations.py", line 1621, in get_secret
pipeline_response = self._client._pipeline.run(request, stream=False, **kwargs)
File "/Users/rishavputatunda/Library/Python/3.7/lib/python/site-packages/azure/core/pipeline/_base.py", line 208, in run
return first_node.send(pipeline_request) # type: ignore
File "/Users/rishavputatunda/Library/Python/3.7/lib/python/site-packages/azure/core/pipeline/_base.py", line 80, in send
response = self.next.send(request)
File "/Users/rishavputatunda/Library/Python/3.7/lib/python/site-packages/azure/core/pipeline/_base.py", line 80, in send
response = self.next.send(request)
File "/Users/rishavputatunda/Library/Python/3.7/lib/python/site-packages/azure/core/pipeline/_base.py", line 80, in send
response = self.next.send(request)
File "/Users/rishavputatunda/Library/Python/3.7/lib/python/site-packages/azure/core/pipeline/policies/_redirect.py", line 157, in send
response = self.next.send(request)
File "/Users/rishavputatunda/Library/Python/3.7/lib/python/site-packages/azure/core/pipeline/policies/_retry.py", line 418, in send
response = self.next.send(request)
File "/Users/rishavputatunda/Library/Python/3.7/lib/python/site-packages/azure/keyvault/secrets/_shared/challenge_auth_policy.py", line 67, in send
self._handle_challenge(request, challenge)
File "/Users/rishavputatunda/Library/Python/3.7/lib/python/site-packages/azure/keyvault/secrets/_shared/challenge_auth_policy.py", line 91, in _handle_challenge
access_token = self._credential.get_token(scope)
AttributeError: 'ServicePrincipalCredentials' object has no attribute 'get_token'

We cannot use the ServicePrincipalCredentials to create a SecretClient. Because the ServicePrincipalCredentials cannot provide the access token. If you want to access key vault, please try to use the Credentials provided by sdk azure.identity.
For example
1. Create a service principal with Azure CLI
az login
az ad sp create-for-rbac -n "MyApp" --sdk-auth
Set access policy
az keyvault set-policy -n <your-unique-keyvault-name> --spn <clientId-of-your-service-principal> --secret-permissions delete get list set --key-permissions create decrypt delete encrypt get list unwrapKey wrapKe
code
from azure.identity import ClientSecretCredential
from azure.keyvault.secrets import SecretClient
tenant_id="<your sp tenant>"
client_id="<your sp client id>"
client_secret="<your sp client secret>"
credential = ClientSecretCredential(tenant_id, client_id, client_secret)
secret_client = SecretClient(vault_url="https://my-key-vault.vault.azure.net/", credential=credential)
secret = secret_client.get_secret("secret-name")
print(secret.name)
print(secret.value)

winerror 10054 google calendar

I have a script that adds events to my calendar that worked fine until this morning. I haven't made any changes, but now it doesn't work.
This is my code:
from __future__ import print_function
import httplib2
import os
import sys
from apiclient import discovery
from oauth2client import client
from oauth2client import tools
from oauth2client.file import Storage
from subprocess import Popen
import datetime
# If modifying these scopes, delete your previously saved credentials
# at ~/.credentials/calendar-python-quickstart.json
SCOPES = 'https://www.googleapis.com/auth/calendar.readonly'
CLIENT_SECRET_FILE = 'client_secret.json'
APPLICATION_NAME = 'Google Calendar API Python Quickstart'
def get_credentials():
"""Gets valid user credentials from storage.
If nothing has been stored, or if the stored credentials are invalid,
the OAuth2 flow is completed to obtain the new credentials.
Returns:
Credentials, the obtained credential.
"""
home_dir = os.path.expanduser('~')
credential_dir = os.path.join(home_dir, '.credentials')
if not os.path.exists(credential_dir):
os.makedirs(credential_dir)
credential_path = os.path.join(credential_dir,
'calendar-python-quickstart.json')
store = Storage(credential_path)
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
credentials = tools.run(flow, store)
print('Storing credentials to ' + credential_path)
return credentials
def main():
"""Shows basic usage of the Google Calendar API.
Creates a Google Calendar API service object and outputs a list of the next
10 events on the user's calendar.
"""
try:
if(len(sys.argv) < 2):
print('No name was provided.')
input()
exit()
credentials = get_credentials()
http = credentials.authorize(httplib2.Http())
service = discovery.build('calendar', 'v3', http=http)
event = {
'summary': 'Teststop ' + " ".join(sys.argv[1:]),
'start': {
'date': str(datetime.date.today() + datetime.timedelta(days=14)),
'timeZone': 'Europe/Stockholm',
},
'end': {
'date': str(datetime.date.today() + datetime.timedelta(days=14)),
'timeZone': 'Europe/Stockholm',
},
}
event = service.events().insert(calendarId='CENSORED#group.calendar.google.com', body=event).execute()
print ('Event created: %s' % (event.get('htmlLink')))
p = Popen([r'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe', r'https://calendar.google.com/calendar/render#main_7%7Cmonth' ], stdin=None, stdout=None, stderr=None)
exit()
except Exception as e:
traceback.print_exc()
print(str(e))
input()
if __name__ == '__main__':
main()
This is the output I'm getting:
C:\Users\jhh\executor\scripts\google calendar>py addEvent.py test
Traceback (most recent call last):
File "addEvent.py", line 63, in main
service = discovery.build('calendar', 'v3', http=http)
File "C:\Python\lib\site-packages\oauth2client\_helpers.py", line 133, in posi
tional_wrapper
return wrapped(*args, **kwargs)
File "C:\Python\lib\site-packages\googleapiclient\discovery.py", line 229, in
build
requested_url, discovery_http, cache_discovery, cache)
File "C:\Python\lib\site-packages\googleapiclient\discovery.py", line 276, in
_retrieve_discovery_doc
resp, content = http.request(actual_url)
File "C:\Python\lib\site-packages\oauth2client\transport.py", line 175, in new
_request
redirections, connection_type)
File "C:\Python\lib\site-packages\oauth2client\transport.py", line 282, in req
uest
connection_type=connection_type)
File "C:\Python\lib\site-packages\httplib2\__init__.py", line 1322, in request
(response, content) = self._request(conn, authority, uri, request_uri, metho
d, body, headers, redirections, cachekey)
File "C:\Python\lib\site-packages\httplib2\__init__.py", line 1072, in _reques
t
(response, content) = self._conn_request(conn, request_uri, method, body, he
aders)
File "C:\Python\lib\site-packages\httplib2\__init__.py", line 995, in _conn_re
quest
conn.connect()
File "C:\Python\lib\http\client.py", line 1400, in connect
server_hostname=server_hostname)
File "C:\Python\lib\ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "C:\Python\lib\ssl.py", line 814, in __init__
self.do_handshake()
File "C:\Python\lib\ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "C:\Python\lib\ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "addEvent.py", line 88, in <module>
main()
File "addEvent.py", line 83, in main
traceback.print_exc()
NameError: name 'traceback' is not defined
I have no idea how to solve this. Any ideas?

OAuth2 InvalidScopeError in Backend Application flow

I am trying to implement the Backend Application flow to get an access token from the Datahug API (Following instructions in https://api.datahug.com/#gettingstarted).
from oauthlib.oauth2 import BackendApplicationClient
from requests_oauthlib import OAuth2Session
def get_access_token():
token_url = 'https://apps.datahug.com/identity/connect/token'
client_id = CLIENT_ID
client_secret = CLIENT_SECRET
scope = 'datahug_api'
client = BackendApplicationClient(client_id=client_id)
client.prepare_request_body(scope=[scope])
oauth = OAuth2Session(client=client)
token = oauth.fetch_token(token_url=token_url,
client_id=client_id,
client_secret=client_secret)
return token
if __name__ == '__main__':
token = get_access_token()
print(token)
When running this code I'm getting an InvalidScopeError, namely
user:dh user$ python so_test.py
Traceback (most recent call last):
File "so_test.py", line 21, in <module>
token = get_access_token()
File "so_test.py", line 17, in get_access_token
client_secret=client_secret)
File "/Users/user/anaconda3/lib/python3.5/site-packages/requests_oauthlib/oauth2_session.py", line 244, in fetch_token
self._client.parse_request_body_response(r.text, scope=self.scope)
File "/Users/user/anaconda3/lib/python3.5/site-packages/oauthlib/oauth2/rfc6749/clients/base.py", line 409, in parse_request_body_response
self.token = parse_token_response(body, scope=scope)
File "/Users/user/anaconda3/lib/python3.5/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 376, in parse_token_response
validate_token_parameters(params)
File "/Users/user/anaconda3/lib/python3.5/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 383, in validate_token_parameters
raise_from_error(params.get('error'), params)
File "/Users/user/anaconda3/lib/python3.5/site-packages/oauthlib/oauth2/rfc6749/errors.py", line 325, in raise_from_error
raise cls(**kwargs)
oauthlib.oauth2.rfc6749.errors.InvalidScopeError: (invalid_scope)
It seems that the problem is in the value scope='datahug_api', but this is the value suggested in the datahug api. Any suggestions on how to solve this much appreciated.

You are missing scope in the oauth.fetch_token().
So, your variable token should be:
token = oauth.fetch_token(token_url=token_url,scope=scope,client_id=client_id,
client_secret=client_secret)

I had success adding the scope to the BackendApplicationClient as can be seen from the developer docs.
client = BackendApplicationClient(client_id=client_id, scope=scope)
oauth = OAuth2Session(client=client)
token = oauth.fetch_token(token_url=url, client_id=client_id, client_secret=client_secret)