I'm looking to do the equivalent of the following in Python with out having to call these command using something like os.system and look at the output.
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/credentials.json"
export PROJECT_ID="my-project-name"
gcloud auth application-default print-access-token
Can this be done with Google SDK?
I think you can do this using the Google Auth Library. You can install it with pip:
$ pip install google-auth
Here's an example code using buckets:
from google.oauth2 import service_account
from google.cloud import storage
KEY='/path/to/key.json'
PROJECT='your_project_id'
# gcloud auth application-default print-access-token is no necessary
credentials = service_account.Credentials.from_service_account_file(KEY)
# Initialize the Cloud Storage client using the credentials
storage_client = storage.Client(PROJECT,credentials)
# List objects in a bucket
blobs = storage_client.list_blobs("a_bucket")
for blob in blobs:
print(blob.name)
Good luck coding!
Here the answer:
import google.auth
import google.auth.transport.requests
from google.oauth2 import service_account
from os.path import expanduser
from os import getenv
# The the service account key ABSOLUTE path from env or current folder
service_account_key = getenv(
'GOOGLE_APPLICATION_CREDENTIALS',
f'{expanduser(".")}/service-account-key.json'
)
# Creates a credentials object from the service account file
credentials = service_account.Credentials.from_service_account_file(
service_account_key, # key path
scopes=['https://www.googleapis.com/auth/cloud-platform'] # scopes
)
# Prepare an authentication request
auth_req = google.auth.transport.requests.Request()
# Request refresh tokens
credentials.refresh(auth_req)
# now we can print the access token
print(credentials.token)
Using python:
import request
requests.get('http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token').json().get('access_token')
Related
I am trying to access a secret stored in secrets manager.
I created a service account with owner role. I created a key from it. I run:
import os
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = './keyfile.json'
from google.cloud import secretmanager
secret_client = secretmanager.SecretManagerServiceClient()
secret_name = f'projects/{project_id}/secrets/{secret_id}/versions/{version_id}'
response = secret_client.access_secret_version(request={"name": secret_name})
but I get:
google.api_core.exceptions.PermissionDenied: 403 Permission 'secretmanager.versions.access'
denied for resource 'projects/myprojnumber/secrets/mysecret/versions/1' (or it may not exist).
I checked the secret_name was the same as the secret's value in secret manager.
I have tried adding Secret Manager Secret Accessor and Secret Manager Viewer roles.
Edit: running this from cloud shell.
I think the issue is that the code is taking the Default Credentials of the Cloud Shell instead of using your SA key.
You can specify the credentials when creating the client
from google.cloud import secretmanager
from google.oauth2 import service_account
credentials = service_account.Credentials.from_service_account_file("./keyfile.json")
secret_client = secretmanager.SecretManagerServiceClient(credentials=credentials)
secret_name = f'projects/{project_id}/secrets/{secret_id}/versions/{version_id}'
response = secret_client.access_secret_version(request={"name": secret_name})
Another option using some of the methods found in the library docs:
from google.cloud import secretmanager
secret_client = secretmanager.SecretManagerServiceClient.from_service_account_file("./keyfile.json")
secret_name = f'projects/{project_id}/secrets/{secret_id}/versions/{version_id}'
response = secret_client.access_secret_version(request={"name": secret_name})
Just as an advice, being newbie does not mean you cannot Google a little more to search for something like how to use a SA as credential for the client of the library you're using.
For example you could easily find this doc which shows a sample.
Anyway, good luck with GCP!
I am trying to initialize the Firebase Admin SDK within a Cloud Run application, using a separate service account (i.e not the default service account).
The documentation suggests:
import firebase_admin
from firebase_admin import credentials
cred = credentials.Certificate("path/to/serviceAccountKey.json")
firebase_admin.initialize_app(cred)
However, I would like to avoid packaging secrets into the Cloud Run container, so I am retrieving the json file from Secret Manager, and trying to create the credentials, and pass it into: firebase_admin.initialize_app(cred)
import firebase_admin
from google.cloud import secretmanager
from google.oauth2 import service_account
# Create credentials object then initialize the firebase admin client
sec_client = secretmanager.SecretManagerServiceClient()
name = sec_client.secret_version_path(GOOGLE_CLOUD_PROJECT_NUMBER, FIREBASE_SA_SECRET_NAME, "latest")
response = sec_client.access_secret_version(name)
service_account_info = json.loads(response.payload.data.decode('UTF-8'))
creds = service_account.Credentials.from_service_account_info(service_account_info)
firebase_admin.initialize_app(creds)
Error received:
ValueError: Illegal Firebase credential provided. App must be
initialized with a valid credential instance.
Any tips are appreciated.
import firebase_admin
from google.cloud import secretmanager
from google.oauth2 import service_account
# Create credentials object then initialize the firebase admin client
sec_client = secretmanager.SecretManagerServiceClient()
name = sec_client.secret_version_path(GOOGLE_CLOUD_PROJECT_NUMBER, FIREBASE_SA_SECRET_NAME, "latest")
response = sec_client.access_secret_version(name)
service_account_info = json.loads(response.payload.data.decode('utf-8'))
# build credentials with the service account dict
creds = firebase_admin.credentials.Certificate(service_account_info)
# initialize firebase admin
firebase_app = firebase_admin.initialize_app(creds)
I'm trying to access a secure cloud run service. If I try from my machine with the SDK it works fine:
curl --header "Authorization: Bearer $(gcloud auth print-identity-token)"
However, I can not get it to work using the Python API using the same service account, I've tried using google-auth to get an access token but that gives me a 401 authentication error. This is the code I am using to try to get a token:
import google.auth
import google.auth.transport.requests
scopes = ['https://www.googleapis.com/auth/cloud-platform']
creds, projects = google.auth.default(scopes=scopes)
auth_req = google.auth.transport.requests.Request()
creds.refresh(auth_req)
# then using creds.token
Looking at the docs: https://cloud.google.com/run/docs/authenticating/service-to-service#calling_from_outside_gcp it says to follow the sample code here: https://cloud.google.com/iap/docs/authentication-howto#iap_make_request-python I can't seem to follow the guide as it says to enable IAP but it seems that IAP is only for app engine and not cloud run?
Has anyone go any advice on this one?
Thanks
OK it seems that there is an IDTokenCredentials class that works for this as it uses Open ID Connect ID Tokens instead of OAuth 2.0 Access Tokens:
https://google-auth.readthedocs.io/en/latest/reference/google.oauth2.service_account.html
from google.oauth2 import service_account
from google.auth.transport.requests import AuthorizedSession
service_url = 'example.com'
key_file = 'key.json'
credentials = service_account.IDTokenCredentials.from_service_account_file(
key_file, target_audience=service_url)
authed_session = AuthorizedSession(credentials)
response = authed_session.get(service_url)
It's confusing as I don't see it in the docs and it's leading me to something else about IAP which I don't think is working with Cloud Run.
If you still want to obtain your ID token and not use method from Dan's response there is an updated code:
import requests
import google.auth
from google.oauth2 import service_account
from google.auth.transport.requests import AuthorizedSession
service_url = 'example.com'
key_file = 'key.json'
credentials = service_account.IDTokenCredentials.from_service_account_file(
key_file, target_audience=service_url)
request = google.auth.transport.requests.Request()
credentials.refresh(request)
token = credentials.token
print('ID Token:', token)
I only know how to do it with the current supported libraries - but now I need to do some work with google cloud Filestore and there is no library for it yet.
Anyone can help me out how to create a session with Python using service account?
Thanks!
Since Filestore API is discovery based you can use google-api-python-client. It is listed as supported. Alternatively you can use google-auth. Here is an example to list instances:
from google.oauth2 import service_account
from google.auth.transport.requests import AuthorizedSession
scopes = ['https://www.googleapis.com/auth/cloud-platform']
credentials = service_account.Credentials.from_service_account_file(
'/path/to/your/service_account_key.json', scopes=scopes)
authed_session = AuthorizedSession(credentials)
response = authed_session.get(
'https://file.googleapis.com/v1beta1/projects/[YOUR_PROJECT]/locations/[LOCATION]/instances')
print response.json()
You are able to manage data for Cloud Filestore from the gcloud tools
You can authenticate the gcloud tool with a service account as described here
I'm trying to use PyDrive to upload files to Google Drive using a local Python script which I want to automate so it can run every day via a cron job. I've stored the client OAuth ID and secret for the Google Drive app in a settings.yaml file locally, which PyDrive picks up to use for authentication.
The problem I'm getting is that although this works some of the time, every so often it decides it needs me to provide a verification code (if I use CommandLineAuth), or it takes me to a browser to enter the Google account password (LocalWebserverAuth), so I can't automate the process properly.
Anybody know which settings I need to tweak - either in PyDrive or on the Google OAuth side - in order to set this up once and then trust it to run automatically without further user input in future?
Here's what the settings.yaml file looks like:
client_config_backend: settings
client_config:
client_id: MY_CLIENT_ID
client_secret: MY_CLIENT_SECRET
save_credentials: True
save_credentials_backend: file
save_credentials_file: credentials.json
get_refresh_token: False
oauth_scope:
- https://www.googleapis.com/auth/drive.file
You can (should) create a service account - with an id and private key from the google API console - this won't require re verification but you'll need to keep the private key private.
Create a credential object based on the google python example and assign it to the PyDrive GoogleAuth() object:
from apiclient.discovery import build
from oauth2client.client import SignedJwtAssertionCredentials
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
# from google API console - convert private key to base64 or load from file
id = "...#developer.gserviceaccount.com"
key = base64.b64decode(...)
credentials = SignedJwtAssertionCredentials(id, key, scope='https://www.googleapis.com/auth/drive')
credentials.authorize(httplib2.Http())
gauth = GoogleAuth()
gauth.credentials = credentials
drive = GoogleDrive(gauth)
EDIT (Sep 2016): For the latest integrated google-api-python-client (1.5.3) you would use the following code, with id and key the same as before:
import StringIO
from apiclient import discovery
from oauth2client.service_account import ServiceAccountCredentials
credentials = ServiceAccountCredentials.from_p12_keyfile_buffer(id, StringIO.StringIO(key), scopes='https://www.googleapis.com/auth/drive')
http = credentials.authorize(httplib2.Http())
drive = discovery.build("drive", "v2", http=http)