Develop Reference

Access CosmosDB Data from Azure App Service by using managed identity (Failed)

A FastAPI-based API written in Python has been deployed as an Azure App Service. The API needs to read and write data from CosmosDB, and I attempted to use Managed Identity for this purpose, but encountered an error, stating Unrecognized credential type
These are the key steps that I took towards that goal
Step One: I used Terraform to configure the managed identity for Azure App Service, and assigned the 'contributor' role to the identity so that it can access and write data to CosmosDB. The role assignment was carried out in the file where the Azure App Service is provisioned.
resource "azurerm_linux_web_app" "this" {
name = var.appname
location = var.location
resource_group_name = var.rg_name
service_plan_id = azurerm_service_plan.this.id
app_settings = {
"PROD" = false
"DOCKER_ENABLE_CI" = true
"DOCKER_REGISTRY_SERVER_URL" = data.azurerm_container_registry.this.login_server
"WEBSITE_HTTPLOGGING_RETENTION_DAYS" = "30"
"WEBSITE_ENABLE_APP_SERVICE_STORAGE" = false
}
lifecycle {
ignore_changes = [
app_settings["WEBSITE_HTTPLOGGING_RETENTION_DAYS"]
]
}
https_only = true
identity {
type = "SystemAssigned"
}
data "azurerm_cosmosdb_account" "this" {
name = var.cosmosdb_account_name
resource_group_name = var.cosmosdb_resource_group_name
}
// built-in role that allow the app-service to read and write to an Azure Cosmos DB
resource "azurerm_role_assignment" "cosmosdbContributor" {
scope = data.azurerm_cosmosdb_account.this.id
principal_id = azurerm_linux_web_app.this.identity.0.principal_id
role_definition_name = "Contributor"
}
Step Two: I used the managed identity library to fetch the necessary credentials in the Python code.
from azure.identity import ManagedIdentityCredential
from azure.cosmos.cosmos_client import CosmosClient
client = CosmosClient(get_endpoint(),credential=ManagedIdentityCredential())
client = self._get_or_create_client()
database = client.get_database_client(DB_NAME)
container = database.get_container_client(CONTAINER_NAME)
container.query_items(query)
I received the following error when running the code locally and from Azure (the error can be viewed from the Log stream of the Azure App Service):
raise TypeError(
TypeError: Unrecognized credential type. Please supply the master key as str, or a dictionary or resource tokens, or a list of permissions.
Any help or discussion is welcome

If you are using the Python SDK, you can directly do this ,check the sample here
aad_credentials = ClientSecretCredential(
tenant_id="<azure-ad-tenant-id>",
client_id="<client-application-id>",
client_secret="<client-application-secret>")
client = CosmosClient("<account-endpoint>", aad_credentials)

Update/Add labels to Google Kubernetes Engine Cluster Workloads through Python API

Has anyone tried to add or update the Clusters from Google Kubernetes Engine through Python API?
I managed to do this for Compute instances, but the guide for Kubernetes Engine says its deprecated:
https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1/projects.zones.clusters.nodePools/update
Tried it and it fails saying it does not find "labels":
googleapiclient.errors.HttpError: <HttpError 400 when requesting
https://container.googleapis.com/v1/projects/testingproject/zones/us-east1/clusters/testing-cluster/resourceLabels?alt=json
returned "Invalid JSON payload received. Unknown name "labels": Cannot
find field.". Details: "[{'#type':
'type.googleapis.com/google.rpc.BadRequest', 'fieldViolations':
[{'description': 'Invalid JSON payload received. Unknown name
"labels": Cannot find field.'}]}]">
My code is this:
credentials = GoogleCredentials.get_application_default()
service = discovery.build('container', 'v1', credentials=credentials)
# Deprecated. The Google Developers Console [project ID or project
# number](https://developers.google.com/console/help/new/#projectnumber).
# This field has been deprecated and replaced by the name field.
project_id = 'testingproject' # TODO: Update placeholder value.
# Deprecated. The name of the Google Compute Engine
# [zone](/compute/docs/zones#available) in which the cluster
# resides.
# This field has been deprecated and replaced by the name field.
zone = 'us-east1' # TODO: Update placeholder value.
# Deprecated. The name of the cluster.
# This field has been deprecated and replaced by the name field.
cluster_id = 'testing-cluster' # TODO: Update placeholder value.
set_labels_request_body = {
'labels': 'value'
}
request = service.projects().zones().clusters().resourceLabels(projectId=project_id, zone=zone, clusterId=cluster_id, body=set_labels_request_body)
response = request.execute()
# TODO: Change code below to process the `response` dict:
pprint(response)
I want to update the Workload named 'matei-testing-2000-gke-ops' inside the cluster 'testing-cluster'.
Any ideas?
Thank you
Update: It does not find the labels because the name is resourceLabels. But I get the following error after:
googleapiclient.errors.HttpError: <HttpError 400 when requesting
https://container.googleapis.com/v1/projects//zones//clusters//resourceLabels?alt=json
returned "Invalid value at 'resource_labels'
(type.googleapis.com/google.container.v1.SetLabelsRequest.ResourceLabelsEntry),
"value"". Details: "[{'#type':
'type.googleapis.com/google.rpc.BadRequest', 'fieldViolations':
[{'field': 'resource_labels', 'description': 'Invalid value at
'resource_labels'
(type.googleapis.com/google.container.v1.SetLabelsRequest.ResourceLabelsEntry),
"value"'}]}]">

I've not now tried this.
But IIUC, you'll need to:
ditch (or use defaults) for e.g. project_id, zone and cluster_id parameters of resourceLabels
add name to your body and it should be of the form: projects/*/locations/*/clusters/*
i.e.
import os
from googleapiclient import discovery
from oauth2client.client import GoogleCredentials
credentials = GoogleCredentials.get_application_default()
service = discovery.build('container', 'v1', credentials=credentials)
PROJECT = os.getenv("PROJECT")
LOCATION = os.getenv("ZONE")
CLUSTER = os.getenv("CLUSTER")
NAME = "projects/{project}/locations/{location}/clusters/{cluster}".format(
project=project_id,
location=zone,
cluster=cluster_id)
# To update `resourceLabels` you must first fingerprint them
# To get the current `labelFingerprint`, you must `get` the cluster
body = {
'name': NAME,
}
request = service.projects().zones().clusters().get(
projectId=project_id,
zone=zone,
clusterId=cluster_id)
response = request.execute()
labelFingerprint = response["labelFingerprint"]
if "resourceLabels" in response:
print("Existing labels")
resourceLabels = response["resourceLabels"]
else:
print("No labels")
resourceLabels = {}
# Add|update a label
resourceLabels["dog"] = "freddie"
# Construct `resourceLabels` request
body = {
'name': NAME,
'resourceLabels': resourceLabels,
'labelFingerprint': labelFingerprint,
}
request = service.projects().zones().clusters().resourceLabels(
projectId=project_id,
zone=zone,
clusterId=cluster_id,
body=body)
# Do something with the `response`
response = request.execute()
clusters.get
`clusters#CLuster
And
gcloud container clusters describe ${CLUSTER} \
--zone=${ZONE} \
--project=${PROJECT} \
--format="value(labelFingerprint,resourceLabels)"
Before:
a9dc16a7
After:
b2c32ec0 dog=freddie

Creating VMs from Instance Template on Cloud Function via API call

The code I've written seems to be what I need, however it doesn't work and I get a 401 error (authentication) I've tried everything: 1. Service account permissions 2. create secret id and key (not sure how to use those to get access token though) 3. Basically, tried everything for the past 2 days.
import requests
from google.oauth2 import service_account
METADATA_URL = 'http://metadata.google.internal/computeMetadata/v1/'
METADATA_HEADERS = {'Metadata-Flavor': 'Google'}
SERVICE_ACCOUNT = [NAME-OF-SERVICE-ACCOUNT-USED-WITH-CLOUD-FUNCTION-WHICH-HAS-COMPUTE-ADMIN-PRIVILEGES]
def get_access_token():
url = '{}instance/service-accounts/{}/token'.format(
METADATA_URL, SERVICE_ACCOUNT)
# Request an access token from the metadata server.
r = requests.get(url, headers=METADATA_HEADERS)
r.raise_for_status()
# Extract the access token from the response.
access_token = r.json()['access_token']
return access_token
def start_vms(request):
request_json = request.get_json(silent=True)
request_args = request.args
if request_json and 'number_of_instances_to_create' in request_json:
number_of_instances_to_create = request_json['number_of_instances_to_create']
elif request_args and 'number_of_instances_to_create' in request_args:
number_of_instances_to_create = request_args['number_of_instances_to_create']
else:
number_of_instances_to_create = 0
access_token = get_access_token()
address = "https://www.googleapis.com/compute/v1/projects/[MY-PROJECT]/zones/europe-west2-b/instances?sourceInstanceTemplate=https://www.googleapis.com/compute/v1/projects/[MY-PROJECT]/global/instanceTemplates/[MY-INSTANCE-TEMPLATE]"
headers = {'token': '{}'.format(access_token)}
for i in range(1,number_of_instances_to_create):
data = {'name': 'my-instance-{}'.format(i)}
r = requests.post(address, data=data, headers=headers)
r.raise_for_status()
print("my-instance-{} created".format(i))
Any advice/guidance? If someone could tell me how to get an access token using secret Id and key. Also, I'm not too sure if OAuth2.0 will work because I essentially want to turn these machines on, and they do some processing and then self destruct. So there is no user involvement to allow access. If OAuth2.0 is the wrong way to go about it, what else can I use?
I tried using gcloud, but subprocess'ing gcloud commands aren't recommended.

I did something similar to this, though I used the Node 10 Firebase Functions runtime, but should be very similar never-the-less.
I agree that OAuth is not the correct solution since there is no user involved.
What you need to use is 'Application Default Credentials' which is based on the permissions available to your cloud functions' default service account which will be the one labelled as "App Engine default service account" here:
https://console.cloud.google.com/iam-admin/serviceaccounts?folder=&organizationId=&project=[YOUR_PROJECT_ID]
(For my project that service account already had the permissions necessary for starting and stopping GCE instances, but for other API's I have grant it permissions manually.)
ADC is for server-to-server API calls. To use it I called google.auth.getClient (of the Google APIs Auth Library) with just the scope, ie. "https://www.googleapis.com/auth/cloud-platform".
This API is very versatile in that it returns whatever credentials you need, so when I am running on cloud functions it returns a 'Compute' object and when I'm running in the emulator it gives me a "UserRefreshClient" object.
I then include that auth object in my call to compute.instances.insert() and compute.instances.stop().
Here the template I used for testing my code...
{
name: 'base',
description: 'Temporary instance used for testing.',
tags: { items: [ 'test' ] },
machineType: `zones/${zone}/machineTypes/n1-standard-1`,
disks: [
{
autoDelete: true, // you will want this!
boot: true,
type: 'PERSISTENT',
initializeParams: {
diskSizeGb: '10',
sourceImage: "projects/ubuntu-os-cloud/global/images/ubuntu-minimal-1804-bionic-v20190628",
}
}
],
networkInterfaces: [
{
network: `https://www.googleapis.com/compute/v1/projects/${projectId}/global/networks/default`,
accessConfigs: [
{
name: 'External NAT',
type: 'ONE_TO_ONE_NAT'
}
]
}
],
}
Hope that helps.

If you’re getting a 401 error that means that the access token you're using is either expired or invalid.
This guide will be able to show you how to request OAuth 2.0 access tokens and make API calls using a Service Account: https://developers.google.com/identity/protocols/OAuth2ServiceAccount
The .json file mentioned is the private key you create in IAM & Admin under your service account.

Trying to create Azure B2C user with azure-graphrbac

We are trying to import our existing users into our B2C tenant. For this, we have been trying to use the azure-graphrbac python library.
I have followed this guide to register an application to be used with the graph api.
I'm using the below code to try and create a user:
from azure.graphrbac import GraphRbacManagementClient
from azure.common.credentials import ServicePrincipalCredentials
from azure.graphrbac.models import UserCreateParameters, PasswordProfile
credentials = ServicePrincipalCredentials(
client_id="<CLIENT ID>",
secret="<SECRET>",
tenant="<TENANT ID>"
)
tenant_id = '<myb2ctenant>.onmicrosoft.com'
graphrbac_client = GraphRbacManagementClient(
credentials,
tenant_id
)
ucp = UserCreateParameters(
user_principal_name="my#mail.com",
account_enabled=True,
display_name='Martin T',
mail_nickname='<mymail>',
additional_properties={
"signInNames": [{"type": "emailAddress", "value": "<mymail>"}]
},
user_type="LocalAccount",
password_profile=PasswordProfile(
password='<somepassword>',
force_change_password_next_login=True
)
)
user = graphrbac_client.users.create(ucp)
I've made sure that the client id, secret and tenant id are correct. However, I keep getting this error:
GraphErrorException: Access Token missing or malformed.
Does anyone have an idea as to what I might be doing wrong?

As Laurent said, you need define resource. The default resource is https://management.core.windows.net/. In your scenario, you want to create a user, the resource is https://graph.windows.net.
Your code also has some mistake, I modify it. The following code works for me.
from azure.graphrbac import GraphRbacManagementClient
from azure.common.credentials import ServicePrincipalCredentials
from azure.graphrbac.models import UserCreateParameters, PasswordProfile
credentials = ServicePrincipalCredentials(
client_id="",
secret="",
resource="https://graph.windows.net",
tenant = ''
)
tenant_id = ''
graphrbac_client = GraphRbacManagementClient(
credentials,
tenant_id
)
ucp = UserCreateParameters(
user_principal_name="",
account_enabled=True,
display_name='Martin T',
##I test in my lab, if I use this line, I will get error log and could not create a user.
#additional_properties={
# "signInNames": [{"type": "emailAddress", "value": ""}]
#},
##user_type only support Member or Guest, see this link https://learn.microsoft.com/en-us/python/api/azure.graphrbac.models.usercreateparameters?view=azure-python
user_type="Member",
mail_nickname = 'shuitest',
password_profile=PasswordProfile(
password='',
force_change_password_next_login=True
)
)
user = graphrbac_client.users.create(ucp)
See SDK in this link.

You service principal authentication needs to define "resource":
https://learn.microsoft.com/en-us/python/api/overview/azure/activedirectory
credentials = UserPassCredentials(
'user#domain.com', # Your user
'my_password', # Your password
resource="https://graph.windows.net"
)

How can I change the owner of a Google Sheets spreadsheet?

With the following, I can programmatically create a spreadsheet in Google sheets, but the owner of the sheet is the developer account (a crazy string ending in "gserviceaccount.com"), and my normal account can't view the spreadsheet. What else do I need to do in order to add Google users to the read/write permissions?
from oauth2client.service_account import ServiceAccountCredentials
from googleapiclient import discovery
# ... json_key is the json blob that has the credentials
scope = ['https://spreadsheets.google.com/feeds']
credentials = ServiceAccountCredentials.from_json_keyfile_dict(json_key, scope)
service = discovery.build('sheets', 'v4', credentials=credentials)
spreadsheet = {
"properties": {"title": "my test spreadsheet"}
}
service.spreadsheets().create(body=spreadsheet).execute()
Edit:
I tried changing the scope to ['https://www.googleapis.com/auth/drive'] but the answer below still doesn't work for me. When I run
print [xx for xx in dir(service) if not xx.startswith('_')]
I get
['new_batch_http_request', u'spreadsheets']
In other words, permissions() isn't a method in service as I have service defined. What should I be doing differently?

I figured it out from reading the comment left by Chris. All that was missing from his comments is you do in fact need to use particular scopes in his drive_service. Notice the changes in scope I use to build the different objects:
from oauth2client.service_account import ServiceAccountCredentials
from googleapiclient.discovery import build
key = '/path/to/service_account.json'
# Build 'Spreadsheet' object
spreadsheets_scope = [ 'https://www.googleapis.com/auth/spreadsheets' ]
sheets_credentials = ServiceAccountCredentials.from_json_keyfile_name(key, spreadsheets_scope)
sheets_service = build('sheets', 'v4', credentials=sheets_credentials)
# returns 'Spreadsheet' dict
# https://developers.google.com/sheets/api/reference/rest/v4/spreadsheets#resource-spreadsheet
spreadsheet = sheets_service.spreadsheets().create(
body={
"properties": {
'title': 'spreadsheets test',
},
"sheets": [],
}
).execute()
# id for the created file
spreadsheetId = spreadsheet['spreadsheetId']
# url of your file
spreadsheetUrl = spreadsheet['spreadsheetUrl']
# Build 'Permissions' object
drive_scope = [ 'https://www.googleapis.com/auth/drive' ]
drive_credentials = ServiceAccountCredentials.from_json_keyfile_name(key, drive_scope)
drive_service = build('drive', 'v3', credentials=drive_credentials)
# returns 'Permissions' dict
permissions = drive_service.permissions().create(
fileId=spreadsheetId,
transferOwnership=True,
body={
'type': 'user',
'role': 'owner',
'emailAddress': 'example#email.com',
}
).execute()
# apply permission
drive_service.files().update(
fileId=spreadsheetId,
body={'permissionIds': [permissions['id']]}
).execute()
print ('\nOpen me:\n\n%s\n' % spreadsheetUrl)
So the logic is, a 'Spreadsheet Resource' is made from build with all its properties and sheet data, with the owner set to your service account. Next, a 'Drive Resource' is made, this is the Resource with the permissions() method. execute() returns a newly created permissions id used to update() the spreadsheet file.

Service is just a generic name for the result of a discovery.build call. In this case, not having the 'permissions' method is just that its not available on the same service. The following code should be sufficient if changing owner isn't required. To add someone with read and write access, the following works for me:
def grant_permissions(spreadsheet_id, writer):
drive_service = discovery.build('drive', 'v3')
permission = drive_service.permissions().create(
fileId=spreadsheet_id,
body={
'type': 'user',
'role': 'writer',
'emailAddress': writer,
}
).execute()
drive_service.files().update(
fileId=spreadsheet_id,
body={'permissionIds': [permission['id']]}
).execute()
To actually change the owner, the transfer ownership flag must be set:
def change_owner(spreadsheet_id, writer):
drive_service = discovery.build('drive', 'v3')
permission = drive_service.permissions().create(
fileId=spreadsheet_id,
transferOwnership=True,
body={
'type': 'user',
'role': 'owner',
'emailAddress': writer,
}
).execute()
drive_service.files().update(
fileId=spreadsheet_id,
body={'permissionIds': [permission['id']]}
).execute()
The service account being used must have the right permissions though. I believe the ones that worked for me was checking the g suite box when first creating the service account.

Try to use the method Permissions: insert from the documentation. You will be able to insert a permission for a file or a Team Drive.
Here is the sample code provided from the documentation:
from apiclient import errors
# ...
def insert_permission(service, file_id, value, perm_type, role):
"""Insert a new permission.
Args:
service: Drive API service instance.
file_id: ID of the file to insert permission for.
value: User or group e-mail address, domain name or None for 'default'
type.
perm_type: The value 'user', 'group', 'domain' or 'default'.
role: The value 'owner', 'writer' or 'reader'.
Returns:
The inserted permission if successful, None otherwise.
"""
new_permission = {
'value': value,
'type': perm_type,
'role': role
}
try:
return service.permissions().insert(
fileId=file_id, body=new_permission).execute()
except errors.HttpError, error:
print 'An error occurred: %s' % error
return None
Use Try it now to test live data and see the API request and response.
For further reading, check this SO post.