Develop Reference

Access CosmosDB Data from Azure App Service by using managed identity (Failed)

A FastAPI-based API written in Python has been deployed as an Azure App Service. The API needs to read and write data from CosmosDB, and I attempted to use Managed Identity for this purpose, but encountered an error, stating Unrecognized credential type
These are the key steps that I took towards that goal
Step One: I used Terraform to configure the managed identity for Azure App Service, and assigned the 'contributor' role to the identity so that it can access and write data to CosmosDB. The role assignment was carried out in the file where the Azure App Service is provisioned.
resource "azurerm_linux_web_app" "this" {
name = var.appname
location = var.location
resource_group_name = var.rg_name
service_plan_id = azurerm_service_plan.this.id
app_settings = {
"PROD" = false
"DOCKER_ENABLE_CI" = true
"DOCKER_REGISTRY_SERVER_URL" = data.azurerm_container_registry.this.login_server
"WEBSITE_HTTPLOGGING_RETENTION_DAYS" = "30"
"WEBSITE_ENABLE_APP_SERVICE_STORAGE" = false
}
lifecycle {
ignore_changes = [
app_settings["WEBSITE_HTTPLOGGING_RETENTION_DAYS"]
]
}
https_only = true
identity {
type = "SystemAssigned"
}
data "azurerm_cosmosdb_account" "this" {
name = var.cosmosdb_account_name
resource_group_name = var.cosmosdb_resource_group_name
}
// built-in role that allow the app-service to read and write to an Azure Cosmos DB
resource "azurerm_role_assignment" "cosmosdbContributor" {
scope = data.azurerm_cosmosdb_account.this.id
principal_id = azurerm_linux_web_app.this.identity.0.principal_id
role_definition_name = "Contributor"
}
Step Two: I used the managed identity library to fetch the necessary credentials in the Python code.
from azure.identity import ManagedIdentityCredential
from azure.cosmos.cosmos_client import CosmosClient
client = CosmosClient(get_endpoint(),credential=ManagedIdentityCredential())
client = self._get_or_create_client()
database = client.get_database_client(DB_NAME)
container = database.get_container_client(CONTAINER_NAME)
container.query_items(query)
I received the following error when running the code locally and from Azure (the error can be viewed from the Log stream of the Azure App Service):
raise TypeError(
TypeError: Unrecognized credential type. Please supply the master key as str, or a dictionary or resource tokens, or a list of permissions.
Any help or discussion is welcome

If you are using the Python SDK, you can directly do this ,check the sample here
aad_credentials = ClientSecretCredential(
tenant_id="<azure-ad-tenant-id>",
client_id="<client-application-id>",
client_secret="<client-application-secret>")
client = CosmosClient("<account-endpoint>", aad_credentials)

How to disable Airflow DAGs with AWS Lambda

I would need to disable Airflow DAGs with AWS Lambda or some other way. Can I use python code in order to do this? Thank you in advance.

You can pause/unpause a DAG with Airflow Rest API
The relevant endpoint is update a DAG.
https://airflow.apache.org/api/v1/dags/{dag_id}
With:
{
"is_paused": true
}
You also have Airflow official python client that you can use to interact with the API. Example:
import time
import airflow_client.client
from airflow_client.client.api import dag_api
from airflow_client.client.model.dag import DAG
from airflow_client.client.model.error import Error
from pprint import pprint
configuration = client.Configuration(
host = "http://localhost/api/v1"
)
# Configure HTTP basic authorization: Basic
configuration = client.Configuration(
username = 'YOUR_USERNAME',
password = 'YOUR_PASSWORD'
)
with client.ApiClient(configuration) as api_client:
# Create an instance of the API class
api_instance = dag_api.DAGApi(api_client)
dag_id = "dag_id_example" # str | The DAG ID.
dag = DAG(
is_paused=True,
)
try:
# Update a DAG
api_response = api_instance.patch_dag(dag_id, dag)
pprint(api_response)
except client.ApiException as e:
print("Exception when calling DAGApi->patch_dag: %s\n" % e)
You can see the full example in the client doc.

Module object not callable error when using Boto3 with DynamoDB

I'm trying to insert an item into my DynamoDB using the Python SDK (I'm not familiar with Python too much). I'm also using MQTT to retrieve some data. Currently the MQTT part works fine (I have some hard coded values for now so I can focus on getting the AWS functionality working)
I'm also using an AWS Educate account so need to pass in my access keys etc (which I've removed from the code for posting here). The code right now is:
from pprint import pprint
import boto3
import paho.mqtt.client as paho
import json
from types import SimpleNamespace
broker = "35.173.177.9"
#MQTT function
def onMessage(client, userdata, message):
print("Message recieved: ", str(message.payload.decode("utf-8", "ignore")))
movie_resp = putItem(1000, "1.1.1.1", "Pi", "06/01/21")
print("Put item succeeded:")
pprint(movie_resp, sort_dicts=False)
def putItem(id, ip, deviceName, clickDate, dynamodb=None):
session = boto3.session(
aws_access_key_id="",
aws_secret_access_key="",
aws_session_token="",
region_name='us-west-1'
)
if not dynamodb:
dynamodb = session.resource('dynamodb')
table = dynamodb.Table('piData')
response = table.put_item(
Item={
'ip': ip,
'id': id,
'deviceName': deviceName,
'clickDate': clickDate
}
)
return response
#MQTT code below
client = paho.Client("awsUbuntuEC2")
client.on_message = onMessage
client.connect(broker)
client.subscribe("jsonData")
client.loop_forever()
When running the code I get:
"TypeError: 'module' object is not callable"
on line
session = boto3.session(
Any ideas?

Try adding from boto3 import Session at the headers and try running the file again, please.

change from session = boto3.session to :
session = boto3.Session (Capital S in session )

How to patch an existing application using Python Azure SDK and Graph?

I am trying to add a reply_url programmatically to an Azure app registration, but I receive an azure.graphrbac.models.graph_error_py3.GraphErrorException: Specified HTTP method is not allowed for the request target.
It fails when I try to update an existing application with new reply_urls.
SDK I am using is: azure-graphrbac==0.61.1
My code:
from azure.common.credentials import ServicePrincipalCredentials
from azure.graphrbac import GraphRbacManagementClient
from azure.graphrbac.models import ApplicationUpdateParameters
class GraphClient:
def __init__(self, client_id, client_secret, tenant_id, object_id):
self._credentials = ServicePrincipalCredentials(
client_id=client_id,
secret=client_secret,
tenant=tenant_id,
resource="https://graph.windows.net"
)
self._graph_client = GraphRbacManagementClient(
credentials=self._credentials,
tenant_id=tenant_id
)
self._object_id = object_id
self._application = self._graph_client.applications.get(self._object_id)
def get_reply_urls(self) -> List[str]:
return self._application.reply_urls
def add_reply_url(self, reply_url) -> None:
reply_urls: list = self.get_reply_urls()
self._graph_client.applications.patch(
self._object_id,
ApplicationUpdateParameters(
reply_urls=[
*reply_urls,
reply_url]
)
)

Could not reproduce your issue, use the same version of azure-graphrbac, I test your code on my side, it works fine.
testclient = GraphClient(client_id = "xxxxx",client_secret = "xxxxx", tenant_id = "xxxxx", object_id = "xxxxx")
testclient.add_reply_url(reply_url = "http://localhost:8085")
Check in the portal:
Also, I test the sdk directly, both work.
from azure.common.credentials import ServicePrincipalCredentials
from azure.graphrbac import GraphRbacManagementClient
from azure.graphrbac.models import ApplicationUpdateParameters
_credentials = ServicePrincipalCredentials(
client_id="xxxxx",
secret="xxxxx",
tenant="xxxxx",
resource="https://graph.windows.net"
)
_graph_client = GraphRbacManagementClient(
credentials=_credentials,
tenant_id="xxxxx"
)
app = _graph_client.applications.patch(
application_object_id = "xxxxx",
parameters = ApplicationUpdateParameters(reply_urls = ["http://localhost:8080","http://localhost:8081"])
)

Update call looks good however it depends on a legacy API (AAD Graph) and tools. It's strongly recommended to move to MS Graph which supports almost all Azure AD Graph operations and will fully support them all in a future. Applications being one of them.
You can use Requests-OAuthlib or Microsoft Graph Python Client Library for this.

python firebase realtime listener

Hi there I'm new in python.
I would like to implement the listener on my Firebase DB.
When I change one or more parameters on the DB my Python code have to do something.
How can I do it?
Thank a lot
my db is like simple list of data from 001 to 200:
"remote-controller"
001 -> 000
002 -> 020
003 -> 230
my code is:
from firebase import firebase
firebase = firebase.FirebaseApplication('https://remote-controller.firebaseio.com/', None)
result = firebase.get('003', None)
print result

It looks like this is supported now (october 2018): although it's not documented in the 'Retrieving Data' guide, you can find the needed functionality in the API reference. I tested it and it works like this:
def listener(event):
print(event.event_type) # can be 'put' or 'patch'
print(event.path) # relative to the reference, it seems
print(event.data) # new data at /reference/event.path. None if deleted
firebase_admin.db.reference('my/data/path').listen(listener)

As Peter Haddad suggested, you should use Pyrebase for achieving something like that given that the python SDK still does not support realtime event listeners.
import pyrebase
config = {
"apiKey": "apiKey",
"authDomain": "projectId.firebaseapp.com",
"databaseURL": "https://databaseName.firebaseio.com",
"storageBucket": "projectId.appspot.com"
}
firebase = pyrebase.initialize_app(config)
db = firebase.database()
def stream_handler(message):
print(message["event"]) # put
print(message["path"]) # /-K7yGTTEp7O549EzTYtI
print(message["data"]) # {'title': 'Pyrebase', "body": "etc..."}
my_stream = db.child("posts").stream(stream_handler)

If Anybody wants to create multiple listener using same listener function and want to get more info about triggered node, One can do like this.
Normal Listener function will get a Event object it has only Data, Node Name, Event type. If you add multiple listener and You want to differentiate between the data change. You can write your own class and add some info to it while creating object.
class ListenerClass:
def __init__(self, appname):
self.appname = appname
def listener(self, event):
print(event.event_type) # can be 'put' or 'patch'
print(event.path) # relative to the reference, it seems
print(event.data) # new data at /reference/event.path. None if deleted
print(self.appname) # Extra data related to change add your own member variable
Creating Objects:
listenerObject = ListenerClass(my_app_name + '1')
db.reference('PatientMonitoring', app= obj).listen(listenerObject.listener)
listenerObject = ListenerClass(my_app_name + '2')
db.reference('SomeOtherPath', app= obj).listen(listenerObject.listener)
Full Code:
import firebase_admin
from firebase_admin import credentials
from firebase_admin import db
# Initialising Database with credentials
json_path = r'E:\Projectz\FYP\FreshOnes\Python\PastLocations\fyp-healthapp-project-firebase-adminsdk-40qfo-f8fc938674.json'
my_app_name = 'fyp-healthapp-project'
xyz = {'databaseURL': 'https://{}.firebaseio.com'.format(my_app_name),'storageBucket': '{}.appspot.com'.format(my_app_name)}
cred = credentials.Certificate(json_path)
obj = firebase_admin.initialize_app(cred,xyz , name=my_app_name)
# Create Objects Here, You can use loops and create many listener, But listener will create thread per every listener, Don't create irrelevant listeners. It won't work if you are running on machine with thread constraint
listenerObject = ListenerClass(my_app_name + '1') # Decide your own parameters, How you want to differentiate. Depends on you
db.reference('PatientMonitoring', app= obj).listen(listenerObject.listener)
listenerObject = ListenerClass(my_app_name + '2')
db.reference('SomeOtherPath', app= obj).listen(listenerObject.listener)

As you can see on the per-language feature chart on the Firebase Admin SDK home page, Python and Go currently don't have realtime event listeners. If you need that on your backend, you'll have to use the node.js or Java SDKs.

You can use Pyrebase, which is a python wrapper for the Firebase API.
more info here:
https://github.com/thisbejim/Pyrebase
To retrieve data you need to use val(), example:
users = db.child("users").get()
print(users.val())

Python Firebase Realtime Listener Full Code :
import firebase_admin
from firebase_admin import credentials
from firebase_admin import db
def listener(event):
print(event.event_type) # can be 'put' or 'patch'
print(event.path) # relative to the reference, it seems
print(event.data) # new data at /reference/event.path. None if deleted
json_path = r'E:\Projectz\FYP\FreshOnes\Python\PastLocations\fyp-healthapp-project-firebase-adminsdk-40qfo-f8fc938674.json'
my_app_name = 'fyp-healthapp-project'
xyz = {'databaseURL': 'https://{}.firebaseio.com'.format(my_app_name),'storageBucket': '{}.appspot.com'.format(my_app_name)}
cred = credentials.Certificate(json_path)
obj = firebase_admin.initialize_app(cred,xyz , name=my_app_name)
db.reference('PatientMonitoring', app= obj).listen(listener)
Output:
put
/
{'n0': '40', 'n1': '71'} # for first time its gonna fetch the data from path whether data is changed or not
put # On data changed
/n1
725
put # On data changed
/n0
401