Develop Reference

How to access Azure Keyvault via private endpoint in Python?

For security purpose, I have disabled public access under Networking Tab in Keyvault and have a private endpoint in place. Both keyvault and private endpoint reside in same resource group. I have an app registration for my application for which I have granted access under Access policies in Keyvault.
Using Python SDK,
from azure.keyvault.secrets import SecretClient
from azure.identity import ClientSecretCredential as cs
keyVaultName = "<NAME>"
kvURI = "https://<NAME>.vault.azure.net"
AZ_TENANT_ID = '<AZ_TENANT_ID>'
AZ_CLIENT_ID = '<AZ_CLIENT_ID>'
AZ_CLIENT_SECRET = '<AZ_CLIENT_SECRET>'
credential = cs(
tenant_id=AZ_TENANT_ID,
client_id=AZ_CLIENT_ID,
client_secret=AZ_CLIENT_SECRET)
def set_secret(secretname,secretvalue):
print(credential)
secret_client = SecretClient(vault_url=kvURI, credential=credential)
secret = secret_client.set_secret(secretname,secretvalue,enabled=True)
sec_dic={}
sec_dic['name']=secret.name
sec_dic['value']=secret.value
sec_dic['properties']=secret.properties.version
return sec_dic
xx=set_secret('g','ff')
print(xx)
When running this code, I get the follwing error,
azure.core.exceptions.HttpResponseError: (Forbidden) Public network access is disabled and request is not from a trusted service nor via an approved private link.
Code: Forbidden
Message: Public network access is disabled and request is not from a trusted service nor via an approved private link.
Inner error: {
"code": "ForbiddenByConnection"
}
What am I doing wrong? How do I connect to keyvault that has no public access only via private endpoint?

I have reproduced in my environment, and got expected results as below:
Firstly, I have done the same process you have explained, and I got same error as you have got:
So, this error comes when you create a private endpoint.
When you create a private endpoint with a particular Virtual Network-Subnet, you need to create a Virtual Machine where Virtual Network is integrated.
Then we need to open the Vm created integrated with the above Virtual Network-Subnet, and now when I try to access the key Vault, I got required result as below:
References:
azure - Unable to get storage account container details - Stack Overflow
Azure Key Vault not allow access via private endpoint connection
Azure functions and Azure KeyVault communicating through service endpoint

How to get DB password from Azure app vault using python ? I am running this python file on google Dataproc cluster

My Sql server DB password is saved on Azure app vault which has DATAREF ID as a identifier. I need that password to create spark dataframe from table which is present in SQL server. I am running this .py file on google Dataproc cluster. How can I get that password using python?

Since you are accessing an Azure service from a non-Azure service, you will need a service principal. You can use certificate or secret. See THIS link for the different methods. You will need to give the service principal proper access and this will depend if you are using RBAC or access policy for your key vault.
So the steps you need to follow are:
Create a key vault and create a secret.
Create a Service principal or application registration. Store the clientid, clientsecret and tenantid.
Give the service principal proper access to the key vault(if you are using access policies) or to the specific secret(if you are using RBAC model)
The python link for the code is HERE.
The code that will work for you is below:
from azure.identity import ClientSecretCredential
from azure.keyvault.secrets import SecretClient
tenantid = <your_tenant_id>
clientsecret = <your_client_secret>
clientid = <your_client_id>
my_credentials = ClientSecretCredential(tenant_id=tenantid, client_id=clientid, client_secret=clientsecret)
secret_client = SecretClient(vault_url="https://<your_keyvault_name>.vault.azure.net/", credential=my_credentials)
secret = secret_client.get_secret("<your_secret_name>")
print(secret.name)
print(secret.value)

Azure ad access token validation at python web api with respect to client app

I am using azure for hosting web app and web api.
Environment configuration :
web api : Developed using django deployed/hosted on linux vm
Web app : Developed using Angular2 deployed/hosted on App service
For authentication I am using OAUTH2 protocol.
App registration details for OAUTH2
Under single tenant: "my-tenant-id-121"
Registered web api and provided access_as_user permissions
here suppose app_id/client_id : "my-api-id-123"
APP uri : 'api://my-api-id-123'
scope : 'api://my-api-id-123/access_as_user'
client secret is generated but not using it.
Registered web app and provided basic details, redirect uri etc
here suppose webapp app_id/client_id : "my-webapp-id-123"
Under app registration, provided api access permissions for above registered api using API permissions
Authentication :
client(web app): Using ng2-adal library
resource (web api) : using python "jwt" library for access token validation
I have done all the configuration, after authentication I am getting id_token for web app and access_token for postman client.
Token Validation:
I am validating access token at api side which I am receiving through Authorisation header.
I have followed all the references available, through jwt.verify() my token getting validated. Here I am also validating audience, for id_token it is aud: client_app_id and when using postman I am specifying scope, in that case aud :"api://my-api-id-123"
Here comes the main part:
While following all process I never used registered web api app_id i.e "my-webapp-id-123" anywhere.
Then how come client app getting authenticated also access token getting validated.
Also I tried to remove linking between web app and web api from azure app registration and tried to authenticate. In both cases I have received token and validated at api side.
My questions are -
why we need to register Web api at app registration on azure as it is not getting used?
In my current scenario which part I am missing, my concern is if I remove linking (on azure, at client app under api permissions) between "client_app" and "api_app", access_token /id_token retrieved at client app should not get validated at web api.
Thanks in advance.

How is the access token generated from a managed identity in Azure used with the Python SDK?

I've setup a VM machine in Azure that has a managed identity.
I follow the guide here https://learn.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/tutorial-linux-vm-access-arm
So now I have an access token. But what I fail to understand is how do I use this token to access my key vault? I'm using the Python SDK. Looking at the docs for the SDK here https://learn.microsoft.com/en-us/python/api/azure-keyvault/azure.keyvault?view=azure-python
There exist a access token class AccessToken(scheme, token, key)
I assume i can use my token i generated earlier here. But what is scheme and key? The docs does not explain it. Or am I looking at the wrong class to use with the token?

If you're using a VM with a managed identity, then you can create a credential for a Key Vault client using azure-identity's ManagedIdentityCredential class. The credential will fetch and use access tokens for you as you use the Key Vault client:
from azure.identity import ManagedIdentityCredential
from azure.keyvault.secrets import SecretClient
credential = ManagedIdentityCredential()
client = SecretClient("https://{vault-name}.vault.azure.net", credential)
secret = client.get_secret("secret-name")
Note that I'm using a SecretClient to fetch secrets from Key Vault; there are new packages for working with Key Vault in Python that replace azure-keyvault:
azure-keyvault-certificates (Migration guide)
azure-keyvault-keys (Migration guide)
azure-keyvault-secrets (Migration guide)
Clients in each of these packages can use any credential from azure-identity for authentication.
(I work on the Azure SDK in Python)

I wouldn't recommend you use the managed identity of a VM to access KeyVault. You should create a service principal if you intend on running scripts / code.
The best way of doing this is with the Azure CLI. See here for instructions on installing the CLI, and refer to this, or this for creating your service principal.
The best way to manage resources in Python is by using ADAL, which is documented:
https://github.com/AzureAD/azure-activedirectory-library-for-python
In your case, however, managing KeyVault is made a little easier since the KeyVault library for Python also provides the means for you to authenticate without directly using ADAL to obtain your access token. See here:
https://learn.microsoft.com/en-us/python/api/overview/azure/key-vault?view=azure-python
from azure.keyvault import KeyVaultClient
from azure.common.credentials import ServicePrincipalCredentials
credentials = ServicePrincipalCredentials(
client_id = '...',
secret = '...',
tenant = '...'
)
client = KeyVaultClient(credentials)
# VAULT_URL must be in the format 'https://<vaultname>.vault.azure.net'
# KEY_VERSION is required, and can be obtained with the KeyVaultClient.get_key_versions(self, vault_url, key_name) API
key_bundle = client.get_key(VAULT_URL, KEY_NAME, KEY_VERSION)
key = key_bundle.key
In the above, client_id, secret, and tenant (id) are all outputs of the az ad sp create-for-rbac --name {APP-NAME} CLI command.
Remember to review and adjust the role assignments for the sp you created. And your KeyVault is only as secure as the devices which have access to your sp's credentials.

AWS Elasticsearch Service Role Permissions

I'm trying to submit documents to an Elasticsearch index from a Python Lambda script running as a role which has the AmazonESFullAccess policy attached. I am creating the requests with the python elasticsearch library, and signing them with the aws_requests_auth library.
cred = boto3.session.Session().get_credentials()
es_host = '<elasticsearch-server>'
auth = AWSRequestsAuth(
aws_access_key=cred.access_key,
aws_secret_access_key=cred.secret_key,
aws_host=es_host,
aws_region='us-east-1',
aws_service='es')
ES_CLIENT = Elasticsearch(
host=es_host,
port=80,
connection_class=RequestsHttpConnection,
http_auth=auth)
Then sending bulk create requests as follows:
ES_CLIENT.bulk(
index='test_index',
body=docs)
This is failing with the following:
TransportError(403, u'{"message": "The security token included in the
request is invalid." }'): AuthorizationException ...
Though the same code works when running with admin user access keys.
Why do these requests fail when running as a role with 'full ES access'?

Because Elasticsearch is a seperate entity to AWS, but a version is hosted by AWS it seems that elasticsearch doesn't treat IAM the same way you'd expect.. AWS has something in place that tries, but doesn't quite work with just an access_key and secret_key. It needs the session Token as well.
The answer to this is to use the cred.token along with the access key and secret key and then pass it into your AWSRequestsAuth object:
auth = AWSRequestsAuth(
aws_access_key=cred.access_key,
aws_secret_access_key=cred.secret_key,
aws_token=cred.token,
aws_host=es_host,
aws_region='us-east-1',
aws_service='es')