I am trying to deploy resources to Azure China with ARM Templates. I had the code to do it for Azure, and now I am adapting it for Azure China, I believe that the only changes that I should perform were in
Change the authority host in the credentials
self.credentials = DefaultAzureCredential(authority = AzureAuthorityHosts.AZURE_CHINA)
Change the Management Url in the Client.
endpoints = get_cloud_from_metadata_endpoint(os.environ.get("ARM_ENDPOINT"))
self.client = ResourceManagementClient(self.credentials, self.subscriptionId, base_url=endpoints.endpoints.resource_manager)
Below is the code that I am using
Python Code:
def __init__(self, subscriptionId, resourceGroup):
self.logger = Logger("Azure China Connection")
self.logger.info("Retrieving the list of available endpoint")
# ARM_ENDPOINT = https://management.chinacloudapi.cn
endpoints = get_cloud_from_metadata_endpoint(os.environ.get("ARM_ENDPOINT"))
self.subscriptionId = subscriptionId
self.resourceGroup = resourceGroup
self.credentials = DefaultAzureCredential(authority = AzureAuthorityHosts.AZURE_CHINA)
self.logger.info("Creating a client for deploying resources on subscription {}".format(self.subscriptionId))
self.client = ResourceManagementClient(self.credentials, self.subscriptionId,
# endpoints.endpoints.resource_manager = https://management.chinacloudapi.cn
base_url=endpoints.endpoints.resource_manager)
self.logger.success("Client was successfully created")
def deploy(self, template, parameters):
resources = ""
for resource in template.get("resources"):
resources += "\n\t {}".format(resource.get("type"))
self.logger.info("The following resources: {}\nwill be deployed".format(resources))
deploymentProperties = DeploymentProperties(
mode = DeploymentMode.incremental,
template = template,
parameters = parameters.get("parameters")
)
self.logger.info("Attempting deploy operation")
deployment = self.client.deployments.begin_create_or_update(
self.resourceGroup,
uuid7(),
Deployment(properties=deploymentProperties)
) # Error occurs here
self.logger.success("Resources deployment successfully triggered")
return deployment.result()
load_dotenv()
connection = new AzureChinaConnection(os.environ.get("AZURE_SUBSCRIPTION_ID"), os.environ.get("AZURE_RESOURCE_GROUP"))
deployment = self.connection.deploy(template.json(), parameter.json())
**Message=**DefaultAzureCredential failed to retrieve a token from the
included credentials. Attempted credentials: EnvironmentCredential:
Authentication failed: AADSTS500011: The resource principal named
https://management.azure.com was not found in the tenant named EY
CHINA. This can happen if the application has not been installed by
the administrator of the tenant or consented to by any user in the
tenant. You might have sent your authentication request to the wrong
tenant. Trace ID: ace63d66-af4b-4457-b6c9-6ce050e34700 Correlation ID:
d85942a5-35fb-493f-8eef-ee9fe1f64b7f Timestamp: 2022-09-29 19:44:47Z
To mitigate this issue, please refer to the troubleshooting guidelines
here at
https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.
Based on the error message, it looks like I am pointing to a wrong endpoint https://management.azure.com instead of https://management.chinacloudapi.cn. The question is, where should I set it?
I though that it was already doing it in the __init__
self.client = ResourceManagementClient(self.credentials, self.subscriptionId,
# endpoints.endpoints.resource_manager = https://management.chinacloudapi.cn
base_url=endpoints.endpoints.resource_manager)
but seems it is not enough.
After struggling a lot, finally found the solution. Seems that no all the properties of Resource Manager Client are listed at:
https://learn.microsoft.com/en-us/python/api/azure-mgmt-resource/azure.mgmt.resource.resources.resourcemanagementclient?view=azure-python
There is a property named credential_scopes which should be set for change
credential_scopes=[CLOUD.endpoints.resource_manager + "/.default"])
so the function looks like
def __init__(self, subscriptionId, resourceGroup):
self.subscriptionId = subscriptionId
self.resourceGroup = resourceGroup
self.credentials = DefaultAzureCredential()
self.logger.info("Creating a client for deploying resources on subscription {}".format(self.subscriptionId))
self.client = ResourceManagementClient(self.credentials, self.subscriptionId,
base_url=CLOUD.endpoints.resource_manager,
credential_scopes=[CLOUD.endpoints.resource_manager + "/.default"])
Related
I am working on deploy resources in Azure using python based on provided templates. As a starting point I am working with https://github.com/Azure-Samples/Hybrid-Resource-Manager-Python-Template-Deployment
Using as it is, I am having an issue at the beginning of the deployment (deployer.py deploy function)
def deploy(self, template, parameters):
"""Deploy the template to a resource group."""
self.client.resource_groups.create_or_update(
self.resourceGroup,
{
'location': os.environ['AZURE_RESOURCE_LOCATION']
}
)
The error message is
Message='ServicePrincipalCredentials' object has no attribute
'get_token'
The statement is correct, ServicePrincipalCredentials get_token attribute doesn't exist, however token is, may be an error due to an outdated version?
Based on the constructor information, the error may be on credentials creation or client creation
def __init__(self, subscription_id, resource_group, pub_ssh_key_path='~/id_rsa.pub'):
mystack_cloud = get_cloud_from_metadata_endpoint(
os.environ['ARM_ENDPOINT'])
subscription_id = os.environ['AZURE_SUBSCRIPTION_ID'] //This may be an error as subscription_id is already provided as a parameter
credentials = ServicePrincipalCredentials(
client_id=os.environ['AZURE_CLIENT_ID'],
secret=os.environ['AZURE_CLIENT_SECRET'],
tenant=os.environ['AZURE_TENANT_ID'],
cloud_environment=mystack_cloud
) --> here
self.subscription_id = subscription_id
self.resource_group = resource_group
self.dns_label_prefix = self.name_generator.haikunate()
pub_ssh_key_path = os.path.expanduser(pub_ssh_key_path)
# Will raise if file not exists or not enough permission
with open(pub_ssh_key_path, 'r') as pub_ssh_file_fd:
self.pub_ssh_key = pub_ssh_file_fd.read()
self.credentials = credentials
self.client = ResourceManagementClient(self.credentials, self.subscription_id,
base_url=mystack_cloud.endpoints.resource_manager) --> here
Do you know how I can fix this?
After struggling a little, I could find a solution. Just replace
credentials = ServicePrincipalCredentials(
client_id=os.environ['AZURE_CLIENT_ID'],
secret=os.environ['AZURE_CLIENT_SECRET'],
tenant=os.environ['AZURE_TENANT_ID'],
cloud_environment=mystack_cloud
)
for
self.credentials = DefaultAzureCredential()
The final code looks like:
from azure.identity import DefaultAzureCredential
def __init__(self, subscriptionId, resourceGroup):
endpoints = get_cloud_from_metadata_endpoint(os.environ.get("ARM_ENDPOINT"))
self.subscriptionId = subscriptionId
self.resourceGroup = resourceGroup
self.credentials = DefaultAzureCredential()
self.client = ResourceManagementClient(self.credentials, self.subscriptionId,
base_url=endpoints.endpoints.resource_manager)
I'm struggling with basic user management while working with Azure's AD B2C framework.
I've successfully set up an Azure AD B2C resource, registered my consumer-facing web application (created a client secret and granted permission, both delegated and application, to User.ReadWrite.All), created custom attributes, and added out-of-the-box signup and signin user flows. Furthermore, I've successfully registered and signed-in users to my web application.
To get to this point, I followed the Python sample provided within the documentation (ms-identity-python-webapp-master):
app.py
#app.route("/login")
def login():
session["state"] = str(uuid.uuid4())
# Technically we could use empty list [] as scopes to do just sign in,
# here we choose to also collect end user consent upfront
auth_url = _build_auth_url(scopes=app_config_b2c.SCOPE, state=session["state"])
return render_template("templates/login.html", auth_url=auth_url, version=msal.__version__)
#app.route(app_config_b2c.REDIRECT_PATH) # Its absolute URL must match your app's redirect_uri set in AAD
def authorized():
if request.args.get('state') != session.get("state"):
return redirect(url_for("index")) # No-OP. Goes back to Index page
if "error" in request.args: # Authentication/Authorization failure
return render_template("auth_error.html", result=request.args)
if request.args.get('code'):
cache = _load_cache()
result = _build_msal_app(cache=cache).acquire_token_by_authorization_code(
request.args['code'],
scopes=app_config_b2c.SCOPE, # Misspelled scope would cause an HTTP 400 error here
redirect_uri=url_for("authorized", _external=True))
if "error" in result:
return render_template("auth_error.html", result=result)
session["user"] = result.get("id_token_claims")
_save_cache(cache)
return redirect(url_for("index"))
#app.route("/logout")
def logout():
session.clear() # Wipe out user and its token cache from session
return redirect( # Also logout from your tenant's web session
app_config_b2c.AUTHORITY + "/oauth2/v2.0/logout" +
"?post_logout_redirect_uri=" + url_for("index", _external=True))
#app.route("/graphcall")
def graphcall():
token = _get_token_from_cache(app_config_b2c.SCOPE)
if not token:
return redirect(url_for("login"))
graph_data = requests.get( # Use token to call downstream service
app_config_b2c.ENDPOINT,
headers={'Authorization': 'Bearer ' + token['access_token']},
).json()
return render_template('templates/display.html', result=graph_data)
def _load_cache():
cache = msal.SerializableTokenCache()
if session.get("token_cache"):
cache.deserialize(session["token_cache"])
return cache
def _save_cache(cache):
if cache.has_state_changed:
session["token_cache"] = cache.serialize()
def _build_msal_app(cache=None, authority=None):
return msal.ConfidentialClientApplication(
app_config_b2c.CLIENT_ID, authority=authority or app_config_b2c.AUTHORITY,
client_credential=app_config_b2c.CLIENT_SECRET, token_cache=cache)
def _build_auth_url(authority=None, scopes=None, state=None):
return _build_msal_app(authority=authority).get_authorization_request_url(
scopes or [],
state=state or str(uuid.uuid4()),
redirect_uri=url_for("authorized", _external=True))
def _get_token_from_cache(scope=None):
cache = _load_cache() # This web app maintains one cache per session
cca = _build_msal_app(cache=cache)
accounts = cca.get_accounts()
if accounts: # So all account(s) belong to the current signed-in user
result = cca.acquire_token_silent(scope, account=accounts[0])
_save_cache(cache)
return result
app_config_b2c.py
import os
b2c_tenant = "myapplication"
signupsignin_user_flow = "b2c_1_signupsignin1"
editprofile_user_flow = "b2c_1_profileediting1"
resetpassword_user_flow = "b2c_1_passwordreset1"
authority_template = "https://{tenant}.b2clogin.com/{tenant}.onmicrosoft.com/{user_flow}"
CLIENT_SECRET = "Enter_the_Client_Secret_Here" # Our Quickstart uses this placeholder
# In your production app, we recommend you to use other ways to store your secret,
# such as KeyVault, or environment variable as described in Flask's documentation here
# https://flask.palletsprojects.com/en/1.1.x/config/#configuring-from-environment-variables
# CLIENT_SECRET = os.getenv("CLIENT_SECRET")
# if not CLIENT_SECRET:
# raise ValueError("Need to define CLIENT_SECRET environment variable")
AUTHORITY = authority_template.format(
tenant=b2c_tenant, user_flow=signupsignin_user_flow)
B2C_PROFILE_AUTHORITY = authority_template.format(
tenant=b2c_tenant, user_flow=editprofile_user_flow)
B2C_RESET_PASSWORD_AUTHORITY = authority_template.format(
tenant=b2c_tenant, user_flow=resetpassword_user_flow)
CLIENT_ID = "xxx.xxxxxx"
REDIRECT_PATH = "/getAToken" # It will be used to form an absolute URL
# And that absolute URL must match your app's redirect_uri set in AAD
# This is the resource that you are going to access in your B2C tenant
ENDPOINT = 'https://graph.microsoft.com/v1.0/users'
# These are the scopes that you defined for the web API
SCOPE = ["User.ReadWrite.All"]
SESSION_TYPE = "filesystem" # So token cache will be stored in server-side session
The graphcall doesn't work within this framework (perhaps it's a b2c issue), which I'm sure is part of the problem, but ultimately I'd just like the application to consume the logged-in user's AD attributes (particularly the custom attributes I've enabled), and modify them when necessary. For instance, say a custom attribute is "paid_subscriber". When a user registers, the attribute is empty. When the user purchases content, I'd like to set the value of the attribute to something relevant (like "true").
Is this possible? Do I need other user flows? What am I missing here (theoretically and practically)?
Microsoft Graph does not support the tokens issued by the Azure AD B2C.
You need to have the access token generated by the Azure AD.
There is a process using Azure ad b2c custom policy where you can integrated the Microsoft Graph and add custom attributes to the claims.
This document helps you to get the Azure AD access token to call Graph, With the above implementation there wont be much changes in the phython code
for adding the custom attributes go through the document
I need to make online predictions from a model that is deployed in cloud ml engine. My code in python is similar to the one found in the docs (https://cloud.google.com/ml-engine/docs/tensorflow/online-predict):
service = googleapiclient.discovery.build('ml', 'v1')
name = 'projects/{}/models/{}'.format(project, model)
if version is not None:
name += '/versions/{}'.format(version)
response = service.projects().predict(
name=name,
body={'instances': instances}
).execute()
However, I receive the "instances" data from outside the script, I wonder if there is a way I could run this script without making the "service = googleapiclient.discovery.build('ml', 'v1')" each time before a request, since it takes time.
pd: this is my very first project on gcp. Thank you.
Something like this will work. You'll want to initialize your service globally then use that service instance to make your call.
import googleapiclient.discovery
AI_SERVICE = None
def ai_platform_init():
global AI_SERVICE
# Set GCP Authentication
credentials = os.environ.get('GOOGLE_APPLICATION_CREDENTIALS')
# Path to your credentials
credentials_path = os.path.join(os.path.dirname(__file__), 'ai-platform-credentials.json')
if credentials is None and os.path.exists(credentials_path):
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = credentials_path
# Create AI Platform Service
if os.path.exists(credentials_path):
AI_SERVICE = googleapiclient.discovery.build('ml', 'v1', cache=MemoryCache())
# Initialize AI Platform on load.
ai_platform_init()
then later on, you can do something like this:
def call_ai_platform():
response = AI_SERVICE.projects().predict(name=name,
body={'instances': instances}).execute()
Bonus! in case you were curious about the MemoryCache class in the googleapiclient.discovery call, that was borrowed from another SO:
class MemoryCache():
"""A workaround for cache warnings from Google.
Check out: https://github.com/googleapis/google-api-python-client/issues/325#issuecomment-274349841
"""
_CACHE = {}
def get(self, url):
return MemoryCache._CACHE.get(url)
def set(self, url, content):
MemoryCache._CACHE[url] = content
I have a Django app created using Django rest framework. Below is the configuration that my setup is using:
Django 1.9
mongoDB as backend
gunicorn
nginx
Now I have created an API to enter data in DB and retrieve it using REST. I have test it via postman and it is working fine. We have a firmware which is consuming those APIs and that team want to use SSL socket connection instead of REST API.
I am new in SSL Socket connection and I am not able to find anything on internet that can help me on this. I know it is possible to create socket in Python but I am not able to understand how to use it in Django app to Read/Write data in mongoDB.
Any guidance will be very helpful.
TO READER : I understand you may want to close this question but please put up a remark on how to get guidance on this as SO is the biggest portal for putting up questions.
EDIT 1 : I am adding the code of my serializer.py API.
from rest_framework import serializers
class SaveUserLogs(serializers.Serializer):
token = serializers.CharField(label=_("Token"))
device_ip = serializers.CharField(label=_('Device IP'))
device_name = serializers.CharField(label=_('Device Name'))
device_os = serializers.CharField(label=_('Device OS'))
device_macid = serializers.CharField(label=_('Device MAC ID'))
dest_port = serializers.CharField(label=_('Device Port'))
conn_type = serializers.CharField(label=_('Connection Type'))
date = serializers.CharField(label=_('modified date'))
def validate(self, attrs):
device_macid = attrs.get('device_macid')
token = attrs.get('token')
if device_macid:
tokendetails = validate_token(token)
if not tokendetails:
msg = _('Invalid token.')
raise serializers.ValidationError(msg)
else:
userdetails = tokendetails.user
if userdetails.check_mac_id(device_macid):
all_logs = UserLog.objects.all().order_by("-id")
log_id = 1
if all_logs:
getid = all_logs[0].id
log_id = getid + 1
new_log = UserLog(
id=log_id,
device_ip=attrs.get('device_ip'),
device_name=attrs.get('device_name'),
device_os=attrs.get('device_os'),
device_macid=attrs.get('device_macid').upper(),
dest_port=attrs.get('dest_port'),
conn_type=attrs.get('conn_type'),
)
new_log.save()
print("saving log", log_id)
else:
msg = _('Invalid MAC ID')
raise serializers.ValidationError(msg)
else:
msg = _('Must include "MAC ID".')
raise serializers.ValidationError(msg)
attrs['mac_id'] = userdetails.mac_id
return attrs
It looks like you require socket library provided with python. But using this is not recommended since you will have to take care of lot of low level networking stuff and your program will get complicated.
You should instead keep your REST api and run your nginx server on https. You can then write the code on firmware to send and receive data from the server using https.
If for some reason you don't want to use https then you should use requests library to write your server.
Got to say, I am not that clear how to use oAuth 2.0 with Picasa API version 2. From the Google doc itself, (cmiiw), I got the impression that Picasa API version 1 was deprecated, which means Python gdata for Picasa (that only support version 1), is simply useless.
Therefore, I develop my own class using the rudimentary Picasa documentation. But, I found that the token generated is very short lived: user got to reauthenticate him/herself again. Can we make user only reauthenticate one, and somehow the token got refreshed automatically if it's expired?
This is my oAuth class that I develop to solve this Picasa problem. Would love to have suggestion on how to fix this class to allow a long term lived token
class OAuth():
"""
Simplify oauth process
"""
def __init__(self, **kwargs):
self.client_secret_file = kwargs.get('client_secret_file')
self.token_location = os.path.join(kwargs.get('token_location'), kwargs.get('token_filename'))
self.scope = kwargs.get('scope')
self.credentials = None
def is_authenticated(self):
storage = Storage(self.token_location)
self.credentials = storage.get()
authenticated = self.credentials is not None and not self.credentials.invalid
Logger.debug('oAuth: authenticated = %s' % authenticated)
return authenticated
def authenticate(self):
if self.scope is None:
Logger.error('oauth: please specify scope')
return
missing_message = '''
WARNING: Please configure OAuth 2.0
To make this code run you will need to populate the client_secrets.json file
found at: %s
with information from the Developers Console
https://console.developers.google.com/
For more information about the client_secrets.json file format, please visit:
https://developers.google.com/api-client-library/python/guide/aaa_client_secrets
''' % self.client_secret_file
flow = flow_from_clientsecrets(self.client_secret_file,
message=missing_message,
scope=self.scope)
# ca_certs_file = os.path.join('cacert.pem')
flags = argparser.parse_args()
storage = Storage(self.token_location)
self.credentials = run_flow(flow, storage, flags)#, http=httplib2.Http(ca_certs=ca_certs_file))
EDITED:
I add my solution here.
def get_album_list(self):
if self._album_list is not None:
return self._album_list
http = httplib2.Http(ca_certs=os.environ['REQUESTS_CA_BUNDLE'])
http = self.oauth.credentials.authorize(http)
response, album_list = http.request(Picasa.PHOTOS_URL, 'GET')
if response['status'] == '403':
self.oauth.credentials.refresh(http)
response, album_list = http.request(Picasa.PHOTOS_URL, 'GET')
album_list = json.load(StringIO(album_list))
self._album_list = {}
for e in album_list['feed']['entry']:
album_id = unicode(e['id']['$t'].split('/')[9]).split('?')[0]
self._album_list[e['title']['$t']] = album_id
return self._album_list
There are two types of tokens: access token and refresh token. Your application should receive both when a user grants you an access for the first time.
{
"access_token":"1/fFAGRNJru1FTz70BzhT3Zg",
"expires_in":3920,
"token_type":"Bearer",
"refresh_token":"1/xEoDL4iW3cxlI7yDbSRFYNG01kVKM2C-259HOF2aQbI"
}
It is important to save refresh token for further use. When the first access token expired you have to use the refresh token to obtain a new one.
https://developers.google.com/accounts/docs/OAuth2WebServer#refresh
Also consider using https://github.com/google/oauth2client