when using DataLakeServiceClient from python3.5: how do i check if login was successfull?
following sample works OK for correct Credentials but if using wrong credentials it endless loads after calling create_directory. It does not throw any exception.
how to check if Login-Data is OK bevor calling create_directory?
from azure.storage.filedatalake import DataLakeServiceClient
from azure.core._match_conditions import MatchConditions
from azure.storage.filedatalake._models import ContentSettings
def initialize_storage_account(storage_account_name, storage_account_key):
try:
global service_client
service_client = DataLakeServiceClient(account_url="{}://{}.dfs.core.windows.net".format(
"https", storage_account_name), credential=storage_account_key)
except Exception as e:
print(e)
def create_directory(container_name,dir_name):
try:
file_system_client = service_client.get_file_system_client(file_system=container_name)
print(file_system_client)
file_system_client.create_directory(dir_name)
print("Created Directory")
except Exception as e:
print(e)
initialize_storage_account("wrongaccountname","a0lyWRONGKEYagAAA==")
create_directory("test_fs","test_dir")
how do i check if login was successfull?
To check if you have provided correct credentials, you will need to perform a network operation. It is recommended that you do a read operation like get_service_properties or try to list file systems.
If the operation is successful, that would mean correct credentials are supplied. If the operation fails, you will need to check the status code in the exception. If the operation fails with Authorization (403) error, that would most likely mean an issue with the credentials (there are other reasons as well because of which you can get 403 error).
Related
according to werkzeug docs, InternalServerError has original_exception parameter. That looks cool and I'would like to use it in following way to handle exceptions:
#app.errorhandler(Exception)
def handle_errors(err):
return err if isinstance(err, HTTPException) else InternalServerError(original_exception=err)
and log it in #app.after_request method to pack all additional info from request and response, where standard request logging occurs. Unfortunatelly I'm not able to find out how to use stored original_exception. Where is it stored?
When I check sys.exc_info() in #app.after_request, it's also empty (or (None, None, None)).
One way is probably log exceptions directly in #app.errorhandler, request info should be accessible there and response is not ready (because of error) anyways, but what happens with original_exception after storing in InternalServerError???
Thanks for answer, Michal
I found myself here looking for the same thing - how to access the original_exception, but I eventually gave up and settled on traceback to give me the info I was seeking.
This is the key component, capturing the traceback stack as a list:
formatted_lines = traceback.format_exc().splitlines()
Here's my final code in which I extracted the line number of the error and the specific trigger, then passed them to the page:
from werkzeug.exceptions import InternalServerError
import traceback
#app.errorhandler(InternalServerError)
def handle_500(e):
templateData = {
'errorAt' : 'Internal Server Error',
'errorIs' : 'Check ~/www/gunicorn.error for details'
}
try:
formatted_lines = traceback.format_exc().splitlines()
for error_location in formatted_lines:
if 'intvlm8r.py' in error_location:
templateData['errorAt'] = error_location
templateData['errorIs'] = formatted_lines[-1]
except Exception as e:
app.logger.debug(f'handle_500: Unhandled exception: {e}')
return render_template('500_generic.html', **templateData)
I'm new to Python. I have a working, monolithic Python program that I'm breaking into individual Python functions. I'd like to use the try: - except: pattern to catch specific exceptions for each function.
Example: Create a Key Vault client and retrieve a secret from Key Vault
import logging
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient
credentials = DefaultAzureCredential()
def create_kv_client(kv_name, credentials):
kv_uri = 'https://' + kv_name + '.vault.azure.net'
kv_client = SecretClient(vault_url=kv_uri, credential=credentials)
return kv_client
kv_client = create_kv_client('mykeyvaultname', credentials)
def retrieve_secret(table_stg_acct_key, kv_client):
retrieved_account_key = kv_client.get_secret(table_stg_acct_key)
return retrieved_account_key
try:
retrieved_account_key = retrieve_secret('mykeyvaultsecretname', kv_client)
print(retrieved_account_key)
except:
logging.error('####### Failed to retrieve key from Key Vault #######')
raise BaseException
Rather than raise BaseException here, I'd like to use the Azure Core exceptions module and log the actual message in the exception.
How is the except: statement handled in the case where two exceptions could be raised?
Example: There could be two exceptions raised by the get_secret method.
If the Key Vault URL is incorrect, a ServiceRequestError is raised:
ServiceRequestError: <urllib3.connection.HTTPSConnection object at 0x000001BFA2299640>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed
If the Key Vault secret name is incorrect, a ResourceNotFoundError is raised:
ResourceNotFoundError: (SecretNotFound) A secret with (name/id) notmykeyvaultsecretname was not found in this key vault. If you recently deleted this secret you may be able to recover it using the correct recovery command. For help resolving this issue, please see https://go.microsoft.com/fwlink/?linkid=2125182
How is this accomplished?
Do I have to import the azure core exception module?
An example of this pattern would be very helpful.
The exception will be caught in order of "except" clauses, but beware of the subclass tree, since a except will catch all subclasses as well. For instance, this one leads to unreachable code.
try:
# do something
except BaseException:
# do something with
except DerivedException:
# assuming DerivedException is an extension of BaseException, you can't reach that code
So put them in most specific first.
In your Azure situation, this brings to something like:
from azure.core.exceptions import (
ClientAuthenticationError,
HttpResponseError,
ServiceRequestError,
ResourceNotFoundError,
AzureError
)
try:
# do KV stuff
except ClientAuthenticationError as e:
# Can occur if either tenant_id, client_id or client_secret is incorrect
logger.critical("Azure SDK was not able to connect to Key Vault", e)
except HttpResponseError as e:
# One reason is when Key Vault Name is incorrect
logger.critical("Possible wrong Vault name given", e)
except ServiceRequestError:
# Network error, I will let it raise to higher level
raise
except ResourceNotFoundError:
# Let's assume it's not big deal here, just let it go
pass
except AzureError as e:
# Will catch everything that is from Azure SDK, but not the two previous
logger.critical("Azure SDK was not able to deal with my query", e)
raise
except Exception as e:
# Anything else that is not Azure related (network, stdlib, etc.)
logger.critical("Unknown error I can't blame Azure for", e)
raise
I wrote the script below to be able to connect to a remote server and get some data from the XML file. I added some error handling to be able to skip issues with some devices. For some reason whenever the script gets a 401 message back, it breaks the whole loop and I get the message "Could not properly read the csv file". I tried other ways of handling the exception and it would fail at other points. Any info on how to properly deal with this?
#!/usr/bin/python
import sys, re, csv, xmltodict
import requests, logging
from requests.packages.urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
def version(ip, username, password):
baseUrl = "https://" + ip
session = requests.Session()
session.verify = False
session.timeout = 45
print "Connecting to " + ip
try:
r = session.get(baseUrl + '/getxml?location=/Status', auth=(username, password))
r.raise_for_status()
except Exception as error:
print err
doc = xmltodict.parse(r.text)
version = str(doc['Status']['#version'])
def main():
try:
with open('list.csv', 'r') as file:
reader = csv.DictReader(file)
for row in reader:
version(row['ip'], row['Username'], row['Password'])
except Exception as error:
print ValueError("Could not properly read the csv file \r")
sys.exit(0)
if __name__ == "__main__":
main()
The doc and version variables in def version are outside the try: catch: so when r is None due to exception, the next 2 operations also fail, raising some uncaught exception. Which surfaces in main. Can you try including doc and version within the try: catch: and see if it works.
A related suggestion: catch specific exceptions as this helps know more about why your code crashed. ex. Response.raise_for_status() raises requests.exceptions.HTTPError. Catch that, raise all other exceptions. xml might raise something else, catch that, instead of catching ALL.
i'm sending apple push notifications via AWS SNS via Lambda with Boto3 and Python.
from __future__ import print_function
import boto3
def lambda_handler(event, context):
client = boto3.client('sns')
for record in event['Records']:
if record['eventName'] == 'INSERT':
rec = record['dynamodb']['NewImage']
competitors = rec['competitors']['L']
for competitor in competitors:
if competitor['M']['confirmed']['BOOL'] == False:
endpoints = competitor['M']['endpoints']['L']
for endpoint in endpoints:
print(endpoint['S'])
response = client.publish(
#TopicArn='string',
TargetArn = endpoint['S'],
Message = 'test message'
#Subject='string',
#MessageStructure='string',
)
Everything works fine! But when an endpoint is invalid for some reason (at the moment this happens everytime i run a development build on my device, since i get a different endpoint then. This will be either not found or deactivated.) the Lambda function fails and gets called all over again. In this particular case if for example the second endpoint fails it will send the push over and over again to endpoint 1 to infinity.
Is it possible to ignore invalid endpoints and just keep going with the function?
Thank you
Edit:
Thanks to your help i was able to solve it with:
try:
response = client.publish(
#TopicArn='string',
TargetArn = endpoint['S'],
Message = 'test message'
#Subject='string',
#MessageStructure='string',
)
except Exception as e:
print(e)
continue
Aws lamdba on failure retries the function till the event expires from the stream.
In your case since the exception on the 2nd endpoint is not handled, the retry mechanism ensures the reexecution of post to the first endpoint.
If you handle the exception and ensure the function successfully ends even when there is a failure, then the retries will not happen.
If i have some bad authorization data (for example wrong password) SUDS rises exception (400, u'Bad Request') from which i cant get anything, but in teh log is response, which contains data that password is wrong, but how to get this response? I tried like this:
except Exception as e:
print str(e)
print self._client.last_received()
It prints:
(400, u'Bad Request')
None
But in log there is long xml which contains <SOAP-ENV:Reason><SOAP-ENV:Text xml:lang="en">Sender not authorized</SOAP-ENV:Text></SOAP-ENV:Reason>
I am pulling this out of a comment and into an answer because of the code block.
import suds.client
try:
auth_url = "https://url.to.my.service/authenticator?wsdl"
auth_client = suds.client.Client(auth_url)
cookie = auth_client.service.authenticate(user,password)
except Exception as e:
print str(e)
print auth_client.last_received()
Using this code, I receive the appropriate response from my service if I pass an invalid password:
Server raised fault: 'error.pwd.incorrect'
None
And an appropriate response if I pass an invalid user id:
Server raised fault: 'error.uid.missing'
None
Something you may want to consider doing, is changing your except statement to catch suds.WebFault instead of the generic exception. There may be something else that is occurring and triggering your exception block.
One other thing that may help with your issue, is to pass faults=True in your Client() call.
The Client can be configured to throw web faults as WebFault or to
return a tuple (, )
The code I posted above would look like this:
auth_client = suds.client.Client(auth_url, faults=True)