Develop Reference

Python Azure module error handling TCP 104 not being caught

I am currently using python 3.8.8 with version 12.9.0 of azure.storage.blob and 1.14.0 of azure.core.
I am downloading multiple files using the azure.storage.blob package. My code looks something like the following
from azure.storage.blob import ContainerClient
from azure.core.exceptions import ResourceNotFoundError, AzureError
from time import sleep
max_attempts = 5
container_client = ContainerClient(DETAILS)
for file in multiple_files:
attempts = 0
while attempts < max_attempts:
try:
data = container.download_blob(file).readall()
break
except ResourceNotFoundError:
# log missing data
break
except AzureError:
# This is mainly here as connections seem to drop randomly.
attempts += 1
sleep(1)
if attempts >= max_attempts:
#log connection error
#do something with the data.
It seems to be running fine, and I don't see any loss of data. However, within my terminal I keep getting the message
Unable to stream download: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer'))
This appears to be a TCP 104 return message but isn't being handled by the azure module. My questions are as follows.
Where is this message coming from? I can't see it in any of the packages I am using.
How do I handle this error better? It doesn't appear to be caught as an exception as it isn't crashing my code.
Can I get this to print to a log?

Where is this message coming from? I can't see it in any of the packages I am using.
Looks like The clients seemed to be connected to the server, but when they attempted to transfer data, they received a Errno 104 Connection reset by peer error. This also means, that the other side has reset the connection else the client would encounter with [Errno 32] Broken pipe exception.
How do I handle this error better? It doesn't appear to be caught as an exception as it isn't crashing my code.
One of the workarounds you can try is to have try and catch block to handle that exception:
from socket import error as SocketError
import errno
try:
response = urllib2.urlopen(request).read()
except SocketError as e:
if e.errno != errno.ECONNRESET:
raise # Not error we are looking for
pass # Handle error here.
Also try referring to this similar issue where sudo pip3 install urllib3 solved the issue.
Can I get this to print to a log?
One workaround is that you can pass exception instance in exc_info argument:
import logging
try:
1/0
except Exception as e:
logging.error('Error at %s', 'division', exc_info=e)
For more information you can refer How to log python exception?
Here is a related issue that you can follow up
azure storage blob download: ConnectionResetError(104, 'Connection reset by peer')
REFERENCE:
Connection broken: ConnectionResetError(104, 'Connection reset by peer') error while streaming

Raising user defined exception when connecting to sqlite

Let's say that I want to connect to database but getting an error on connect. In most of examples what I found people catch basic Error exception. But it also will catch any other mistake. So, the question is, what should I do to raise different user defined exceptions? In other words, how to throw user defined exception instead of predefined exception?

Like this:
class MySpecificException(Exception):
print('Handle exception here:')
try:
1/0
except ZeroDivisionError as e:
print('error text')
raise MySpecificException('SpecificException')

Don't Log 'Certificate did not match expected hostname' Error Messages

My web app requests several URLs and sometimes SSL certificate errors are raised. They are all third party URLs so I can't fix their errors and I prefer not to log them. Nevertheless, something is logging this by itself: 2017-08-05 00:22:49,496 ERROR -- : Certificate did not match expected hostname: www.improving-autonomy.org. Certificate: {'subjectAltName': [('DNS', '*.wordpress.com'), ('DNS', 'wordpress.com')], 'subject': ((('commonName', u'*.wordpress.com'),),)} Anyone knows how can I stop it? Please find my code bellow. Many thanks in advance!
try :
ua = UserAgent()
headers = {'Content-Type' : 'text/html', 'Accept-Encoding' : None, 'User-Agent' : ua.random}
response = requests.get(url, headers=headers, timeout=10)
except ssl.CertificateError as e :
pass
UPDATED -- :
It looks like requests module logs it (connection.py). Why it keeps logging if I'm already catching the same exception?
def _match_hostname(cert, asserted_hostname):
try:
match_hostname(cert, asserted_hostname)
except CertificateError as e:
log.error(
'Certificate did not match expected hostname: %s. '
'Certificate: %s', asserted_hostname, cert
)
# Add cert to exception and reraise so client code can inspect
# the cert when catching the exception, if they want to
e._peer_cert = cert
raise

Sure. You are catching the same exception, but what you are not seeing is where this is happening. Let's take a look at the snippet of what is happening here:
except CertificateError as e:
log.error(
'Certificate did not match expected hostname: %s. '
'Certificate: %s', asserted_hostname, cert
)
# Add cert to exception and reraise so client code can inspect
# the cert when catching the exception, if they want to
e._peer_cert = cert
raise
So, when the exception is first raised, that code catches the CertificateError, then it makes a log.error, assigns the cert as an attribute, per the comment in the code, then, a call to raise is made.
That empty raise call is now going to re-raise the last exception made, which is the CertificateError exception, and that is what you are catching. So the log call has already been made by that code, and your exception catching is being made from that specific raise call.

You can catch the exception and then print it's type:
except Exception as exc:
print exc, exc.message, exc.__class__
Then use this specific exception type in your code, which should work. Also you can add an else clause after the except statement, and put the logging code there. This code will be executed only if the try block executed successfully

Raise exception for error logs

I'm using a library that is logging an error. But I'd like it to raise an exception instead.
Is there quick way to have an exception raised instead of just an error being logged?
The library I'm using is cssutils, but for simplicity, suppose my code looks like:
from foo import do_something
do_something(x)
and suppose do_something will log an error if x was unacceptable input. Is there some quick hack that would cause an exception to be raised if do_something logs an error? Or is the only way to edit foo's source?

It's not clear if you want to still log the error, but you can try something like:
if error_is_true:
raise Exception("error message...")
With log:
try:
my_function()
except Exception:
logger.exception("error msg...")

Python handle custom exception from java web service

I have a java web service and python client using suds. My server raises custom exceptions which I would like to handle in the python script. Is it possible to catch them or it always will be caught as suds.WebFault exception?

suds.WebFault has fault field that has information about fault.
except suds.WebFault, e:
print e.fault.faultstring
print e.document
You can have your program to analyze server custom exception from WebFault and create new exception class(es) for every specific server exception then catch suds.WebFault exception, read server exception details and raise your custom exception.
class MyException(suds.WebFault):
pass
def convertServerException(e):
if e.fault.faultstring == 'exception1':
return MyException()
#...add more exception handling cases here
#...
try:
#...make a WebService call
except suds.WebFault, e:
print e
print e.fault
raise convertServerException(e)