Develop Reference

Python Azure module error handling TCP 104 not being caught

I am currently using python 3.8.8 with version 12.9.0 of azure.storage.blob and 1.14.0 of azure.core.
I am downloading multiple files using the azure.storage.blob package. My code looks something like the following
from azure.storage.blob import ContainerClient
from azure.core.exceptions import ResourceNotFoundError, AzureError
from time import sleep
max_attempts = 5
container_client = ContainerClient(DETAILS)
for file in multiple_files:
attempts = 0
while attempts < max_attempts:
try:
data = container.download_blob(file).readall()
break
except ResourceNotFoundError:
# log missing data
break
except AzureError:
# This is mainly here as connections seem to drop randomly.
attempts += 1
sleep(1)
if attempts >= max_attempts:
#log connection error
#do something with the data.
It seems to be running fine, and I don't see any loss of data. However, within my terminal I keep getting the message
Unable to stream download: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer'))
This appears to be a TCP 104 return message but isn't being handled by the azure module. My questions are as follows.
Where is this message coming from? I can't see it in any of the packages I am using.
How do I handle this error better? It doesn't appear to be caught as an exception as it isn't crashing my code.
Can I get this to print to a log?

Where is this message coming from? I can't see it in any of the packages I am using.
Looks like The clients seemed to be connected to the server, but when they attempted to transfer data, they received a Errno 104 Connection reset by peer error. This also means, that the other side has reset the connection else the client would encounter with [Errno 32] Broken pipe exception.
How do I handle this error better? It doesn't appear to be caught as an exception as it isn't crashing my code.
One of the workarounds you can try is to have try and catch block to handle that exception:
from socket import error as SocketError
import errno
try:
response = urllib2.urlopen(request).read()
except SocketError as e:
if e.errno != errno.ECONNRESET:
raise # Not error we are looking for
pass # Handle error here.
Also try referring to this similar issue where sudo pip3 install urllib3 solved the issue.
Can I get this to print to a log?
One workaround is that you can pass exception instance in exc_info argument:
import logging
try:
1/0
except Exception as e:
logging.error('Error at %s', 'division', exc_info=e)
For more information you can refer How to log python exception?
Here is a related issue that you can follow up
azure storage blob download: ConnectionResetError(104, 'Connection reset by peer')
REFERENCE:
Connection broken: ConnectionResetError(104, 'Connection reset by peer') error while streaming

Python: smtplib.SMTPServerDisconnected: please run connect() first

I'm attempting to create a program that reads the unread emails and responds to the send with the usage of auto-reply which would be triggered by the use of certain phrases. I'm doing this in Mac OSX in Visual Code. I'm able to connect to IMAP and SMTP but then I get the following error,
smtplib.SMTPServerDisconnected: please run connect() first.
I tried to use an exception that was part of the smtplib which should be raised if the SMTP server disconnects, but it doesn't do anything.
def smtp_init():
print("Initializing STMP . . .",end = '')
global s
s = smtplib.SMTP(smtpserver,smtpserverport)
status_code = s.starttls()[0]
if status_code is not 220:
raise Exception('Starting tls failed: '+ str(status_code))
status_code = s.login(radr,pwd)[0]
if status_code is not 235:
raise Exception('SMTP login failed: '+ str(status_code))
print("Done. ")
except smtplib.SMTPServerDisconnected:
smtp_init()
continue
The expected results would be to have the program in a loop checking the emails and responding to them if they have a phrase that corresponds to the auto-reply.

Don't Log 'Certificate did not match expected hostname' Error Messages

My web app requests several URLs and sometimes SSL certificate errors are raised. They are all third party URLs so I can't fix their errors and I prefer not to log them. Nevertheless, something is logging this by itself: 2017-08-05 00:22:49,496 ERROR -- : Certificate did not match expected hostname: www.improving-autonomy.org. Certificate: {'subjectAltName': [('DNS', '*.wordpress.com'), ('DNS', 'wordpress.com')], 'subject': ((('commonName', u'*.wordpress.com'),),)} Anyone knows how can I stop it? Please find my code bellow. Many thanks in advance!
try :
ua = UserAgent()
headers = {'Content-Type' : 'text/html', 'Accept-Encoding' : None, 'User-Agent' : ua.random}
response = requests.get(url, headers=headers, timeout=10)
except ssl.CertificateError as e :
pass
UPDATED -- :
It looks like requests module logs it (connection.py). Why it keeps logging if I'm already catching the same exception?
def _match_hostname(cert, asserted_hostname):
try:
match_hostname(cert, asserted_hostname)
except CertificateError as e:
log.error(
'Certificate did not match expected hostname: %s. '
'Certificate: %s', asserted_hostname, cert
)
# Add cert to exception and reraise so client code can inspect
# the cert when catching the exception, if they want to
e._peer_cert = cert
raise

Sure. You are catching the same exception, but what you are not seeing is where this is happening. Let's take a look at the snippet of what is happening here:
except CertificateError as e:
log.error(
'Certificate did not match expected hostname: %s. '
'Certificate: %s', asserted_hostname, cert
)
# Add cert to exception and reraise so client code can inspect
# the cert when catching the exception, if they want to
e._peer_cert = cert
raise
So, when the exception is first raised, that code catches the CertificateError, then it makes a log.error, assigns the cert as an attribute, per the comment in the code, then, a call to raise is made.
That empty raise call is now going to re-raise the last exception made, which is the CertificateError exception, and that is what you are catching. So the log call has already been made by that code, and your exception catching is being made from that specific raise call.

You can catch the exception and then print it's type:
except Exception as exc:
print exc, exc.message, exc.__class__
Then use this specific exception type in your code, which should work. Also you can add an else clause after the except statement, and put the logging code there. This code will be executed only if the try block executed successfully

Xmlrpc ServerProxy returns socket.gaierror

I'm trying to connect to a Magento API using Xmlrpc.
When the url is valid, i have no problem. But i'd like to catch errors if the url is not valid. If i try with an invalid url i have :
socket.gaierror: [Errno 8] nodename nor servname provided, or not known
I'm trying to catch it but i can't find a way to do it ..
I'm using Python 3.5 :
from xmlrpc.client import ServerProxy
from socket import gaierror
params = {
"encoding: "utf-8",
"verbose": False,
"transport": SpecialTransport() # I use a SpecialTransport class
}
try:
client = ServerProxy("https://ma.bad.url, **params)
except gaierror:
print("Error")
The problem is, that i never go through the except ..
I don't understand what i'm doing wrong..
Thanks!

I'm answering to myself.
I've finally been able to make it works like this :
# Connect to the url
client = ServerProxy('https://my.bad.url', **params)
# Try to login to Magento to get a session
try:
session = client.login('username', 'password')
except gaierror:
# Error resolving / connecting to the url
print('Connection error')
sys.exit(2)
except Fault:
# Error with the login
print('Login error')
sys.exit(2)
else:
print('Success')

Flask cannot raise HTTP exception after try catching Runtime error

When I try to raise a HTTP exception status code 400 it only prints the json error message on the browser but does not state HTTP/1.1 400 BAD REQUEST in the console like it is supposed to. The exception raising works for all other parts of my program but it doesn't work when I do it in a try-catch for a runtime error.
My exception handler is exactly this:
http://flask.pocoo.org/docs/0.11/patterns/apierrors/
my try-catch:
try:
// run some program
catch RuntimeError as e:
raise InvalidUsage(e.message, status_code=400)

You should use the abort function of flask, something like:
from flask import abort
#app.route("/some_route")
def some_route():
try:
# do something
except SomeException:
abort(400, "Some message")