I have a python class with many methods():
Method1()
Method2()
...........
...........
MethodN()
All methods -- while performing different tasks -- have the same scheme:
do something
do something else
has anything gone wrong?
raise an exception
I want to be able to get an email whenever an exception is raised anywhere in the class.
Is there some easy way to combine this logic into the class, rather than calling SendEmail() before every raise Exception statement? what is the right, pythonic way to deal with such a case? canh a 'generalized' Exception handler be the solution? I'd be glad for any ideas you may have.
like #User said before Python has logging.handlers.SMTPHandler to send logged error message. Use logging module! Overriding exception class to send an email is a bad idea.
Quick example:
import logging
import logging.handlers
smtp_handler = logging.handlers.SMTPHandler(mailhost=("smtp.example.com", 25),
fromaddr="from#example.com",
toaddrs="to#example.com",
subject=u"AppName error!")
logger = logging.getLogger()
logger.addHandler(smtp_handler)
try:
break
except Exception as e:
logger.exception('Unhandled Exception')
Note: Although this is a simple, obvious solution to the problem as stated, the below answer is probably better in most cases.
If the alternative is this:
if problem_test():
SendEmail()
raise Exception
Then why don't you just define a custom raise_email method?
def raise_email(self, e):
SendEmail()
raise e
Python stdlib has dedicated class to do what you want. See logging.handlers.SMTPHandler
Gist Link
The most important trick is here if secure parameter is not passed, the default value is None which raises exception if you are trying to authenticate with TLS/SSL enabled STMP Servers lik Gmail's, Yahoo's, Yandex's, STMP servers.
We passed an empty tuple to trigger smtp.ehlo() to authenticate correctly with SSL.
...
if self.secure is not None:
smtp.ehlo()
smtp.starttls(*self.secure)
smtp.ehlo()
...
import logging
import logging.handlers
__author__ = 'Ahmed Şeref GÜNEYSU'
def foo():
raise Exception("Foo Bar")
def main():
logger = logging.getLogger()
logger.addHandler(logging.handlers.SMTPHandler(
mailhost=("smtp.mail.yahoo.com", 587),
fromaddr="boss#example.com",
toaddrs="me#example.com",
subject="EXCEPTION",
credentials=('smtpuser#example.com', 'MY SECRET PASSWORD'),
secure=()))
try:
foo()
except Exception, e:
logging.exception(e)
if __name__ == '__main__':
main()
Beware the wizard's apprentice!
It would be better to log those errors, then check to see when the last email was sent, and if the timespan is too short, do not send another message because the human being will already be looking at the log file. For many things, one message per day would be enough, but even for system critical things, if you have already had one failure, what could go wrong if you wait two hours to send the next email?
If you send one email per two hour timespan, then the maximum number of emails per day is 12. And if you get a cascading failure (you will!) then it will most likely happen within a couple of hours of the first failure event.
Most large networking companies offer an SLA of 4 hour to fix a failure, measured from the time it first occurs (because cascading failures tend to repeat) until the customer is satisified that it is fixed. If you have a tighter SLA than that, then unless it is some finance industry service, you probably are offering too high of a service level.
But if you do have a 4 hour SLA, then I would make sure that any email sent within 2 - 4 hours of the last email, should use whatever bells and whistles you can to prioritise it, highlight it, etc. For instance use the X-Priority header and put the word URGENT in the subject so that your mail client can display it in large bold red letters.
How about this:
class MyException(Exception):
def __init__(self):
SendEmail()
Something like this, perhaps?
def mailexception(ex):
# Be creative.
print 'Mailing... NOW!'
def pokemontrainer(cls):
class Rye(cls):
def __getattribute__(self, name):
def catcher(func):
def caller(*args, **kwargs):
try:
func(*args, **kwargs)
except Exception, e:
mailexception(e)
raise
return caller
ref = cls.__getattribute__(self, name)
if hasattr(cls, name) and hasattr(getattr(cls, name), '__call__'):
return catcher(ref)
return Rye
#pokemontrainer
class Exceptor(object):
def toss(self, e):
raise e('Incoming salad!')
ex = Exceptor()
ex.toss(ValueError)
By 2019, the easiest and best option seems to be sentry.
You need just two lines of code:
import sentry_sdk
sentry_sdk.init("https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx#sentry.io/xxxxxxx")
and it will send you a detailed email with any raised error.
After this you can further inspect the error on the Sentry website where there is impressive debug info - exceptions traceback, logs, data passed to functions, packages versions etc...
I'd just subclass Exception and send the e-mail in the custom Exception.
You can use an except hook to send an email when an exception is not caught.
see sys.excepthook
You can use python alerting library. It supports email (via mailgun and sendgrid), telegram and slack notifications for sending alerts.
https://github.com/sinarezaei/alerting
Sample code:
from alerting import Alerting
from alerting.clients import AlertingMailGunClient, AlertingSlackClient, AlertingTelegramClient
alerts = Alerting(
clients=[
AlertingMailGunClient(your_mailgun_api_key, your_domain, from_email, target_email),
AlertingSlackClient(your_bot_user_oauth, target_channel),
AlertingTelegramClient(bot_token, chat_id)
]
)
try:
# something
except Exception as ex:
alerting.send_alert(title='some bad error happened', message=str(ex))
I like the answers that use the logging module, but I use the smtp library (smtplib) for this. To send error message, I do something like the following in the exception branch of the try/except block:
import smtplib as smtp
s = smtplib.SMTP('smtp.gmail.com', 587)
s.starttls()
from_user = r"foo#gmail.com"
to_user = r"bar#gmail.com"
password = "xxxxxxxx"
s.login(from_user, password)
subject = "Uh oh"
text = "XYZ error message you blew it!"
message = f"Subject: {subject}\n\n{text}"
s.sendmail(from_user, to_user, message);
This works well, but isn't the most secure option in the world. You actually have to tell google you want to let less secure apps connect (you can change this setting here: https://myaccount.google.com/lesssecureapps).
Related
I'm using kafka-python library for my fastapi consumer app and I'm consuming messages in batch with maximum of 100 records. Since the topic has huge traffic and have only one partition, consuming, processing and committing should be as quick as possible hence I want to use commit_async(), instead of synchronous commit().
But I'm not able to find a good example of commit_async(). I'm looking for an example for commit_async() with callback so that I can log in case of commit failure. But I'm not sure what does that callback function takes as argument and what field those arguments contain.
However the docs related to commit_async mentions the arguments, I'm not completely sure how to use them.
I need help in completing my callback function on_commit(), someone please help here
Code
import logging as log
from kafka import KafkaConsumer
from message_handler_impl import MessageHandlerImpl
def on_commit():
pass
class KafkaMessageConsumer:
def __init__(self, bootstrap_servers: str, topic: str, group_id: str):
self.bootstrap_servers = bootstrap_servers
self.topic = topic
self.group_id = group_id
self.consumer = KafkaConsumer(topic, bootstrap_servers=bootstrap_servers, group_id=group_id, enable_auto_commit=False, auto_offset_reset='latest')
def consume_messages(self, max_poll_records: int,
message_handler: MessageHandlerImpl = MessageHandlerImpl()):
try:
while True:
try:
msg_pack = self.consumer.poll(max_records=max_poll_records)
for topic_partition, messages in msg_pack.items():
message_handler.process_messages(messages)
self.consumer.commit_async(callback=on_commit)
except Exception as e:
log.error("Error while consuming message due to: %s", e, exc_info=True)
finally:
log.error("Something went wrong, closing consumer...........")
self.consumer.close()
if __name__ == "__main__":
kafka_consumer = KafkaMessageConsumer("localhost:9092", "test-topic", "test-group")
kafka_consumer.consume_messages(100)
The docs are fairly clear.
Called as callback(offsets, response) with response as either an Exception or an OffsetCommitResponse struct.
def on_commit(offsets, response):
# or maybe try checking type(response)
if hasattr(response, '<some attribute unique to OffsetCommitResponse>'):
print('committed ' + str(offsets))
else:
print(response) # exception
I'm sure you could look at the source code an maybe find a unit test that covers a full example
I am experimenting with smtplib in Python3.
I want to send the content of a variable to an email address. If there is an smtplib.SMTPAuthenticationError, I want to send that variable to an alternative email address. This works (see code below). But what if I want to add a third email address (if the first two fail for some reason)?
I don't think try and except allow me to add another block of the same code (with different email login details).
I know with subprocess, it's possible to acquire the returncode of a variable and then use if.
For example:
result = subprocess.run(["ls", "-al"], capture_output = True)
if result !=0:
do_something_to_list_the_directory
I don't know how this can be done without using subprocess. Can anyone please advise?
Code below:
try:
mail_sending_attempt = smtplib.SMTP("smtp_provider", 587)
mail_sending_attempt.starttls()
mail_sending_attempt.login(send, passinfo) ### this will not work
mail_sending_attempt.sendmail(send, receive, message)
mail_sending_attempt.quit()
except Exception:
mail_sending_attempt = smtplib.SMTP("smtp_provider", 587)
mail_sending_attempt.starttls()
mail_sending_attempt.login(send2, passinfo2) ### this will not work
mail_sending_attempt.sendmail(send2, receive2, message)
mail_sending_attempt.quit()
In case there are more email, you can use following snippet
from dataclasses import dataclass
#dataclass
class EmailData:
send: str
passinfo: str
receive: str
main = EmailData("send1", "passinfo1", "receive1")
backup_1 = EmailData("send2", "passinfo2", "receive2")
...
for data in [main, backup_1, ...]:
try:
mail_sending_attempt = smtplib.SMTP("smtp_provider", 587)
mail_sending_attempt.starttls()
mail_sending_attempt.login(data.send, data.passinfo)
mail_sending_attempt.sendmail(data.send, data.receive, message)
mail_sending_attempt.quit()
break
except Exception:
continue
else:
# the case when we won't encounter break, so every login failed.
raise Exception
I've been given some web scraping code on Python and now need to write some code so that if there is an error, it will notify someone by email. I have found this code online:
def send_email():
to = request.form.get('to')
if not to:
return ('Please provide an email address in the "to" query string '
'parameter.'), 400
sg = sendgrid.SendGridAPIClient(apikey=SENDGRID_API_KEY)
to_email = mail.Email(to)
from_email = mail.Email(SENDGRID_SENDER)
subject = 'This is a test email'
content = mail.Content('text/plain', 'Example message.')
message = mail.Mail(from_email, subject, to_email, content)
response = sg.client.mail.send.post(request_body=message.get())
if response.status_code != 202:
return 'An error occurred: {}'.format(response.body), 500
return 'Email sent.'
Is this the kind of thing that I should use? If not, what is the best way to go about this problem?
Thanks in advance.
What I meant:
try:
# Web scrapping code
except requests.packages.urllib3.exceptions.MaxRetryError as e:
print repr(e)
send_email()
OR
use except without specifying for passive handling.
try:
# Web scrapping code
except:
send_email()
BUT:
From the PEP-8 Style Guide for Python:
When catching exceptions, mention specific exceptions whenever
possible instead of using a bare except: clause.
A bare except: clause will catch SystemExit and KeyboardInterrupt
exceptions, making it harder to interrupt a program with Control-C,
and can disguise other problems. If you want to catch all exceptions
that signal program errors, use except Exception: (bare except is
equivalent to except BaseException:).
A good rule of thumb is to limit use of bare 'except' clauses to two
cases:
If the exception handler will be printing out or logging the
traceback; at least the user will be aware that an error has occurred.
If the code needs to do some cleanup work, but then lets the exception
propagate upwards with raise. try...finally can be a better way to
handle this case.
I use CherryPy to run a very simple web server. It is intended to process the GET parameters and, if they are correct, do something with them.
import cherrypy
class MainServer(object):
def index(self, **params):
# do things with correct parameters
if 'a' in params:
print params['a']
index.exposed = True
cherrypy.quickstart(MainServer())
For example,
http://127.0.0.1:8080/abcde:
404 Not Found
The path '/abcde' was not found.
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\cherrypy\_cprequest.py", line 656, in respond
response.body = self.handler()
File "C:\Python27\lib\site-packages\cherrypy\lib\encoding.py", line 188, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "C:\Python27\lib\site-packages\cherrypy\_cperror.py", line 386, in __call__
raise self
NotFound: (404, "The path '/abcde' was not found.")
Powered by CherryPy 3.2.4
I am trying to catch this exception and show a blank page because the clients do not care about it. Specifically, the result would be an empty body, no matter the url or query string that resulted in an exception.
I had a look at documentation on error handling cherrypy._cperror, but I did not find a way to actually use it.
Note: I gave up using CherryPy and found a simple solution using BaseHTTPServer (see my answer below)
Docs somehow seem to miss this section. This is what I found while looking for detailed explanation for custom error handling from the source code.
Custom Error Handling
Anticipated HTTP responses
The 'error_page' config namespace can be used to provide custom HTML output for
expected responses (like 404 Not Found). Supply a filename from which the
output will be read. The contents will be interpolated with the values
%(status)s, %(message)s, %(traceback)s, and %(version)s using plain old Python
string formatting.
_cp_config = {
'error_page.404': os.path.join(localDir, "static/index.html")
}
Beginning in version 3.1, you may also provide a function or other callable as
an error_page entry. It will be passed the same status, message, traceback and
version arguments that are interpolated into templates
def error_page_402(status, message, traceback, version):
return "Error %s - Well, I'm very sorry but you haven't paid!" % status
cherrypy.config.update({'error_page.402': error_page_402})
Also in 3.1, in addition to the numbered error codes, you may also supply
error_page.default to handle all codes which do not have their own error_page
entry.
Unanticipated errors
CherryPy also has a generic error handling mechanism: whenever an unanticipated
error occurs in your code, it will call
Request.error_response to
set the response status, headers, and body. By default, this is the same
output as
HTTPError(500). If you want to provide
some other behavior, you generally replace "request.error_response".
Here is some sample code that shows how to display a custom error message and
send an e-mail containing the error
from cherrypy import _cperror
def handle_error():
cherrypy.response.status = 500
cherrypy.response.body = [
"<html><body>Sorry, an error occurred</body></html>"
]
sendMail('error#domain.com',
'Error in your web app',
_cperror.format_exc())
#cherrypy.config(**{'request.error_response': handle_error})
class Root:
pass
Note that you have to explicitly set
response.body
and not simply return an error message as a result.
Choose what's most suitable for you: Default Methods, Custom Error Handling.
I don't think you should use BaseHTTPServer. If your app is that simple, just get a lightweight framework (e. g. Flask), even though it might be a bit overkill, OR stay low level but still within the WSGI standard and use a WSGI-compliant server.
CherryPy IS catching your exception. That's how it returns a valid page to the browser with the caught exception.
I suggest you read through all the documentation. I realize it isn't the best documentation or organized well, but if you at least skim through it the framework will make more sense. It is a small framework, but does almost everything you'd expect from a application server.
import cherrypy
def show_blank_page_on_error():
"""Instead of showing something useful to developers but
disturbing to clients we will show a blank page.
"""
cherrypy.response.status = 500
cherrypy.response.body = ''
class Root():
"""Root of the application"""
_cp_config = {'request.error_response': show_blank_page_on_error}
#cherrypy.expose
def index(self):
"""Root url handler"""
raise Exception
See this for the example in the documentation on the page mentioned above for further reference.
You can simply use a try/except clause:
try:
cherrypy.quickstart(MainServer())
except: #catches all errors, including basic python errors
print("Error!")
This will catch every single error. But if you want to catch only cherrypy._cperror:
from cherrypy import _cperror
try:
cherrypy.quickstart(MainServer())
except _cperror.CherryPyException: #catches only CherryPy errors.
print("CherryPy error!")
Hope this helps!
import cherrypy
from cherrypy import HTTPError
def handle_an_exception():
cherrypy.response.status = 500
cherrypy.response.headers['content-type'] = 'text/plain;charset=UTF-8'
cherrypy.response.body = b'Internal Server Error'
def handle_a_404(status=None, message=None, version=None, traceback=None):
cherrypy.response.headers['content-type'] = 'text/plain;charset=UTF-8'
return f'Error page for 404'.encode('UTF-8')
def handle_default(status=None, message=None, version=None, traceback=None):
cherrypy.response.headers['content-type'] = 'text/plain;charset=UTF-8'
return f'Default error page: {status}'.encode('UTF-8')
class Root:
"""Root of the application"""
_cp_config = {
# handler for an unhandled exception
'request.error_response': handle_an_exception,
# specific handler for HTTP 404 error
'error_page.404': handle_a_404,
# default handler for any other HTTP error
'error_page.default': handle_default
}
#cherrypy.expose
def index(self):
"""Root url handler"""
raise Exception("an exception")
#cherrypy.expose
def simulate400(self):
raise HTTPError(status=400, message="Bad Things Happened")
cherrypy.quickstart(Root())
Test with:
http://127.0.0.1:8080/
http://127.0.0.1:8080/simulate400
http://127.0.0.1:8080/missing
Though this was the one of the top results when I searched for cherrypy exception handling, accepted answer did not fully answered the question. Following is a working code against cherrypy 14.0.0
# Implement handler method
def exception_handler(status, message, traceback, version)
# Your logic goes here
class MyClass()
# Update configurations
_cp_config = {"error_page.default": exception_handler}
Note the method signature. Without this signature your method will not get invoked.Following are the contents of method parameters,
status : HTTP status and a description
message : Message attached to the exception
traceback : Formatted stack trace
version : Cherrypy version
Maybe you could use a 'before_error_response' handler from cherrypy.tools
#cherrypy.tools.register('before_error_response', priority=90)
def handleexception():
cherrypy.response.status = 500
cherrypy.response.body = ''
And don't forget to enable it:
tools.handleexception.on = True
I gave up using CherryPy and ended up using the follwing code, which solves the issue in a few lines with the standard BaseHTTPServer:
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
from urlparse import urlparse, parse_qs
class GetHandler(BaseHTTPRequestHandler):
def do_GET(self):
url = urlparse(self.path)
d = parse_qs(url[4])
if 'c' in d:
print d['c'][0]
self.send_response(200)
self.end_headers()
return
server = HTTPServer(('localhost', 8080), GetHandler)
server.serve_forever()
I have sleekXMPP for python and I have used the API to create functions to send messages, although when I researched into receiving them I can't find anything, can someone please help me to work this out, or disprove the possibility. Thanks.
Below is the code I used to send Messages, If its any help.
to = config.get('usermap', to[4:])
gmail_from_user = config.get('auth', 'email')
gmail_from_secret = config.get('auth', 'secret')
sys.stdout = stdouttmp
sys.stderr = stderrtmp
print "Sending chat message to " + to
xmpp = SendMsgBot(gmail_from_user, gmail_from_secret, to, message)
xmpp.register_plugin('xep_0030') # Service Discovery
xmpp.register_plugin('xep_0199') # XMPP Ping
sys.stdout = stdouttmp
if xmpp.connect(('talk.google.com', 5222)):
xmpp.process(block=True)
else:
sys.stdout = stdouttmp
print("Unable to connect.")
sys.stdout = stdouttmp
sys.stderr = stderrtmp
btw I'm using a .cfg text file for the users email and password, along with some contacts, which is then parsed in
I see that you're using the send_client.py example. The intent of that example is how to reliably log in, send a single message, and then log out. Your use case is to both send and receive messages, so you would be better served looking at the echo_client.py example.
Notably, in order to receive a message you would do:
# in your __init__ method:
def __init__(...):
# ...
self.add_event_handler('message', self.recv_message)
def recv_message(self, msg):
# You'll probably want to ignore error and headline messages.
# If you want to handle group chat messages, add 'groupchat' to the list.
if msg['type'] in ('chat', 'normal'):
print "%s says: %s" % (msg['from'], msg['body'])
Again, you will need to switch from using the SendMsgBot example because it automatically disconnects after sending its message.
Don't forget that there is the sleek#conference.jabber.org chat room if you need any help.
-- Lance