I am using wsgi as suggested in the cookbook, and it logs the webpy messages into the log. I am really struggling with adding my own messages to the logs.
eg: I want to add query parameters and other warnings.
For those with similar problems please use:
logger = web.ctx.env.get('wsgilog.logger')
to get the logger and then,
logger.info('hello world')
to log the message
Related
I’m developing a Python Django REST API. Currently, I need to incorporate a logger to Syslog.
Do I need to just define the logger and Syslog handler in the Settings.py?
I’m relatively new to Django and using the Syslog protocol, so I appreciate any help and what Python modules to use.
Yes, you can for example use the syslog library to log messages.
Using it, is pretty easy.
import syslog
# create syslog handle
syslog.openlog(ident="settings.py", facility=syslog.LOG_LOCAL0)
# log some message
syslog.syslog(syslog.LOG_INFO, "Some log message.")
The log message will not get written to /var/log/local0.log - probably it will also get written to /var/log/rsyslog or /var/log/messages depending on the standard configuration of your /etc/rsyslog.conf file.
Our log formatter has a user defined parameter. It is defined as:
'%(asctime)s|%(levelname)s|%(name)s|REQID:%(req_id)s|%(module)s:%(lineno)s|%(message)s'
where req_id is a request id, generated by application code for every request. When we are processing the requests, on our application code, we have access to this req_id, and we use it for logging purposes like this:
logger = logging.LoggerAdapter(logging.getLogger(service_name), {'req_id': req_id})
logger.debug('A debug message')
I am trying to make the tornado logger conforming to our log format, but since tornado has no access to our application level req_id it fails with:
KeyError: 'req_id'
How can I tell tornado to use a LoggerAdapter for tornado.access, with a user provided context?
EDIT
As a workaround, I tried the following:
Since it is not possible for me to tell tornado what loggers to use, I managed to hack my way around this limitation by reconfiguring the tornado logger in each request, adding the contextual information using a logging filter.
Unfortunately, reconfiguring the log for each request does not work since tornado will be serving requests in parallel, and we will get an inconsistent state.
How can we pass user context for the tornado loggers then?
The tornado.access log can be controlled by overriding Application.log_request in a subclass or using the log_function application setting. This method defaults to writing to the tornado.access log, but you can override it to log however you want.
Note however that the tornado.general and tornado.application loggers cannot be overridden in this way, so your log formatters/filters must still be able to handle messages that do not have the req_id field.
The Python logger library has the option of logging timestamps and file information in log file/console using the Formatter class as below:
import logging
logformatter = logging.Formatter('%(asctime)s (%(filename)s:%(lineno)s)- %(levelname)s - %(message)s')
streamlogger = logging.getLogger()
streamlogger.setLevel('DEBUG')
consolelogger = logging.StreamHandler()
consolelogger.setFormatter(logformatter)
consolelogger.setLevel('DEBUG')
streamlogger.addHandler(consolelogger)
streamlogger.debug('ZiZi')
and the output would look like this this:
2017-01-19 16:06:15,381 (testlogger.py:19)- DEBUG - ZiZi
In Robot Framework, the keyword LOG is used to log into report file and/or console. There is also a LOG TO CONSOLE keyword which only prints out given message into console. But none of these two keywords have an API for deploying what Formatter performs in Python's logging library.
Is there any trick to embed this functionality into Robot Framework? Are there any other Robot Framework keywords/libraries which I'm not aware of?
In my mind there are two ways you can achieve this kind of logging. Both of them would generate a new file, in your desired format.
The first one is to use the Robot Listener functionality. This is a set of predefined events that you can create a class for. Log Message and Message being two of particular interest to you.
The other one is a recently released project Robot Background logger that extends the standard logger class of Robotframework. This should provide some control over the formatting of the message.
Is it possible to hook into Django's built in error reporting emails from a try-except code block? In other words, email the default error report and stack trace to the ADMINS/MANAGERS while still having situation specific error handling.
Specific example:
In a project performing complex calculations and generating large reports, the view displaying the report page does all the calculations and generates a long html page with lots of pretty tables and graphs and also generates downloadable PDFs from sections of that same HTML.
Recently we had errors in the PDF generation from issues with storage on S3. Now this is obviously an error we need to track down and attend, but most users are happy if they can just see the report on screen. If the PDFs download links just weren't displayed the issue could go entirely unnoticed for hours or even days - but the dev team should be notified.
Ideally, but not necessarily, I would love a solution that is logger agnostic, where it will use whatever error logger is used and trigger the default 500 error handler, and return back to the finally block or after the except block.
All you need to do is to use Python's logging framework to raise an appropriate message at the appropriate level. In your settings.py there is a LOGGING variable that defines how things are logged. By default I believe Django has any ERROR in django.request will be handled by mail_admins.
So in your code, all you need to do is
import logging
logger = logging.getLogger(__name__) # this will create a logger with the module being the logger name
try:
#do stuff you watch to catch
except:
# we're going to catch and just log it
logger.error('Some error title', exc_info=True) # exc_info=True will include the stacktrace
finally:
# what you want to do in your finally block.
Note, this will swallow the exception and won't bubble it up. Your response will return as a 200. If you want to bubble up the exception, just call raise in your except block. However, if all you care about is logging the error, but the view is still functional, then just log and swallow it.
In your LOGGING variable, you can add additional entries to loggers for the different logger names. You can have an app log at a different logging status, say INFO if you want to debug a certain code path. As long as you create a logger with the module name, you have a lot of flexibility of segmenting your logging to different handlers such as mail_admins.
Lastly, I'd recommend to look into sentry, as it's a really great error logging tool.
I'm new to GAE, and have not been able to figure out how to configure 'print' statements to the logging console rather than the browser. For example:
class Feed(webapp.RequestHandler):
def post(self):
feeditem = Feeditem()
feeditem.author = self.request.get('from')
feeditem.content = self.request.get('content')
feeditem.put()
notify_friends(feeditem)
self.redirect('/')
def notify_friends(feeditem):
"""Alerts friends of a new feeditem"""
print 'Feeditem = ', feeditem
When I do something like the above, the print in notify_friends outputs to the browser and somehow prevents the self.redirect('/') in the post method that called it. Commenting it out corrects the issue.
Is there a way to change this behavior?
EDIT: Google App Engine tag removed as this is general.
You should instead use the logging module, like so:
import logging
def notify_friends(feeditem):
"""Alerts friends of a new feeditem"""
logging.info('Feeditem = %s', feeditem)
There are a variety of logging levels you can use, from debug to critical. By default, though, the App Engine SDK only shows you log messages at level info and above, so that's what I've suggested here. You can ask it to show you debug messages if you want, but you'll probably be overwhelmed with useless (to you) logging information, at least when running in the SDK.
See the logging module docs for more info.
Oh, and the nice thing about using the logging module is that you'll have access to your log messages in production, under the "Logs" section of your app's the App Engine dashboard.
This is not just a problem with GAE. It is a general issue. You can't print out HTML and then try to have a redirect header. The answer to your question is no, you can't change the behavior. What exactly are you trying to achieve? You might be able to get what you want a different way.