Develop Reference

How to make a log system for tornado

I've read the documentation: http://www.tornadoweb.org/en/stable/log.html
But I still don't know how to make a suitable log for my server, which is built with tornado.
For now, I need such a log system:
It can log everything with time format, and for each day it create a new log file.
It seems that TimedRotatingFileHandler is what I need but I don't know how to use it with tornado.

The Tornado logging streams are just standard loggers from the "logging" python module.
There is nice tutorial on the python website https://docs.python.org/3/howto/logging.html#advanced-logging-tutorial
As per how to set the handler (same tutorial)
https://docs.python.org/3/howto/logging.html#handlers

Heroku Django Logging

I'm trying to debug an internal server error in my django app, running on heroku. I'm
completely new to all of this web server stuff so I really have no idea what to do.
It seems like the stdout output is sometimes getting logged in heroku logs and sometimes not. I was reasonably sure that the program was reaching a certain line but the prints at that point are simply not showing up.
I am seeing the 500 error in my heroku logs file, but there is no stack trace or anything else in there. I am trying to create a web server to respond to GET and POST requests from various applications I have running, meaning I don't know how to debug this in a web browser, if thats even applicable. The current error is on a POST request sent to the webserver. I can't replicate this locally because the Http module I am using, http://www.python-requests.org/en/latest/ seems to be unable to connect to a local ip address.
I have done some extensive googling for the last hour and I haven't found any help. Do I need to enable logging or something somewhere in heroku? I am completely new to this so please be explicit in your explanations. I have heard mention of a way to get stack traces emailed to you but I haven't seen an explanation of how to do that. is that possible?
Thanks!

I would recommend 2 things in this case:
First: use python's logging facility rather than print statements (http://docs.python.org/2/howto/logging-cookbook.html). This gives you much more control over where your statements end up, and allows you to filter them.
Second: use a logging add-on. This vastly increases the amount of logging you can store (loggly keeps all your logs for 24 hours even in the "free" size), so you don't have to worry about the relevant information falling out before you get around to looking at it.

Python RabbitMQ and kombu: heartbeats

I have a Twisted application (Python 2.7) that uses the kombu module to communicate with a RabbitMQ message server.
We're having problems with closing connections (probably firewall related) and I'm trying to implement the heartbeat_check() method to handle this. I've got a heartbeat value of 10 set on the connection and I've got a Twisted LoopingCall that calls the heartbeat_check(rate=2) method every 5 seconds.
However, once things get rolling I'm getting an exception thrown every other call to heartbeat_check() (based on the logging information I've got in the function that LoopingCall calls, which includes the call to heartbeat_check). I've tried all kinds of variations of heartbeat and rate values and nothing seems to help. When I debug into the heartbeat_call() it looks like some minor message traffic is being sent back and forth, am I supposed to respond to that in my message consumer?
Any hints or pointers would be very helpful, thanks in advance!!
Doug

generating errors and access logs in python

We are realizing that we need error logs and access logs for our service processes. These are long running process like services; they respond to calls made to them.
Hence, we need your help to simply achieve the following:
# for developers
from MyLogger import log
log.error 'something bad something wrong'
log.access 'something something'
I am thinking of designing this MyLogger which will simply redirect an error to stderr and access to stdout, so that I can collect errors to a specific file through configuration for both stderr and stdout.
One more point: these services are nothing but web.py instances.
I guess I'm not looking for a controlling log at various levels, like warn, debug, error, info, etc. My aim is more to have an error log and access log similar to the apache web server. So my developers should not be concerned about using warn, debug, etc. as follows:
log.warn msg
log.debug msg
This is not required.
I just want to have an error log and an access log similar to that of a web server or service.

That "battery" is already included for you in the Python standard logging module.

Python ships with the logging module.

Python logging SMTPHandler - handling offline SMTP server

I have setup the logging module for my new python script. I have two handlers, one sending stuff to a file, and one for email alerts. The SMTPHandler is setup to mail anything at the ERROR level or above.
Everything works great, unless the SMTP connection fails. If the SMTP server does not respond or authentication fails (it requires SMTP auth), then the whole script dies.
I am fairly new to python, so I am trying to figure out how to capture the exception that the SMTPHandler is raising so that any problems sending the log message via email won't bring down my entire script. Since I am also writing errors to a log file, if the SMTP alert fails, I just want to keep going, not halt anything.
If I need a "try:" statement, would it go around the logging.handlers.SMTPHandler setup, or around the individual calls to my_logger.error()?

Exceptions which occur during logging should not stop your script, though they may cause a traceback to be printed to sys.stderr. In order to prevent this printout, do the following:
logging.raiseExceptions = 0
This is not the default (because in development you typically want to know about failures) but in production, raiseExceptions should not be set.
You should find that the SMTPHandler will attempt a re-connection the next time an ERROR (or higher) is logged.
logging does pass through SystemExit and KeyboardInterrupt exceptions, but all others should be handled so that they do not cause termination of an application which uses logging. If you find that this is not the case, please post specific details of the exceptions which are getting through and causing your script to terminate, and about your version of Python/operating system. Bear in mind that the script may appear to hang if there is a network timeout which is causing the handler to block (e.g. if a DNS lookup takes a long time, or if the SMTP connection takes a long time).

You probably need to do both. To figure this out, I suggest to install a local mail server and use that. This way, you can shut it down while your script runs and note down the error message.
To keep the code maintainable, you should extends SMTPHandler in such a way that you can handle the exceptions in a single place (instead of wrapping every logger call with try-except).