My cherrypy application (3.2.2 on Python 2.6) is using 3rd party libs, and these libs use standard logging internally, like so:
logger = logging.getLogger(__name__)
logger.info("a message from some library")
Now, in my cherrypy config, I have:
log.access_file = '/path/access.log'
log.error_file = '/path/error.log'
but only CP messages ever appear in these two files, not any of the other logging. But I need all logging there, not just the logging that CP itself issues internally.
Is there a way to capture all output, including the 3rd party logging (which I assume goes into stdout/stderr and then disappears, as the process is a detached daemon), into /path/error.log? Plug arbitrary stdout/stderr into CP's log somehow?
Is this what you're looking for? All output into the error log?
python yourCherryServer.py &>> /path/error.log
Related
I just discovered this very strange behaviour of the logging module in Spyder:
import logging
logging.getLogger("hurricane").handlers
Out[2]: [] # expected
logging.getLogger("tornado").handlers
Out[3]: [<StreamHandler <stderr> (NOTSET)>] # Where does that StreamHandler come from?!
Note that these are the first lines from a freshly started interpreter. So I haven't imported tornado or any other package except logging. Yet, unlike any other logger I tried to get, it comes with a StreamHandler attached.
Why?
Related question: How to prevent Python Tornado from logging to stdout/console?
I think Tornado uses logging for its normal-operation request logging. This is not a great idea - an application is supposed to work exactly as before if all logging is disabled (aside from the logging part, of course) but numerous web server authors use logging functionality as a convenience to do part of their normal operation rather than just using it for diagnostics. I believe you can turn this off using a logging=none configuration option; see this page for more information.
I use Python standard logging extensively in my libraries within my Robot Framework test suites. These log messages are appearing in the RF log, as expected, save for two problems:
Some of the libraries create threads. Log messages on these additional threads do not reach the RF log.
For each library I follow the standard practice of creating a logging channel named after the module/class self._logger = logging.getLogger( __name__ ) , but I cannot seem to format the logging in any way to get these channel names to appear in the RF logs.
If I run these libraries from regular Python scripts, instead of RF, I get log messages from the additional threads, and I can format all messages to show the channel names. So there is some issue when using them within RF.
I'm using RF3, Python3 and running under Raspbian.
Quote from the Robot Framework User Guide:
"Messages logged by non-main threads using the normal logging methods from programmatic logging APIs are silently ignored."
http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#communication-when-using-threads
You could try this user contributed module as a workaround:
https://github.com/robotframework/robotbackgroundlogger
In an application I want to collect messages related to some dedicated part of the processing, and then show these messages later at user request. I would like to report severity (e.g. info, warning), time of message, etc., so for this I considered to use the Python standard logging module to collect the messages and related information. However, I don't want these messages to go to a console or file.
Is there a way to create a Python logger, using logging, where the messages are kept internally (in memory) only, until read out by the application. I would expect start of code like:
log = logging.getLogger('my_logger')
... some config of log for internal only; not to console
log.error('Just some error')
... some code to get/clear messages in log until now
I have tried to look in logging — Logging facility for Python, but most example are for immediate output to file or console, so an example for internal logging or reference is appreciated.
You should just use another handler. You could use a StreamHandler over an io.StringIO that would simply log to memory:
log = logging.getLogger('my_logger')
memlog = io.StringIO()
log.addHandler(logging.StreamHandler(memlog))
All logging sent to log can be found in memlog.getvalue()
Of course, this is just a simple Handler that concatenates everything in one single string, even if for versions >= 3.2 each record is terminated, by default with a \n. For more specific requirements, you could have a look at a QueueHandler or implement a dedicated Handler.
References: logging.handlers in the Python Standard Library reference manual.
I'm looking at how to log to syslog from within my Python app, and I found there are two ways of doing it:
Using syslog.syslog() routines
Using the logger module SysLogHandler
Which is the best option to use, advantages/disadvantages of each one, etc, because I really don't know which one should I use.
syslog.syslog() can only be used to send messages to the local syslogd. SysLogHandler can be used as part of a comprehensive, configurable logging subsystem, and can log to remote machines.
The logging module is a more comprehensive solution that can potentially handle all of your log messages, and is very flexible. For instance, you can setup multiple handers for your logger and each can be set to log at a different level. You can have a SysLogHandler for sending errors to syslog, and a FileHandler for debugging logs, and an SMTPHandler to email the really critical messages to ops. You can also define a hierarchy of loggers within your modules, and each one has its own level so you can enable/disable messages from specific modules, such as:
import logging
logger = logging.getLogger('package.stable_module')
logger.setLevel(logging.WARNING)
And in another module:
import logging
logger = logging.getLogger('package.buggy_module')
logger.setLevel(logging.DEBUG)
The log messages in both of the these modules will be sent, depending on the level, to the 'package' logger and ultimately to the handlers you've defined. You can also add handlers directly to the module loggers, and so on. If you've followed along this far and are still interested, then I recommend jumping to the logging tutorial for more details.
So far, there is a disadvantage in logging.handlers.SysLogHander which is not mentioned yet. That is I can't set options like LOG_ODELAY or LOG_NOWAIT or LOG_PID. On the other hands, LOG_CONS and LOG_PERROR can be achieved with adding more handlers, and LOG_NDELAY is already set by default, because the connection opens when the handler is instantiated.
Sometimes I need to check the output of a python web apllication.
If I execute the application directly, I can see it from terminal screen.
But I have no idea how to check that for mod_wsgi. Will it appear in a seperate log of apache? Or do I need to add some codes for logging?
Instead of print "message", you could use sys.stderr.write("message")
For logging to stderr with a StreamHandler:
import logging
handler = logging.StreamHandler(stream=sys.stderr)
log = logging.getLogger(__name__)
log.setLevel(logging.INFO)
log.addHandler(handler)
log.info("Message")
wsgilog is simple, and can redirect standard out to a log file automatically for you. Breeze to set up, and I haven't had any real problems with it.
No WSGI application component which claims to be portable should write to standard output. That is, an application should not use the Python print statement without directing output to some alternate stream. An application should also not write directly to sys.stdout. (ModWSGI Wiki)
So don't... Instead, I recommend using a log aggregation tool like sentry. It is useful while developing and a must-have in production.