Develop Reference

Logging with WSGI server and flask application

I am using wsgi server to spawn the servers for my web application. I am having problem with information logging.
This is how I am running the app
from gevent import monkey; monkey.patch_all()
from logging.handlers import RotatingFileHandler
import logging
from app import app # this imports app
# create a file to store weblogs
log = open(ERROR_LOG_FILE, 'w'); log.seek(0); log.truncate();
log.write("Web Application Log\n"); log.close();
log_handler = RotatingFileHandler(ERROR_LOG_FILE, maxBytes =1000000, backupCount=1)
formatter = logging.Formatter(
"[%(asctime)s] {%(pathname)s:%(lineno)d} %(levelname)s - %(message)s"
)
log_handler.setFormatter(formatter)
app.logger.setLevel(logging.DEBUG)
app.logger.addHandler(log_handler)
# run the application
server= wsgi.WSGIServer(('0.0.0.0', 8080), app)
server.serve_forever()
However, on running the application it is not logging anything. I guess it must be because of WSGI server because app.logger works in the absence of WSGI. How can I log information when using WSGI?

According to the gevent uwsgi documentation you need to pass your log handler object to the WSGIServer object at creation:
log – If given, an object with a write method to which request (access) logs will be written. If not given, defaults to sys.stderr. You may pass None to disable request logging. You may use a wrapper, around e.g., logging, to support objects that don’t implement a write method. (If you pass a Logger instance, or in general something that provides a log method but not a write method, such a wrapper will automatically be created and it will be logged to at the INFO level.)
error_log – If given, a file-like object with write, writelines and flush methods to which error logs will be written. If not given, defaults to sys.stderr. You may pass None to disable error logging (not recommended). You may use a wrapper, around e.g., logging, to support objects that don’t implement the proper methods. This parameter will become the value for wsgi.errors in the WSGI environment (if not already set). (As with log, wrappers for Logger instances and the like will be created automatically and logged to at the ERROR level.)
so you should be able to do wsgi.WSGIServer(('0.0.0.0', 8080), app, log=app.logger)

You can log like this -
import logging
import logging.handlers as handlers
.
.
.
logger = logging.getLogger('MainProgram')
logger.setLevel(10)
logHandler = handlers.RotatingFileHandler(filename.log,maxBytes =1000000, backupCount=1)
logger.addHandler(logHandler)
logger.info("Logging configuration done")
.
.
.
# run the application
server= wsgi.WSGIServer(('0.0.0.0', 8080), app, log=logger)
server.serve_forever()

flask + uwsgi: No handlers could be found for logger

I am running a python flask app using uwsgi and nginx. I am having trouble getting the app modules to log. If i run the flask app by itself, i can see the logs properly formatted, but in uwsgi, i see 'no handlers could be found for logger...' and the logs are missing. prints show up fine. Could someone help with what I am doing wrong?
Thanks
I run uwsgi as
/usr/local/bin/uwsgi --ini /root/uwsgi.ini
# cat /root/uwsgi.ini
[uwsgi]
base=/root/mainapp
app = mainapp
module = %(app)
pythonpath = %(base)
socket = /tmp/mainapp.sock
chmod-socket = 666
callable = app
logto = /var/log/mainapp/app.log
paste-logger = %p
[formatters]
keys: detailed
[handlers]
keys: console
[loggers]
keys: root, module1, module2, module3
[formatter_detailed]
format: %(asctime)s %(name)s:%(levelname)s %(module)s:%(lineno)d: %(message)s
[handler_console]
class: StreamHandler
args: []
formatter: detailed
[logger_root]
level: DEBUG
handlers:
[logger_module1]
level: DEBUG
qualname: module1
handlers: console
[logger_module2]
level: DEBUG
qualname: module2
handlers: console
[logger_module3]
level: DEBUG
qualname: module3
handlers: console
and in the module, i call
import logging
log = logging.getLogger('module1')
log.info('hello world')

Have you configured Flask to use the logger? Off the top of my head something like this should work
app.config['LOG_FILE'] = 'application.log'
# Configure logger.
if not app.debug:
import logging
from logging import FileHandler
file_handler = FileHandler(app.config['LOG_FILE'])
file_handler.setLevel(logging.WARNING)
app.logger.addHandler(file_handler)

looks like i have to instantiate flask's app.logger before i can do anything...this did the trick...set addhandler on app.logger and
import logging
import logging.config
shandler = logging.StreamHandler()
shandler.setLevel(logging.DEBUG)
app.logger.addHandler(shandler)
logging.config.fileConfig('logging.conf')

Pyramid uWSGI logging in daemon mode is not working

I've been trying multiple things on this one, but with no success.
I want to save log to file (SqlAlchemy logs, app debug logs, stack traces on errors, etc.).
I'm starting uwsgi with the following command:
uwsgi --ini-paste-logged myapp.ini
And here is the content of the ini file (where apiservice is my pakage)
[loggers]
keys = root, apiservice, sqlalchemy
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = INFO
handlers = console
[logger_apiservice]
level = DEBUG
handlers =
qualname = apiservice
[logger_sqlalchemy]
level = INFO
handlers =
qualname = sqlalchemy.engine
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s
[uwsgi]
socket = /tmp/myapp-uwsgi.sock
virtualenv = /var/www/myapp/env
pidfile = ./uwsgi.pid
daemonize = ./uwsgi.log
master = true
processes = 4
The uwsgi.log contains only request log, without any actual logging data.
I've tried with INI options like:
paste: config:%p
paste-logger: %p
logto: file
Nothing seem to work.

Apparently, the uwsgi config section was fine.
After closer look at the uwsgi.log, even though the server was launched and running successfully, you could see an error:
ImportError: No module named script.util.logging_config
I've installed following packages to solve my problems:
pip install pastescript
pip install pastedeploy

Logging configuration for Flask applications run by paster.deploy

The access log and root log for my Flask app was helped by zcbuildout. That's fine. Now I wondered how do I get logging from my own app. I know how to use logging library but paster just do not log it in the console or anywhere.
Thanks
Here's my config:
[loggers]
keys = root, wsgi, myapp
[handlers]
keys = console, accesslog
[formatters]
keys = generic, accesslog
[formatter_generic]
format = %(asctime)s %(levelname)s [%(name)s] %(message)s
[formatter_accesslog]
format = %(message)s
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[handler_accesslog]
class = FileHandler
args = (os.path.join(r'.', 'access.log'), 'a')
level = INFO
formatter = accesslog
[logger_root]
level = INFO
handlers = console
[logger_wsgi]
level = INFO
handlers = accesslog
qualname = wsgi
propagate = 0
[logger_myapp]
level = DEBUG
handlers = console
qualname = myapp
[filter:translogger]
use = egg:Paste#translogger
setup_console_handler = False
logger_name = wsgi
[app:main]
use = egg:myapp#debug
filter-with = translogger
...
Here's how I tried to log:
import logging as log
def myfunc():
log.debug("show me the log")

After a long time, this logging problem was resolved. Referring to the manual, Flask has configured its own logger. So to do logging, please use flask.logger
Documentation is here:
http://flask.pocoo.org/docs/api/#flask.Flask.logger

Why is my log level not being used when using loadapp from paste.deploy?

I want to temporailiy turn on debug messages in a production pyramid web project so I adjusted the production.ini file, pushed it to Heroku and saw only error and warn level messages.
So I thought, that seems odd since if I start the pyramid application like the following on my local PC I get all the log level messages.
env/bin/pserve production.ini
OK, so that's not exactly how it runs on Heroku, it is actually run from a little bit of python that looks like this (in a file called runapp.py):
import os
from paste.deploy import loadapp
from waitress import serve
if __name__ == "__main__":
port = int(os.environ.get("PORT", 5000))
app = loadapp('config:production.ini', relative_to='.')
serve(app, host='0.0.0.0', port=port)
Now, sure enough if I do this on my local PC I get the same behavior as when it is deployed to Heroku (hardly surprising).
python runapp.py
My question is, what am I missing here? Why does running it the second way result in no log messages other than ERROR and WARN being output to standard out? Surely, since it is using the same production.ini file it should work the same as if I use the pserve process?
Here is my logging section from production.ini:
###
# logging configuration
# http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/logging.html
###
[loggers]
keys = root, test
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = DEBUG
handlers = console
[logger_test]
level = DEBUG
handlers = console
qualname = test
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = DEBUG
formatter = generic
[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s

PasteDeploy does not actually assume responsibility for configuring logging. This is a little quirk where the INI file is dual-purposed. There are sections that PasteDeploy cares about, and there are sections that logging.config.fileConfig cares about, and both must be run to fully load an INI file.
If you follow the pyramid wrappers for doing this, you'd do:
pyramid.paster.setup_logging(inipath)
pyramid.paster.get_app(inipath)
The main reason you would use these instead of doing it yourself is that they support doing "the right thing" when inipath contains a section specifier like development.ini#myapp, which fileConfig would crash on.