I've been learning about logging, and got help here earlier on setting up a logger w/external config file.
I've setup based on the example, however the messages only seen on the console and not in a long file (not created.
Can you please see what I'm doing wrong?
utilityLogger:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
'''
My app
'''
# ~~~~~ LOGGING SETUP ~~~~~ #
# set up the first logger for the app
import os
import testLogging as vlog
# path to the current script's dir
scriptdir = os.path.dirname(os.path.realpath(__file__))
LOG_CONFIG = '../config/logging.conf'
print scriptdir
def logpath():
'''
Return the path to the main log file; needed by the logging.yml
use this for dynamic output log file paths & names
'''
global scriptdir
return (vlog.logpath(scriptdir = scriptdir, logfile = 'log.txt'))
logger = vlog.log_setup(config_file=LOG_CONFIG, logger_name="app")
logger.debug("App is starting...")
testLogging:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
'''
Functions to set up the app logger
'''
import logging
import logging.config
import os
LOG_CONFIG = '../config/logging.conf'
def logpath(scriptdir, logfile):
'''
Return the path to the main log file; needed by the logging.yml
use this for dynamic output log file paths & names
'''
log_file = os.path.join(scriptdir, logfile)
print log_file
print scriptdir
print logfile
return(logging.FileHandler(log_file))
def log_setup(config_file, logger_name):
'''
Set up the logger for the script
config = path to YAML config file
'''
# Config file relative to this file
logging.config.fileConfig(config_file)
return(logging.getLogger(logger_name))
logging.conf file:
[loggers]
keys=root
[handlers]
keys=consoleHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=DEBUG
handlers=consoleHandler
qualname=app
[logger_app]
level=DEBUG
handlers=consoleHandler
qualname=app
propagate=true
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)
[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=fileFormatter
args=('%(logfilename)s',)
[main]
()=__main__.logpath
level=DEBUG
formatter=simpleFormatter
[formatter_fileFormatter]
format=%(asctime)s (%(name)s:%(funcName)s:%(lineno)d:%(levelname)s) %
(message)s # %(module)s:
datefmt="%Y-%m-%d %H:%M:%S"
[formatter_simpleFormatter]
format=%(asctime)s (%(name)s:%(funcName)s:%(lineno)d:%(levelname)s) %(message)s # %(module)s:
datefmt="%Y-%m-%d %H:%M:%S"
Update, the question has already been marked as answered and I appreciate #zwer help!
Last objective, to understand, is there more pythonic way to instantiate a logger to Class (but I want to be able to log in main as well). With the marked answer I've put together the following, but I'm not sure it's the most elegant solution for both main and classes logging.
class TestLog(object):
def __init__(self, logger):
self.logger = logger
self.__sub_test = 0
def add_test(self):
self.logger.debug('addition')
a = 1 + 1
self.logger.debug('result {}'.format(a, 1))
def sub_test(self):
self.logger.debug('subtraction')
b = 5 -2
self.logger.debug('result {}'.format(b, 1))
def main():
logger = vlog.log_setup(config_file=LOG_CONFIG, logger_name="app",
log_file=LOG_PATH)
logger.debug("App is starting...")
test1 = TestLog(logger)
print test1.add_test()
print test1.sub_test()
if __name__ == "__main__":
sys.exit(main())
Alright, let's pack it as an answer to avoid comment constraints.
The main issue with your config is that you're not initializing your fileHandler at all. If you want to use it, make sure you add it to the [handlers] section, e.g.:
[handlers]
keys=fileHandler
As for your other error, since in your [handler_fileHandler] you define a dynamic argument logfilename for the name of the file so you need to provide it when you're loading your logging config in Python, e.g.:
logging.config.fileConfig(config_file, defaults={"logfilename": "your_log_filename.log"})
That should do the trick.
UPDATE - As long as you provide a proper file path, the stated should work, but you still need to modify your config a little bit more to enable the file logger in all your loggers. So change your config to:
[loggers]
keys=root
[handlers]
keys=consoleHandler,fileHandler
[formatters]
keys=simpleFormatter,fileFormatter
[logger_root]
level=DEBUG
handlers=consoleHandler,fileHandler
qualname=app
[logger_app]
level=DEBUG
handlers=consoleHandler,fileHandler
qualname=app
propagate=true
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)
[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=fileFormatter
args=('%(logfilename)s',)
[main]
()=__main__.logpath
level=DEBUG
formatter=simpleFormatter
[formatter_fileFormatter]
format=%(asctime)s (%(name)s:%(funcName)s:%(lineno)d:%(levelname)s) %(message)s # %(module)s:
datefmt="%Y-%m-%d %H:%M:%S"
[formatter_simpleFormatter]
format=%(asctime)s (%(name)s:%(funcName)s:%(lineno)d:%(levelname)s) %(message)s # %(module)s:
datefmt="%Y-%m-%d %H:%M:%S"
Also, to make it more flexible, change your testLogging.log_setup() to something like:
def log_setup(config_file, logger_name, log_file):
# Config file relative to this file
logging.config.fileConfig(config_file, defaults={"logfilename": log_file})
return logging.getLogger(logger_name)
And finally, when you're setting it up just invoke it as:
LOG_CONFIG = '../config/logging.conf'
LOG_PATH = r"C:\PycharmProjects\scrap\test.log" # make sure it exists and is accessible!
logger = vlog.log_setup(config_file=LOG_CONFIG, logger_name="app", log_file=LOG_PATH)
logger.debug("App is starting...")
Adjusted for your local paths it should work as expected. I just tested it on my side and it's giving a proper result.
Related
How do I get imported modules to show up in my filehandler without specifying each one?
When I run the "main.py", the 'sLogger' information is specified in the "log.txt" file, but not the 'module_a' information. The 'module_a' info only shows up in the console window.
I know I can fix this by adding a separate logger for 'module_a', but don't want to do this for every single module I want to get logging information from.
I thought setting 'disable_existing_loggers=False' to False would resolve this, but it isn't.
Am I doing something incorrectly there?
I also don't want to push everything with the root logger.
If I add fileHandler to the root I get everything, but I don't want to get info from the root logger.
main.py
import logging
from logging import config
logging.config.fileConfig("logger_config.txt", defaults={'logfilename': "test.log"}, disable_existing_loggers=False)
logger = logging.getLogger("sLogger")
logger.info("test")
logging.info("test")
import module_a
module_A.py:
logger = loging.getLogger(__name__)
logger.info("Test Log Message")
file_config.txt
[loggers]
keys=root, sLogger
[handlers]
keys=consoleHandler,fileHandler
[formatters]
keys=fileFormatter,consoleFormatter
[logger_root]
level=DEBUG
handlers=consoleHandler
[logger_sLogger]
level=DEBUG
handlers=consoleHandler,fileHandler
qualname=sLogger
propagate=0
[logger_parser]
level=DEBUG
handlers=consoleHandler,fileHandler
qualname=sLogger
propagate=1
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=consoleFormatter
args=(sys.stdout,)
[handler_fileHandler]
class=FileHandler
level=INFO
formatter=fileFormatter
args=('%(logfilename)s',)
[formatter_fileFormatter]
format=%(asctime)s - %(name)s - %(levelname)s: %(message)s
datefmt=
[formatter_consoleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s: %(message)s
datefmt=
I'm having a bit of an issue here. I have set up a scratch py file just to test out my logging. Neither the consoleHandler nor the fileHandler are getting the right output (any output). Can anyone see if they can eyeball any issues? Here is my log config file
[loggers]
keys=root
[handlers]
keys=fileHandler, consoleHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=DEBUG
handlers=fileHandler, consoleHandler
formatter=simpleFormatter
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout, )
[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=simpleFormatter
args=(os.path.join(os.getcwd(), 'logging.log'), 'w')
[formatter_simpleFormatter]
format=%(asctime)s - %(levelname)s - %(message)s
datefmt="%Y-%m-%d %H:%M:%S"
Here is the scratch python file,
import logging
from logging import config
LOGGER_NAME = 'Work'
logger = logging.getLogger(LOGGER_NAME)
logging.config.fileConfig('C:\\Users\\cschuma1\\PycharmProjects\\workstuff\\config\\logging.conf')
addition = 'add' + 'ition'
logger.debug('print addition %s', addition)
I have looked closely at other's configs on stackoverflow and I believe that I have all of the necessary levels set, and formatter/handler objects, etc. for my task. Does anyone know what I am missing?
It turns out that at least in python 3, the line
logging.config.fileConfig('C:\\Users\\cschuma1\\PycharmProjects\\workstuff\\config\\logging.conf')
needs to come before,
LOGGER_NAME = 'Work'
logger = logging.getLogger(LOGGER_NAME)
This is the fixed sample program I had above,
import logging
from logging import config
logging.config.fileConfig('C:\\Users\\cschuma1\\PycharmProjects\\workstuff\\config\\logging.conf')
LOGGER_NAME = 'Work'
logger = logging.getLogger(LOGGER_NAME)
addition = 'add' + 'ition'
logger.debug('print addition %s', addition)
So I am trying to implement logging within my Python program. The goal is to set it up so that a log file is created and everything the program does through it's various modules is logged (based on logging level). This is what my current code looks like:
Text File for Log Configuration:
#logging.conf
[loggers]
keys=root,MainLogger
[handlers]
keys=consoleHandler
[formatters]
keys=consoleFormatter
[logger_root]
level=DEBUG
handlers=consoleHandler
[logger_MainLogger]
level=DEBUG
handlers=consoleHandler
qualname=MainLogger
propagate=0
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=consoleFormatter
args=(sys.stdout,)
[formatter_consoleFormatter]
format=%(asctime)s | %(levelname)-8s | %(filename)s-%(funcName)s-%lineno)04d | %(message)s
External Module to Test Logs:
#test.py
import logging
logger = logging.getLogger(__name__)
def testLog():
logger.debug("Debug Test")
logger.info("Info Test")
logger.warning("Warning Test")
logger.error("Error Test")
Main file:
#__init__.py
import logging
import logging.config
from datetime import datetime
logging.config.fileConfig('logging.conf', disable_existing_loggers = False)
logger = logging.getLogger('MainLogger')
fileHandler = logging.FileHandler('{:%Y-%m-%d}.log'.format(datetime.now()))
formatter = logging.Formatter('%(asctime)s | %(levelname)-8s | %(lineno)04d | %(message)s')
fileHandler.setFormatter(formatter)
logger.addHandler(fileHandler)
if __name__ == "__main__":
import test
logger.debug("Debug Test")
test.testLog()
Currently, all log messages are currently being displayed withing the IDLE3 shell when I run __init__.py and the log file is being created. However within the log file itself the only message being recording is the "Debug Test" from __init__.py. None of the messages from the test.py module are being recorded in the log file.
What is my problem?
In test.py it grabs a logger object before you configure it later in your __init__.py. Make sure you configure the logging module first before grabbing any logger instance.
Why in python logger.info("print something") does not output. I have seen questions asked before, but solution doesnt exist. I do not want to use logger.debug or logger.warning to see text.
Simply logger.info should print the text, otherwise whats the use of this?
logging.conf file as below
[loggers]
keys=root
[handlers]
keys=stream
[formatters]
keys=formatter
[logger_root]
level=INFO
handlers=stream
[handler_stream]
class=StreamHandler
level=INFO
formatter=formatter
args=(sys.stderr,)
[formatter_formatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
Demo code that access logger:
import logging
logger = logging.getLogger()
if __name__ == '__main__':
logger.info("logger")
print("print")
Output is only print, not the logger. So logger.info does not work.
By default, the root logger (the one you use when you say logger.info) is set at a level of WARN.
You can either do:
logging.basicConfig(level=logging.INFO)
or logging.getLogger().setLevel(logging.INFO)
Seems you do not load your configuration file. You should add this:
logging.config.fileConfig('path_to_logging.conf')
before logger = logging.getLogger()
because right now you are using the default WARNING level.
EDIT: in order to use logging.config, you have to import it too:
import logging.config
So the complete code should be:
import logging
import logging.config
logging.config.fileConfig('path_to_logging.conf')
logger = logging.getLogger()
if __name__ == '__main__':
logger.info("logger")
print("print")
The code above, with the following logging.conf (same as you except I removed the sentry parts):
[loggers]
keys=root
[handlers]
keys=stream
[formatters]
keys=formatter
[logger_root]
level=INFO
handlers=stream
[handler_stream]
class=StreamHandler
level=INFO
formatter=formatter
args=(sys.stderr,)
[formatter_formatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
does work:
$ ./test_script3.py
2016-05-23 15:37:40,437 - root - INFO - logger
print
I want to use a configuration file to create two loggers which will log in two distinct files. My configuration file looks like:
[loggers]
keys=root,main,zipper
[handlers]
keys=main,zip
[formatters]
keys=fmt
[logger_root]
level=DEBUG
handlers=main
[logger_main]
level=DEBUG
handlers=main
qualname=MAIN
[logger_zipper]
level=DEBUG
qualname=UPLOAD
handlers=zip
[handler_zip]
class=FileHandler
level=DEBUG
formatter=fmt
args=('zipper.log','a+')
[handler_main]
class=FileHandler
level=DEBUG
formatter=fmt
args=('main.log','a+')
[formatter_fmt]
format=%(asctime)s - %(levelname)s - %(name)s - %(message)s
I try to use this configuration file like this:
import logging
import logging.config
logging.config.fileConfig("logging.conf")
# Logs to the first file
log = logging.getLogger("")
log.debug("unspec - debug")
log.error("unspec - error")
# Logs to the first file
log_r = logging.getLogger("main")
log_r.debug("main - debug")
log_r.error("main - error")
# Also logs to the first file :(
log_z = logging.getLogger("zipper")
log_z.debug("zipper - debug")
log_z.error("zipper - error")
For some reason I don't understand, all logging messages go to the first file, when I expect the last two to be written to 'zipper.log'. What am I missing ?
The problem is that the qualified name used in the configuration file:
[logger_zipper]
level=DEBUG
qualname=UPLOAD
handlers=zip
doesn't match the one used in the code:
log_z = logging.getLogger("zipper")
Use any of these combinations:
qualname=zipper and logging.getLogger("zipper")
qualname=UPLOAD and logging.getLogger("UPLOAD")