I seem to be having some issues while attempting to implement logging into my python project.
I'm simply attempting to mimic the following configuration:
Python Logging to Multiple Destinations
However instead of doing this inside of code, I'd like to have it in a configuration file.
Below is my config file:
[loggers]
keys=root
[logger_root]
handlers=screen,file
[formatters]
keys=simple,complex
[formatter_simple]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
[formatter_complex]
format=%(asctime)s - %(name)s - %(levelname)s - %(module)s : %(lineno)d - %(message)s
[handlers]
keys=file,screen
[handler_file]
class=handlers.TimedRotatingFileHandler
interval=midnight
backupCount=5
formatter=complex
level=DEBUG
args=('logs/testSuite.log',)
[handler_screen]
class=StreamHandler
formatter=simple
level=INFO
args=(sys.stdout,)
The problem is that my screen output looks like:
2010-12-14 11:39:04,066 - root - WARNING - 3
2010-12-14 11:39:04,066 - root - ERROR - 4
2010-12-14 11:39:04,066 - root - CRITICAL - 5
My file is output, but looks the same as above (although with the extra information included). However the debug and info levels are not output to either.
I am on Python 2.7
Here is my simple example showing failure:
import os
import sys
import logging
import logging.config
sys.path.append(os.path.realpath("shared/"))
sys.path.append(os.path.realpath("tests/"))
class Main(object):
#staticmethod
def main():
logging.config.fileConfig("logging.conf")
logging.debug("1")
logging.info("2")
logging.warn("3")
logging.error("4")
logging.critical("5")
if __name__ == "__main__":
Main.main()
It looks like you've set the levels for your handlers, but not your logger. The logger's level filters every message before it can reach its handlers and the default is WARNING and above (as you can see). Setting the root logger's level to NOTSET as you have, as well as setting it to DEBUG (or whatever is the lowest level you wish to log) should solve your issue.
Adding the following line to the root logger took care of my problem:
level=NOTSET
Just add log level in [logger_root]. It is worked.
[logger_root]
level=DEBUG
handlers=screen,file
A simple approach to both write to terminal and file would be as following:
import logging.config
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [%(levelname)s] %(message)s",
handlers=[
logging.FileHandler("log_file.log"),
logging.StreamHandler()
]
)
logger = logging.getLogger(__name__)
And then use it in your code like this:
logger.info('message')
logger.error('message')
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import logging
import logging.handlers
from logging.config import dictConfig
logger = logging.getLogger(__name__)
DEFAULT_LOGGING = {
'version': 1,
'disable_existing_loggers': False,
}
def configure_logging(logfile_path):
"""
Initialize logging defaults for Project.
:param logfile_path: logfile used to the logfile
:type logfile_path: string
This function does:
- Assign INFO and DEBUG level to logger file handler and console handler
"""
dictConfig(DEFAULT_LOGGING)
default_formatter = logging.Formatter(
"[%(asctime)s] [%(levelname)s] [%(name)s] [%(funcName)s():%(lineno)s] [PID:%(process)d TID:%(thread)d] %(message)s",
"%d/%m/%Y %H:%M:%S")
file_handler = logging.handlers.RotatingFileHandler(logfile_path, maxBytes=10485760,backupCount=300, encoding='utf-8')
file_handler.setLevel(logging.INFO)
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(default_formatter)
console_handler.setFormatter(default_formatter)
logging.root.setLevel(logging.DEBUG)
logging.root.addHandler(file_handler)
logging.root.addHandler(console_handler)
[31/10/2015 22:00:33] [DEBUG] [yourmodulename] [yourfunction_name():9] [PID:61314 TID:140735248744448] this is logger infomation from hello module
I think you should add the disable_existing_loggers to false.
Related
It can be practical to have two or more distinct log files. For example for a Rest service, have one log file for general failures and another for faults in the content.
I tried doing this using INI-file but for some reason all the logs would go to both files. So...
How would the logging.ini look like if I want all logs from logger1 to go to logger1.log and all logs from logger2 to go to logger2.log:
logging.config.fileConfig('logging.ini')
logger1 = logging.getLogger('name1')
logger2 = logging.getLogger('name2')
This works:
python_logging.py
import logging
from logging import config
logging.config.fileConfig('logging.ini')
logger1 = logging.getLogger('name1')
logger2 = logging.getLogger('name2')
logger1.debug('This is logger1')
logger2.info('This is logger2')
logger1.warning('This is logger1')
logger1.error('This is logger1')
logger2.warning('This is logger2')
logger2.error('This is logger2')
logging.ini:
[loggers]
keys=root,name1,name2
[handlers]
keys=console_handler,file_handler_name1,file_handler_name2
[formatters]
keys=console_formatter,file_formatter
[logger_root]
level=INFO
handlers=
[logger_name1]
level=INFO
handlers=console_handler,file_handler_name1
qualname=name1
[logger_name2]
level=INFO
handlers=console_handler,file_handler_name2
qualname=name2
[handler_console_handler]
class=StreamHandler
formatter=console_formatter
args=(sys.stdout,)
[handler_file_handler_name1]
class=handlers.RotatingFileHandler
formatter=file_formatter
args=('name1.log','a',1000000,100)
[handler_file_handler_name2]
class=handlers.RotatingFileHandler
formatter=file_formatter
args=('name2.log','a',1000000,100)
[formatter_console_formatter]
format=%(asctime)s %(levelname)s | %(name)s | %(message)s'
datefmt='%d-%m-%Y %H:%M:%S
[formatter_file_formatter]
format=%(asctime)s %(levelname)s | %(name)s | %(message)s'
datefmt='%d-%m-%Y %H:%M:%S
So I am trying to implement logging within my Python program. The goal is to set it up so that a log file is created and everything the program does through it's various modules is logged (based on logging level). This is what my current code looks like:
Text File for Log Configuration:
#logging.conf
[loggers]
keys=root,MainLogger
[handlers]
keys=consoleHandler
[formatters]
keys=consoleFormatter
[logger_root]
level=DEBUG
handlers=consoleHandler
[logger_MainLogger]
level=DEBUG
handlers=consoleHandler
qualname=MainLogger
propagate=0
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=consoleFormatter
args=(sys.stdout,)
[formatter_consoleFormatter]
format=%(asctime)s | %(levelname)-8s | %(filename)s-%(funcName)s-%lineno)04d | %(message)s
External Module to Test Logs:
#test.py
import logging
logger = logging.getLogger(__name__)
def testLog():
logger.debug("Debug Test")
logger.info("Info Test")
logger.warning("Warning Test")
logger.error("Error Test")
Main file:
#__init__.py
import logging
import logging.config
from datetime import datetime
logging.config.fileConfig('logging.conf', disable_existing_loggers = False)
logger = logging.getLogger('MainLogger')
fileHandler = logging.FileHandler('{:%Y-%m-%d}.log'.format(datetime.now()))
formatter = logging.Formatter('%(asctime)s | %(levelname)-8s | %(lineno)04d | %(message)s')
fileHandler.setFormatter(formatter)
logger.addHandler(fileHandler)
if __name__ == "__main__":
import test
logger.debug("Debug Test")
test.testLog()
Currently, all log messages are currently being displayed withing the IDLE3 shell when I run __init__.py and the log file is being created. However within the log file itself the only message being recording is the "Debug Test" from __init__.py. None of the messages from the test.py module are being recorded in the log file.
What is my problem?
In test.py it grabs a logger object before you configure it later in your __init__.py. Make sure you configure the logging module first before grabbing any logger instance.
Why in python logger.info("print something") does not output. I have seen questions asked before, but solution doesnt exist. I do not want to use logger.debug or logger.warning to see text.
Simply logger.info should print the text, otherwise whats the use of this?
logging.conf file as below
[loggers]
keys=root
[handlers]
keys=stream
[formatters]
keys=formatter
[logger_root]
level=INFO
handlers=stream
[handler_stream]
class=StreamHandler
level=INFO
formatter=formatter
args=(sys.stderr,)
[formatter_formatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
Demo code that access logger:
import logging
logger = logging.getLogger()
if __name__ == '__main__':
logger.info("logger")
print("print")
Output is only print, not the logger. So logger.info does not work.
By default, the root logger (the one you use when you say logger.info) is set at a level of WARN.
You can either do:
logging.basicConfig(level=logging.INFO)
or logging.getLogger().setLevel(logging.INFO)
Seems you do not load your configuration file. You should add this:
logging.config.fileConfig('path_to_logging.conf')
before logger = logging.getLogger()
because right now you are using the default WARNING level.
EDIT: in order to use logging.config, you have to import it too:
import logging.config
So the complete code should be:
import logging
import logging.config
logging.config.fileConfig('path_to_logging.conf')
logger = logging.getLogger()
if __name__ == '__main__':
logger.info("logger")
print("print")
The code above, with the following logging.conf (same as you except I removed the sentry parts):
[loggers]
keys=root
[handlers]
keys=stream
[formatters]
keys=formatter
[logger_root]
level=INFO
handlers=stream
[handler_stream]
class=StreamHandler
level=INFO
formatter=formatter
args=(sys.stderr,)
[formatter_formatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
does work:
$ ./test_script3.py
2016-05-23 15:37:40,437 - root - INFO - logger
print
I set up logging throughout my python package using a logconfig.ini file.
[loggers]
keys=extracts,root
[formatters]
keys=simple,detailed
[handlers]
keys=file_handler
[formatter_simple]
format=%(module)s - %(levelname)s - %(message)s
datefmt=%Y-%m-%d %H:%M:%S
[formatter_detailed]
format=%(asctime)s %(name)s:%(lineno)s %(levelname)s %(message)s
datefmt=%Y-%m-%d %H:%M:%S
[handler_file_handler]
class=logging.handlers.RotatingFileHandler
level=DEBUG
formatter=detailed
args=('/ebs/logs/foo.log', 'a', 100000000, 3)
[logger_extracts]
level=DEBUG
handlers=file_handler
propagate=1
qualname=extracts
[logger_root]
level=NOTSET
handlers=
But whenever I run my application, I get the following warning message in prompt,
No handlers found for logger __main__
How can I fix this?
You have to call logging.basicConfig() first:
Logging HOWTO
The call to basicConfig() should come before any calls to debug(),
info() etc. As it’s intended as a one-off simple configuration
facility, only the first call will actually do anything: subsequent
calls are effectively no-ops.
Or all logging.info('Starting logger for...') which will call logging.basicConfig() automatically. So something like:
import logging
logging.info('Starting logger for...') # or call logging.basicConfig()
LOG = logging.getLogger(name)
The module author's reason for this behavior is here
I found my error.
It turns out the the root logger is used for main.
I just need to attach a handler to the root logger as so,
[logger_root]
level=NOTSET
handlers=file_handler
I am trying to use a config file for configure Python Logging, but also adding handlers after the dict config has been loaded. SO my config file is like
version: 1
formatters:
default_formatter:
format: '%(asctime)s : %(levelname)s : %(message)s'
datefmt: '%d-%b-%Y %H:%M:%S'
plain_formatter:
format: '%(message)s'
handlers:
console_default_handler:
class: logging.StreamHandler
level: INFO
formatter: default_formatter
stream: ext://sys.stdout
root:
level: INFO
handlers: [console_default_handler]
Then in the code - I do
log_config_dict=yaml.load(open(log_config_file, 'r'))
logging.config.dictConfig(log_config_dict)
And I want to add loggers in this way -
fhandler1=logging.FileHandler(log_file_name,mode="w")
fhandler1.setFormatter(log_config_dict['formatters']['plain_formatter'])
fhandler1.setLevel(logging.DEBUG)
This is not working. Is there any way I catch fetch values defined in the dictConfig for using them in my manual log configuration please?
Thanks
Ohh I figure it out. What I need to do is
formatter =logging.Formatter(log_config_dict['formatters']['plain_formatter']['format'])
fhandler1.setFormatter(formatter)
So, I need to create a Formatter object.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import logging
import logging.handlers
from logging.config import dictConfig
logger = logging.getLogger(__name__)
DEFAULT_LOGGING = {
'version': 1,
'disable_existing_loggers': False,
}
def configure_logging(logfile_path):
"""
Initialize logging defaults for Project.
:param logfile_path: logfile used to the logfile
:type logfile_path: string
This function does:
- Assign INFO and DEBUG level to logger file handler and console handler
"""
dictConfig(DEFAULT_LOGGING)
default_formatter = logging.Formatter(
"[%(asctime)s] [%(levelname)s] [%(name)s] [%(funcName)s():%(lineno)s] [PID:%(process)d TID:%(thread)d] %(message)s",
"%d/%m/%Y %H:%M:%S")
file_handler = logging.handlers.RotatingFileHandler(logfile_path, maxBytes=10485760,backupCount=300, encoding='utf-8')
file_handler.setLevel(logging.INFO)
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(default_formatter)
console_handler.setFormatter(default_formatter)
logging.root.setLevel(logging.DEBUG)
logging.root.addHandler(file_handler)
logging.root.addHandler(console_handler)
[31/10/2015 22:00:33] [DEBUG] [yourmodulename] [yourfunction_name():9] [PID:61314 TID:140735248744448] this is logger infomation from hello module
I think config log with function is more convenient.