I have a logger function defined in my_logging.py:
def my_logger(name):
print("warn:", name)
logger = logging.getLogger(name)
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('logs/demo.log')
c_handler.setLevel(logging.INFO)
f_handler.setLevel(logging.INFO)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logger
logger.addHandler(c_handler)
logger.addHandler(f_handler)
return logger
Then I use it in test.py:
import my_logging
logger = my_logging.my_logger(__name__)
logger.info("This is a test!")
It doesn't log at all! The reason I want to put the logger into a function because I want it to be used in multiple modules, using the same logging configuration.
What's the issue here? I tested and seems it has something to do with the handler's setLevel() method. logging.INFO doesn't have an effect.
In doc for setLevel() you can see that root logger uses WARRING level and probably it blocks INFO levels in handlers.
You have to set INFO level for root logger
logger = logging.getLogger(name)
logger.setLevel(logging.INFO)
If you need also DEBUG messages then you have to set DEBUG level for root logger and for handlers.
logger.setLevel(logging.DEBUG)
c_handler.setLevel(logging.DEBUG)
f_handler.setLevel(logging.DEBUG)
Using different levels for handlers you can send debug message only on screen or only to file.
Related
I am trying to do a pretty simple logging setup. I just want all of my log output to go to the terminal and to my log file. I found the below example on Real Python that demonstrates the setup of stream and file log handlers:
# logging_example.py
import logging
# Create a custom logger
logger = logging.getLogger(__name__)
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('file.log')
c_handler.setLevel(logging.WARNING)
f_handler.setLevel(logging.ERROR)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logger
logger.addHandler(c_handler)
logger.addHandler(f_handler)
logger.warning('This is a warning')
logger.error('This is an error')
## Log Output
# 2019-08-31 22:16:02,478 - __main__ - WARNING - This is a warning
# 2019-08-31 22:16:02,478 - __main__ - ERROR - This is an error
And this logs to the console and to the file as you would expect. However, when I modify the program so that it will also log INFO, like so:
import logging
# Create a custom logger
logger = logging.getLogger(__name__)
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('file.log')
c_handler.setLevel(logging.DEBUG)
f_handler.setLevel(logging.DEBUG)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logger
logger.addHandler(c_handler)
logger.addHandler(f_handler)
logger.warning('logs')
logger.error('logs')
logger.info('should log but doesn\'t')
logger.debug('should log but doesn\'t')
## Log Output
# __main__ - WARNING - logs
# __main__ - ERROR - logs
What am I doing wrong? Any help would be greatly appreciated!
You will also need to invoke setLevel(level) on the logger object itself, as, by default, it will use ROOT's logging level (if it doesn't have any other ancestors), which is WARNING:
logger.setLevel(logging.DEBUG)
Full code with output:
import logging
logger = logging.getLogger(__name__)
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('file.log')
logger.setLevel(logging.DEBUG) # <<< Added Line
c_handler.setLevel(logging.DEBUG)
f_handler.setLevel(logging.INFO)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logger
logger.addHandler(c_handler)
logger.addHandler(f_handler)
logger.warning('logs')
logger.error('logs')
logger.info('should log but doesn\'t')
logger.debug('should log but doesn\'t')
Output:
__main__ - WARNING - logs
__main__ - ERROR - logs
__main__ - INFO - should log but doesn't
__main__ - DEBUG - should log but doesn't
there is two place you can set level: Logger , Handler. both of them will affect the output of your logging setting -- in different node of the logging flow. also you can add Fliter to these two
instance to filter log record with more sophisticate rule.
flowing char in the office docment make clear what happening when logging.
I ran the following using both the Python shell, and run it as a Python file from the command line. I don's see my log output at all.
import logging
formatter = logging.Formatter('%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s')
stream_handler = logging.StreamHandler()
stream_handler.setLevel(logging.INFO)
stream_handler.setFormatter(formatter)
logger = logging.getLogger()
logger.addHandler(stream_handler)
logger.info(("info logging"))
Your logging output was almost correct with the exception of setLevel. The logging level needs to be defined on the logger instance instead of the handler instance. Your code therefore only needs a very small tweak to make it work:
import logging
formatter = logging.Formatter(
'%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s'
)
stream_handler = logging.StreamHandler()
stream_handler.setFormatter(formatter)
logger = logging.getLogger()
logger.addHandler(stream_handler)
logger.setLevel(logging.INFO)
logger.info("info logging")
This piece of code produces the following output:
2019-08-21 15:04:55,118,118 INFO [testHandler.py:11] info logging
Note, I also removed the double brackets on the logger.info call as these are not necessary.
Use logging.basicConfig() to initialize the logging system.
You need to set the level of your logger,
logger.setLevel(logging.INFO)
Below statement can be removed from your code,
stream_handler.setLevel(logging.INFO)
A logger and a handler can have different levels. You have only set the level for the handler, not the logger itself.
import logging
formatter = logging.Formatter('%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s')
stream_handler = logging.StreamHandler()
stream_handler.setLevel(logging.INFO)
stream_handler.setFormatter(formatter)
logger = logging.getLogger()
logger.setLevel(logging.INFO) # Required
logger.addHandler(stream_handler)
logger.info(("info logging"))
Have this part of code:
import logging
logFormatter = logging.Formatter(u'#%(levelname)-8s [%(asctime)s] [LINE:%(lineno)d] %(filename)-8s %(message)s',)
logger = logging.getLogger()
logger.setLevel(logging.INFO)
consoleHandler = logging.StreamHandler()
consoleHandler.setFormatter(logFormatter)
logger.addHandler(consoleHandler)
How I can handle logging.error calling, for example to save log in database or do something else
Probably the simplest answer is to use multiple handlers, setup with different logging levels. From the logging docs, here is a perfect example of what you are trying to achieve:
import logging
logger = logging.getLogger('simple_example')
logger.setLevel(logging.DEBUG)
# create file handler which logs even debug messages
fh = logging.FileHandler('spam.log')
fh.setLevel(logging.DEBUG)
# create console handler with a higher log level
ch = logging.StreamHandler()
ch.setLevel(logging.ERROR)
# create formatter and add it to the handlers
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
ch.setFormatter(formatter)
fh.setFormatter(formatter)
# add the handlers to logger
logger.addHandler(ch)
logger.addHandler(fh)
# 'application' code
logger.debug('debug message')
logger.info('info message')
logger.warn('warn message')
logger.error('error message')
logger.critical('critical message')
The example provides both the DEBUG and ERROR level handler formatting for these log level types.
Perhaps another solution is to switch to using dictConfig for handling the python logging module configuration. That would handle the structure in a dictionary type way, and you may be able to provide multiple values to the level attribute.
I'm trying to figure out how to capture messages generated by Python/NumPy intractive shell when running my script. I would like to log all generated by console messages (errors, warnings) to same file as defined in my code log messages with time stamps:
def LogToFile():
global logger
logger = logging.getLogger('MyApp')
logger.setLevel(logging.DEBUG)
file_log_handler = RotatingFileHandler('logfile.log', maxBytes=1024, backupCount=5)
logger.addHandler(file_log_handler)
stderr_log_handler = logging.StreamHandler()
logger.addHandler(stderr_log_handler)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
file_log_handler.setFormatter(formatter)
stderr_log_handler.setFormatter(formatter)
return logger
Afaik, you'd have to specify this in basicConfig, not in your logger:
logging.basicConfig(filename=LOG_FILE,
level=logging.DEBUG)
before you do
logger = logging.getLogger('MyApp')
logger.setLevel(logging.DEBUG)
I am trying to get module level logging via code working for three outputs: file, console and application internal(QTextEdit).
I can get all three loggers working with the code below but the application internal logger is not logging all events and the console logger (only) prints each line twice.
I have tried using
logging.getLogger(__name__)
for the file logger instead of root (no logs generated), same for the console (works fine with only 1 line per log output) and same for the MyLogHandler (no logs generated) and tried various combinations of root logger and 'name' logger but can't get all logs working and console only printing one line per log event.
def configCodeRootExample_(self):
logFileName = self.getLogLocation()
rootLogger = logging.getLogger('')
#This logger works
fileLogger = logging.FileHandler(logFileName)
fileLogger.setLevel(logging.INFO)
fileFormatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
fileLogger.setFormatter(fileFormatter)
rootLogger.addHandler(fileLogger)
#This logger works but prints output twice
consoleFormatter = logging.Formatter('%(asctime)s - %(levelname)s - %(name)s - %(module)s - %(funcName)s - %(lineno)d - %(message)s')
console = logging.StreamHandler()
console.setLevel(logging.DEBUG)
console.setFormatter(consoleFormatter)
rootLogger.addHandler(console)
#This logger works but only logs a subset of DEBUG events and no INFO events
myLogHandler = GSLLogHandler()
myLogHandler.setLevel(logging.DEBUG)
myLogHandler.setFormatter(fileFormatter)
rootLogger.addHandler(myLogHandler)
also for the record here is the log handler to output to a listening QTextEdit:
import logging
from loggerpackage.logsignals import LogSignals
class MyLogHandler(logging.Handler):
def __init__(self):
logging.Handler.__init__(self)
self.logSignals = LogSignals()
def emit(self, logMsg):
logMsg = self.format(logMsg)
self.logSignals.logEventTriggered.emit(logMsg)
If I change the console logger to the module level:
logger = logging.getLogger(__name__)
consoleFormatter = logging.Formatter('%(asctime)s - %(levelname)s - %(name)s - %(module)s - %(funcName)s - %(lineno)d - %(message)s')
console = logging.StreamHandler()
console.setLevel(logging.DEBUG)
console.setFormatter(consoleFormatter)
logger.addHandler(console)
Then only one line is printed for each log event but the formatting is incorrect, it seems to be some sort of default formatter
See here for a solution to the duplicate console logging: How to I disable and re-enable console logging in Python?
logger = logging.getLogger()
lhStdout = logger.handlers[0]
... add log handlers
logger.removeHandler(lhStdout)
The issue I was having with the MyLogHandler was that the slot on the QTextEdit wasn't connected in time to receive the first few DEBUG and INFO events.