Here is my code
logger = logging.getLogger("JarvisAI")
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler(logname)
c_handler.setLevel(logging.WARNING)
f_handler.setLevel(logging.INFO)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s', "%Y-%m-%d %H:%M:%S")
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s', "%Y-%m-%d %H:%M:%S")
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logging
logger.addHandler(c_handler)
logger.addHandler(f_handler)
Running logger.info("Test") does not produce anything in the logfile.
However logger.warning and other higher log level works fine both in console and file.
Pls help.
The logger itself also has a logging level, and that needs to be below (or above, depending on your point of view) that of the handlers to show the handlers' output:
logger.setLevel(logging.INFO)
(or even debug level: the formatters' levels will prevent debugging info from being output anyway) will do that.
This is also shown in the first code block of the Python logging cookbook. Have a read through it.
The reason you are getting warning and higher log level output, is because warning is the default logging level.
Related
I am trying to do a pretty simple logging setup. I just want all of my log output to go to the terminal and to my log file. I found the below example on Real Python that demonstrates the setup of stream and file log handlers:
# logging_example.py
import logging
# Create a custom logger
logger = logging.getLogger(__name__)
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('file.log')
c_handler.setLevel(logging.WARNING)
f_handler.setLevel(logging.ERROR)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logger
logger.addHandler(c_handler)
logger.addHandler(f_handler)
logger.warning('This is a warning')
logger.error('This is an error')
## Log Output
# 2019-08-31 22:16:02,478 - __main__ - WARNING - This is a warning
# 2019-08-31 22:16:02,478 - __main__ - ERROR - This is an error
And this logs to the console and to the file as you would expect. However, when I modify the program so that it will also log INFO, like so:
import logging
# Create a custom logger
logger = logging.getLogger(__name__)
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('file.log')
c_handler.setLevel(logging.DEBUG)
f_handler.setLevel(logging.DEBUG)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logger
logger.addHandler(c_handler)
logger.addHandler(f_handler)
logger.warning('logs')
logger.error('logs')
logger.info('should log but doesn\'t')
logger.debug('should log but doesn\'t')
## Log Output
# __main__ - WARNING - logs
# __main__ - ERROR - logs
What am I doing wrong? Any help would be greatly appreciated!
You will also need to invoke setLevel(level) on the logger object itself, as, by default, it will use ROOT's logging level (if it doesn't have any other ancestors), which is WARNING:
logger.setLevel(logging.DEBUG)
Full code with output:
import logging
logger = logging.getLogger(__name__)
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('file.log')
logger.setLevel(logging.DEBUG) # <<< Added Line
c_handler.setLevel(logging.DEBUG)
f_handler.setLevel(logging.INFO)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logger
logger.addHandler(c_handler)
logger.addHandler(f_handler)
logger.warning('logs')
logger.error('logs')
logger.info('should log but doesn\'t')
logger.debug('should log but doesn\'t')
Output:
__main__ - WARNING - logs
__main__ - ERROR - logs
__main__ - INFO - should log but doesn't
__main__ - DEBUG - should log but doesn't
there is two place you can set level: Logger , Handler. both of them will affect the output of your logging setting -- in different node of the logging flow. also you can add Fliter to these two
instance to filter log record with more sophisticate rule.
flowing char in the office docment make clear what happening when logging.
I have a logger function defined in my_logging.py:
def my_logger(name):
print("warn:", name)
logger = logging.getLogger(name)
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('logs/demo.log')
c_handler.setLevel(logging.INFO)
f_handler.setLevel(logging.INFO)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logger
logger.addHandler(c_handler)
logger.addHandler(f_handler)
return logger
Then I use it in test.py:
import my_logging
logger = my_logging.my_logger(__name__)
logger.info("This is a test!")
It doesn't log at all! The reason I want to put the logger into a function because I want it to be used in multiple modules, using the same logging configuration.
What's the issue here? I tested and seems it has something to do with the handler's setLevel() method. logging.INFO doesn't have an effect.
In doc for setLevel() you can see that root logger uses WARRING level and probably it blocks INFO levels in handlers.
You have to set INFO level for root logger
logger = logging.getLogger(name)
logger.setLevel(logging.INFO)
If you need also DEBUG messages then you have to set DEBUG level for root logger and for handlers.
logger.setLevel(logging.DEBUG)
c_handler.setLevel(logging.DEBUG)
f_handler.setLevel(logging.DEBUG)
Using different levels for handlers you can send debug message only on screen or only to file.
I'm using 'schedule' for a current project:
https://pypi.python.org/pypi/schedule
It's great, but I want to suppress the "Running job Every x seconds" log message that gets triggered every time a scheduled task is run. Example of what I mean below:
Is there any way to achieve this? Below is my current logging.basicConfig, I'm quite new to configuring logging beyond the absolute basics, so the solution may lie more with that:
# Define overall logging settings; these log levels/format go to file
logging.basicConfig(level=variables.settings['log_level_file'],
format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
filename='logs\log.log')
# Set up Handlers and Formatters; these log levels/format go to console
console = logging.StreamHandler()
console.setLevel(variables.settings['log_level_console'])
formatter = logging.Formatter('%(asctime)s %(name)-12s %(levelname)-8s %(message)s')
console.setFormatter(formatter)
logging.getLogger('').addHandler(console)
As Meloman pointed out, you can directly set the individual 'schedule' logger to a higher level than the INFO default:
logging.getLogger('schedule').setLevel(logging.CRITICAL)
I am trying to get module level logging via code working for three outputs: file, console and application internal(QTextEdit).
I can get all three loggers working with the code below but the application internal logger is not logging all events and the console logger (only) prints each line twice.
I have tried using
logging.getLogger(__name__)
for the file logger instead of root (no logs generated), same for the console (works fine with only 1 line per log output) and same for the MyLogHandler (no logs generated) and tried various combinations of root logger and 'name' logger but can't get all logs working and console only printing one line per log event.
def configCodeRootExample_(self):
logFileName = self.getLogLocation()
rootLogger = logging.getLogger('')
#This logger works
fileLogger = logging.FileHandler(logFileName)
fileLogger.setLevel(logging.INFO)
fileFormatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
fileLogger.setFormatter(fileFormatter)
rootLogger.addHandler(fileLogger)
#This logger works but prints output twice
consoleFormatter = logging.Formatter('%(asctime)s - %(levelname)s - %(name)s - %(module)s - %(funcName)s - %(lineno)d - %(message)s')
console = logging.StreamHandler()
console.setLevel(logging.DEBUG)
console.setFormatter(consoleFormatter)
rootLogger.addHandler(console)
#This logger works but only logs a subset of DEBUG events and no INFO events
myLogHandler = GSLLogHandler()
myLogHandler.setLevel(logging.DEBUG)
myLogHandler.setFormatter(fileFormatter)
rootLogger.addHandler(myLogHandler)
also for the record here is the log handler to output to a listening QTextEdit:
import logging
from loggerpackage.logsignals import LogSignals
class MyLogHandler(logging.Handler):
def __init__(self):
logging.Handler.__init__(self)
self.logSignals = LogSignals()
def emit(self, logMsg):
logMsg = self.format(logMsg)
self.logSignals.logEventTriggered.emit(logMsg)
If I change the console logger to the module level:
logger = logging.getLogger(__name__)
consoleFormatter = logging.Formatter('%(asctime)s - %(levelname)s - %(name)s - %(module)s - %(funcName)s - %(lineno)d - %(message)s')
console = logging.StreamHandler()
console.setLevel(logging.DEBUG)
console.setFormatter(consoleFormatter)
logger.addHandler(console)
Then only one line is printed for each log event but the formatting is incorrect, it seems to be some sort of default formatter
See here for a solution to the duplicate console logging: How to I disable and re-enable console logging in Python?
logger = logging.getLogger()
lhStdout = logger.handlers[0]
... add log handlers
logger.removeHandler(lhStdout)
The issue I was having with the MyLogHandler was that the slot on the QTextEdit wasn't connected in time to receive the first few DEBUG and INFO events.
I have setup logging as follows:
def setUp():
LOG_FORMAT = '%(asctime)s %(levelname)-8s %(name)s %(message)s'
#LOG_FORMAT = '%(asctime)s %(name)s %(message)s'
logging.basicConfig(level=logging.DEBUG, format=LOG_FORMAT)
formatter = logging.Formatter(LOG_FORMAT)
ch = logging.StreamHandler()
ch.setLevel(logging.ERROR)
ch.setFormatter(formatter)
logging.getLogger().addHandler(ch)
LOG_FILENAME = 'file.log'
fh = logging.FileHandler(LOG_FILENAME, 'w')
fh.setLevel(logging.DEBUG)
fh.setFormatter(formatter)
logging.getLogger().addHandler(fh)
However, the console still shows DEBUG messages. Am I missing something here?
Note that setting the level to ERROR on fh works fine.
I think you need to remove the call to logging.basicConfig. That function adds another logging.StreamHandler that probably is the one that is printing the messages you don't want to be printed.
To check this you can take a look at the handlers attribute for the root logger (it's a list with all the handlers being used) and verify how many logging.StreamHandlers there are. Also, probably the message with level set to logging.ERROR are printed twice because of the two logging.StreamHandlers.
My final advice is avoid using logging.basicConfig if you're going to explicitly configure the handlers in the code.
Edit: Just for completeness, the source code of logging.BasicConfig is as follows:
if len(root.handlers) == 0:
filename = kwargs.get("filename")
if filename:
mode = kwargs.get("filemode", 'a')
hdlr = FileHandler(filename, mode)
else:
stream = kwargs.get("stream")
hdlr = StreamHandler(stream)
fs = kwargs.get("format", BASIC_FORMAT)
dfs = kwargs.get("datefmt", None)
fmt = Formatter(fs, dfs)
hdlr.setFormatter(fmt)
root.addHandler(hdlr)
level = kwargs.get("level")
if level is not None:
root.setLevel(level)
where you can see that unless filename is passed, a logging.StreamHandler is created.
From Python docs on logging.basicConfig:
Does basic configuration for the logging system by creating a
StreamHandler with a default Formatter and adding it to the root
logger.
As you set debug level of root logger to logging.DEBUG and you didn't switched off forwarding messages up to the root logger your DEBUG messages get logged by this StreamHandler created by basicConfig