I am trying to get module level logging via code working for three outputs: file, console and application internal(QTextEdit).
I can get all three loggers working with the code below but the application internal logger is not logging all events and the console logger (only) prints each line twice.
I have tried using
logging.getLogger(__name__)
for the file logger instead of root (no logs generated), same for the console (works fine with only 1 line per log output) and same for the MyLogHandler (no logs generated) and tried various combinations of root logger and 'name' logger but can't get all logs working and console only printing one line per log event.
def configCodeRootExample_(self):
logFileName = self.getLogLocation()
rootLogger = logging.getLogger('')
#This logger works
fileLogger = logging.FileHandler(logFileName)
fileLogger.setLevel(logging.INFO)
fileFormatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
fileLogger.setFormatter(fileFormatter)
rootLogger.addHandler(fileLogger)
#This logger works but prints output twice
consoleFormatter = logging.Formatter('%(asctime)s - %(levelname)s - %(name)s - %(module)s - %(funcName)s - %(lineno)d - %(message)s')
console = logging.StreamHandler()
console.setLevel(logging.DEBUG)
console.setFormatter(consoleFormatter)
rootLogger.addHandler(console)
#This logger works but only logs a subset of DEBUG events and no INFO events
myLogHandler = GSLLogHandler()
myLogHandler.setLevel(logging.DEBUG)
myLogHandler.setFormatter(fileFormatter)
rootLogger.addHandler(myLogHandler)
also for the record here is the log handler to output to a listening QTextEdit:
import logging
from loggerpackage.logsignals import LogSignals
class MyLogHandler(logging.Handler):
def __init__(self):
logging.Handler.__init__(self)
self.logSignals = LogSignals()
def emit(self, logMsg):
logMsg = self.format(logMsg)
self.logSignals.logEventTriggered.emit(logMsg)
If I change the console logger to the module level:
logger = logging.getLogger(__name__)
consoleFormatter = logging.Formatter('%(asctime)s - %(levelname)s - %(name)s - %(module)s - %(funcName)s - %(lineno)d - %(message)s')
console = logging.StreamHandler()
console.setLevel(logging.DEBUG)
console.setFormatter(consoleFormatter)
logger.addHandler(console)
Then only one line is printed for each log event but the formatting is incorrect, it seems to be some sort of default formatter
See here for a solution to the duplicate console logging: How to I disable and re-enable console logging in Python?
logger = logging.getLogger()
lhStdout = logger.handlers[0]
... add log handlers
logger.removeHandler(lhStdout)
The issue I was having with the MyLogHandler was that the slot on the QTextEdit wasn't connected in time to receive the first few DEBUG and INFO events.
Related
Here is my code
logger = logging.getLogger("JarvisAI")
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler(logname)
c_handler.setLevel(logging.WARNING)
f_handler.setLevel(logging.INFO)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s', "%Y-%m-%d %H:%M:%S")
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s', "%Y-%m-%d %H:%M:%S")
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logging
logger.addHandler(c_handler)
logger.addHandler(f_handler)
Running logger.info("Test") does not produce anything in the logfile.
However logger.warning and other higher log level works fine both in console and file.
Pls help.
The logger itself also has a logging level, and that needs to be below (or above, depending on your point of view) that of the handlers to show the handlers' output:
logger.setLevel(logging.INFO)
(or even debug level: the formatters' levels will prevent debugging info from being output anyway) will do that.
This is also shown in the first code block of the Python logging cookbook. Have a read through it.
The reason you are getting warning and higher log level output, is because warning is the default logging level.
I am trying to do a pretty simple logging setup. I just want all of my log output to go to the terminal and to my log file. I found the below example on Real Python that demonstrates the setup of stream and file log handlers:
# logging_example.py
import logging
# Create a custom logger
logger = logging.getLogger(__name__)
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('file.log')
c_handler.setLevel(logging.WARNING)
f_handler.setLevel(logging.ERROR)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logger
logger.addHandler(c_handler)
logger.addHandler(f_handler)
logger.warning('This is a warning')
logger.error('This is an error')
## Log Output
# 2019-08-31 22:16:02,478 - __main__ - WARNING - This is a warning
# 2019-08-31 22:16:02,478 - __main__ - ERROR - This is an error
And this logs to the console and to the file as you would expect. However, when I modify the program so that it will also log INFO, like so:
import logging
# Create a custom logger
logger = logging.getLogger(__name__)
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('file.log')
c_handler.setLevel(logging.DEBUG)
f_handler.setLevel(logging.DEBUG)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logger
logger.addHandler(c_handler)
logger.addHandler(f_handler)
logger.warning('logs')
logger.error('logs')
logger.info('should log but doesn\'t')
logger.debug('should log but doesn\'t')
## Log Output
# __main__ - WARNING - logs
# __main__ - ERROR - logs
What am I doing wrong? Any help would be greatly appreciated!
You will also need to invoke setLevel(level) on the logger object itself, as, by default, it will use ROOT's logging level (if it doesn't have any other ancestors), which is WARNING:
logger.setLevel(logging.DEBUG)
Full code with output:
import logging
logger = logging.getLogger(__name__)
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('file.log')
logger.setLevel(logging.DEBUG) # <<< Added Line
c_handler.setLevel(logging.DEBUG)
f_handler.setLevel(logging.INFO)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logger
logger.addHandler(c_handler)
logger.addHandler(f_handler)
logger.warning('logs')
logger.error('logs')
logger.info('should log but doesn\'t')
logger.debug('should log but doesn\'t')
Output:
__main__ - WARNING - logs
__main__ - ERROR - logs
__main__ - INFO - should log but doesn't
__main__ - DEBUG - should log but doesn't
there is two place you can set level: Logger , Handler. both of them will affect the output of your logging setting -- in different node of the logging flow. also you can add Fliter to these two
instance to filter log record with more sophisticate rule.
flowing char in the office docment make clear what happening when logging.
I have a logger function defined in my_logging.py:
def my_logger(name):
print("warn:", name)
logger = logging.getLogger(name)
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('logs/demo.log')
c_handler.setLevel(logging.INFO)
f_handler.setLevel(logging.INFO)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logger
logger.addHandler(c_handler)
logger.addHandler(f_handler)
return logger
Then I use it in test.py:
import my_logging
logger = my_logging.my_logger(__name__)
logger.info("This is a test!")
It doesn't log at all! The reason I want to put the logger into a function because I want it to be used in multiple modules, using the same logging configuration.
What's the issue here? I tested and seems it has something to do with the handler's setLevel() method. logging.INFO doesn't have an effect.
In doc for setLevel() you can see that root logger uses WARRING level and probably it blocks INFO levels in handlers.
You have to set INFO level for root logger
logger = logging.getLogger(name)
logger.setLevel(logging.INFO)
If you need also DEBUG messages then you have to set DEBUG level for root logger and for handlers.
logger.setLevel(logging.DEBUG)
c_handler.setLevel(logging.DEBUG)
f_handler.setLevel(logging.DEBUG)
Using different levels for handlers you can send debug message only on screen or only to file.
I am using below configuration for logging in python,
logger = logging.getLogger(__name__)
hdlr = logging.FileHandler(r'/var/log/resource_checker.log')
formatter = logging.Formatter('%(asctime)s : %(name)s : %(levelname)s : %(message)s')
hdlr.setFormatter(formatter)
logger.addHandler(hdlr)
logger.setLevel(logging.INFO)
This works well for me. However if I add logging.basicConfig(level=logging.DEBUG) with existing logging configuration, all my logs pops up on screen screen (with additional DEBUG logs, which is good to have for me) and logfile does not populate at all.
how can i push those logs back to log file as my older config.?
This will log everything to a file, all from the logging.basicConfig:
import logging
logging.basicConfig(
level=logging.DEBUG,
filename='/var/log/resource_checker.log',
format='%(asctime)s : %(name)s : %(levelname)s : %(message)s'
)
logger = logging.getLogger(__name__)
I'm trying to figure out how to capture messages generated by Python/NumPy intractive shell when running my script. I would like to log all generated by console messages (errors, warnings) to same file as defined in my code log messages with time stamps:
def LogToFile():
global logger
logger = logging.getLogger('MyApp')
logger.setLevel(logging.DEBUG)
file_log_handler = RotatingFileHandler('logfile.log', maxBytes=1024, backupCount=5)
logger.addHandler(file_log_handler)
stderr_log_handler = logging.StreamHandler()
logger.addHandler(stderr_log_handler)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
file_log_handler.setFormatter(formatter)
stderr_log_handler.setFormatter(formatter)
return logger
Afaik, you'd have to specify this in basicConfig, not in your logger:
logging.basicConfig(filename=LOG_FILE,
level=logging.DEBUG)
before you do
logger = logging.getLogger('MyApp')
logger.setLevel(logging.DEBUG)