with the following code I use to write info logs to info.log and error log to error.log
import logging
logger_info = logging.getLogger('info')
logger_err = logging.getLogger('err')
logger_info.setLevel(logging.INFO)
logger_err.setLevel(logging.WARNING)
info_file_handler = logging.FileHandler('info.log')
error_file_handler = logging.FileHandler('error.log')
logger_info.addHandler(info_file_handler)
logger_err.addHandler(error_file_handler)
logger_info.info('info test')
logger_err.error('error test')
Now I using 2 logger : logger_err and logger_info.
Can I marge those 2 logger to 1 logger , that logger_info.info will write into info.log and logger_info.error will write to error.log
It is uncommon because logging usually processes messages that have a higher severity than a threshold, but it is possible by using 2 handlers and a custom filter:
you attach a handler to the logger with a level of ERROR and make it write to the error.log file
you attach a second handler to the same logger with a level of INFO and make it write to the info.log file
you add a custom filter to that second handler to reject messages with a level higher than INFO
Demo:
class RevFilter:
"""A filter to reject messages ABOVE a maximum level"""
def __init__(self, maxLev):
self.maxLev = maxLev
def filter(self, record):
return record.levelno <= self.maxLev
hinf = logging.FileHandler('/path/to/info.log')
herr = logging.FileHandler('/path/to/error.log')
herr.setLevel(logging.ERROR)
hinf.setLevel(logging.INFO)
hinf.addFilter(RevFilter(logging.INFO))
logger = logging.getLogger(name)
logger.addHandler(hinf)
logger.addHandler(herr)
logger.setLevel(logging.INFO) # or lower of course...
From that point, the file error.log will receive messages send by logger at a level of ERROR or above, and info.log will only receive message at a level of INFO, neither higher nor lower.
I'm not sure I understand what you're looking to achieve, but if you're simply wanting to log both types of messages to the same file, you can just specify the same output file when you create the two FileHandlers:
import logging
logger_info = logging.getLogger('info')
logger_err = logging.getLogger('err')
logger_info.setLevel(logging.INFO)
logger_err.setLevel(logging.WARNING)
info_file_handler = logging.FileHandler('combined.log')
error_file_handler = logging.FileHandler('combined.log')
logger_info.addHandler(info_file_handler)
logger_err.addHandler(error_file_handler)
logger_info.info('info test')
logger_err.error('error test')
Related
I set up logging module wide like so:
def setup_logging(app):
"""
Set up logging so as to include RequestId and relevant logging info
"""
RequestID(app)
handler = logging.StreamHandler()
handler.setStream(sys.stdout)
handler.propagate=False
handler.setFormatter(
logging.Formatter("[MHPM][%(module)s][%(funcName)s] %(levelname)s : %(request_id)s - %(message)s")
)
handler.addFilter(RequestIDLogFilter()) # << Add request id contextual filter
logging.getLogger().addHandler(handler)
logging.getLogger().setLevel(level="DEBUG")
and I use it so:
# in init.py
setup_logging(app)
# in MHPMService.py
logger = logging.getLogger(__name__)
But here's what I see on my console:
DEBUG:src.service.MHPMService:MHPMService.__init__(): initialized
[MHPM][MHPMService][__init__] DEBUG : 5106ec8e-9ffa-423d-9401-c34a92dcfa23 - MHPMService.__init__(): initialized
I only want the second type of logs in my application, how do I do this?
I reset the handlers and got the expected behaviour:
logger.handlers=[]
swap the current handlers
logging.getLogger().handlers[0] = handler
instead of doing this
logging.getLogger().addHandler(handler)
I know how to adjust the level emitted from logs in individual modules
logging.getLogger(<module>).setLevel(logging.INFO)
How to implement different levels for specific modules in Python
My question is - can it be customized per-handler? (I have 3 handlers - File, Network, and commandline) (My problem is that one module, sqlalchemy, is emitting far to many messages at the info level. My CL logger is INFO, network and file are DEBUG. I want the messages in the network and file loggers, just not on the command line.)
Or, alternatively, is there a way to "demote" messages from a log - i.e. turn all INFO into DEBUG, for example?
So, Ideally, I would solve this with a filter on the source logger - and change the level of the messages originating - but I couldn't get the filter to register (I added it, but it never got called.)
Try 1:
class SA_Filter(logging.Filter):
def filter(self, record):
if record.levelno == logging.INFO:
record.level = logging.DEBUG
record.levelname = 'DEBUG'
return True
saLogger = logging.getLogger('sqlalchemy.engine')
saLogger.addFilter(SA_Filter())
Try 2 -
so, based on this info from the Logging documentation .
"Note that filters attached to handlers are consulted before an event is emitted by the handler" - I assumed that that meant prior to determining if the record should be emitted from the handler based on level, it ran the filters -
So I had this, which modified the record:
class SA_Filter(logging.Filter):
def filter(self, record):
if record.name.startswith('sqlalchemy.engine') and record.levelno == logging.INFO:
record.level = logging.DEBUG
record.levelname = 'DEBUG'
return True
# change info messages to Debug for sqlalchemy
handlers = logging.getLogger().handlers
for handler in handlers:
if isinstance(handler, logging.StreamHandler):
handler.addFilter(SA_Filter())
However, it turns out that the sequence is:
Check record level against handler level
Run Filters
Emit record.
So I ended with
Try 3
Which runs on the handler side, and gets the handler level - if the handler is Info or above, the messages are filtered, out, ow they are emitted.
# create log filter to filter out sql alchemy logs at INFO level - for the commandline handler
class SA_Filter(logging.Filter):
"""a class to filter out sqlalchemy logs at INFO level - for the commandline handler"""
def __init__(self, handler_level: int, *args, **kwargs) -> None:
self.handler_level = handler_level if handler_level else 0
super().__init__(*args, **kwargs)
def filter(self, record):
"""
filter out sqlalchemy logs at INFO level
"""
# if we're dealing with sqlalchemy logs, filter them if they're at the INFO level and so is the handler.
if record.name.startswith('sqlalchemy.engine') and record.levelno == logging.INFO and self.handler_level >= logging.INFO:
return False
else:
return True
...
# change info messages to Debug for sqlalchemy
handlers = logging.getLogger().handlers
for handler in handlers:
if isinstance(handler, logging.StreamHandler):
handler.addFilter(SA_Filter(handler.level))
I want to create a custom python logging Handler to send messages via slack.
I found this package however its no longer being updated so i created a very bare bone version of it. however it doesnt seem to work, I added a print call for debugging purposes and emit is not being evoked.
import logging
# import slacker
class SlackerLogHandler(logging.Handler):
def __init__(self, api_key, channel):
super().__init__()
self.channel = channel
# self.slacker = slacker.Slacker(api_key)
def emit(self, record):
message = self.format(record)
print('works')
# self.slacker.chat.post_message(text=message, channel=self.channel)
slack_handler = SlackerLogHandler('token', 'channel')
slack_handler.setLevel(logging.INFO)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
slack_handler.setFormatter(formatter)
logger = logging.getLogger('my_app')
logger.addHandler(slack_handler)
logger.info('info_try') # 'works' is not printed and no slack message is sent
I saw this answer and tried to also inherit from StreamHandler but to no avail.
I think I am missing something very basic.
edited to remove slack logic for ease of reproduction.
After some additional searching I found out that the logging level that is set for the handler is separate from the level that is set for the logger.
meaning, that adding:
logger.setLevel(logging.INFO)
fixes the problem.
I've created a logger with the custom handler which emits message to telegram via bot. It works, but for some reason the message is also emitted to stderr (or stdout).
My code:
class TelegramHandler(logging.Handler):
def emit(self, record):
log_entry = self.format(record)
payload = {
'chat_id': TELEGRAM_CHAT_ID,
'text': log_entry,
'parse_mode': 'HTML'
}
return requests.post("https://api.telegram.org/bot{token}/sendMessage".format(token=TELEGRAM_TOKEN),
data=payload).content
# setting up root logger
logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', level=logging.WARNING)
# setting up my logger
logger_bot = logging.getLogger('bot')
handler = TelegramHandler()
logger_bot.addHandler(handler)
logger_bot.setLevel(logging.DEBUG)
And the following code:
logger_bot.info('bot test')
logging.warning('root test')
results in
2019-12-06 22:24:14,401 - bot - INFO - bot test # *(plus message in telegram)*
2019-12-06 22:24:14,740 - root - WARNING - root test
I've checked handlers
for h in logger_bot.handlers:
print(h)
and only one is present
<TelegramHandler (NOTSET)>
Also noticed when I don't set up root logger, bot logger doesn't emit to std. So those are somehow connected, but I can't figure out what is going on exactly.
Thank you.
You need to set the propagate attribute on the bot logger to False.
So add logger_bot.propagate = False somewhere in the setup for it, and that should cause each log to only be handled by its own handlers.
fmtter = logging.Formatter('%(asctime)s,%(msecs)05.1f (%(funcName)s) %(message)s', '%H:%M:%S')
rock_log = '%s/rock.log' % Build.path
hdlr = logging.FileHandler(rock_log, mode='w')
hdlr.setFormatter(fmtter)
hdlr.setLevel(logging.DEBUG)
rock_logger = logging.getLogger('rock')
rock_logger.addHandler(hdlr)
I have the above logger
rock_logger.info("hi") doesnt print anything to the log BUT
rock_logger.error("hi") DOES print to the log
I think it has something to do with level but i specifically set it to logging.DEBUG
Anyone know what I am doing wrong?
In your case rock_logger is the logger and hdlr is the handler. When you try to log something, the logger first checks if it should handle the message (depending on it's own log level). Then it passes the message to the handler. The handler then checks the level against it's own log level and decided whether to write it to the file or not.
Your rock_logger might have the logging level set to errors. So it's not passing the messages to the handler when you try info() or debug().
rock_logger.setLevel(logging.DEBUG)
That should fix the issue.