I ran the following using both the Python shell, and run it as a Python file from the command line. I don's see my log output at all.
import logging
formatter = logging.Formatter('%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s')
stream_handler = logging.StreamHandler()
stream_handler.setLevel(logging.INFO)
stream_handler.setFormatter(formatter)
logger = logging.getLogger()
logger.addHandler(stream_handler)
logger.info(("info logging"))
Your logging output was almost correct with the exception of setLevel. The logging level needs to be defined on the logger instance instead of the handler instance. Your code therefore only needs a very small tweak to make it work:
import logging
formatter = logging.Formatter(
'%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s'
)
stream_handler = logging.StreamHandler()
stream_handler.setFormatter(formatter)
logger = logging.getLogger()
logger.addHandler(stream_handler)
logger.setLevel(logging.INFO)
logger.info("info logging")
This piece of code produces the following output:
2019-08-21 15:04:55,118,118 INFO [testHandler.py:11] info logging
Note, I also removed the double brackets on the logger.info call as these are not necessary.
Use logging.basicConfig() to initialize the logging system.
You need to set the level of your logger,
logger.setLevel(logging.INFO)
Below statement can be removed from your code,
stream_handler.setLevel(logging.INFO)
A logger and a handler can have different levels. You have only set the level for the handler, not the logger itself.
import logging
formatter = logging.Formatter('%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s')
stream_handler = logging.StreamHandler()
stream_handler.setLevel(logging.INFO)
stream_handler.setFormatter(formatter)
logger = logging.getLogger()
logger.setLevel(logging.INFO) # Required
logger.addHandler(stream_handler)
logger.info(("info logging"))
Related
I have a bootstrap script for a Raspberry Pi that runs in python. I am looking to create a logger that logs to a file as well as to the console.
I was going to do something like this:
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [%(threadName)-12.12s] [%(levelname)-5.5s] %(message)s",
handlers=[
logging.FileHandler("{0}/{1}.log".format(logPath, fileName)),
logging.StreamHandler()
])
But what I would really like is to log INFO to the StreamHandler and DEBUG to the FileHandler... I cannot seem to figure that out.
Can anyone help me out?
Using Python 3.7.5
You could build the logger yourself (either through a config file or in pure python)
The tricky thing that I have wasted several hours on is forgetting to set the log level on the logger as well as on each of the handlers. Ensure that the logger is as permissive as the most permissive handler.
example script
# emits the info line to the console and
# both the info & debug lines to the log file
# test_pylog.py
import logging
log_format = logging.Formatter(
'%(asctime)s %(threadName)s %(levelname)s %(message)s'
)
logger = logging.getLogger(__name__)
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)
console_handler.setFormatter(log_format)
logger.addHandler(console_handler)
file_handler = logging.FileHandler('logfile.txt')
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(log_format)
logger.addHandler(file_handler)
logger.setLevel(logging.DEBUG)
if __name__ == '__main__':
logger.debug('Panic! at the disco')
logger.info('Weezer')
I have a logger function defined in my_logging.py:
def my_logger(name):
print("warn:", name)
logger = logging.getLogger(name)
# Create handlers
c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('logs/demo.log')
c_handler.setLevel(logging.INFO)
f_handler.setLevel(logging.INFO)
# Create formatters and add it to handlers
c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)
# Add handlers to the logger
logger.addHandler(c_handler)
logger.addHandler(f_handler)
return logger
Then I use it in test.py:
import my_logging
logger = my_logging.my_logger(__name__)
logger.info("This is a test!")
It doesn't log at all! The reason I want to put the logger into a function because I want it to be used in multiple modules, using the same logging configuration.
What's the issue here? I tested and seems it has something to do with the handler's setLevel() method. logging.INFO doesn't have an effect.
In doc for setLevel() you can see that root logger uses WARRING level and probably it blocks INFO levels in handlers.
You have to set INFO level for root logger
logger = logging.getLogger(name)
logger.setLevel(logging.INFO)
If you need also DEBUG messages then you have to set DEBUG level for root logger and for handlers.
logger.setLevel(logging.DEBUG)
c_handler.setLevel(logging.DEBUG)
f_handler.setLevel(logging.DEBUG)
Using different levels for handlers you can send debug message only on screen or only to file.
I'm trying to figure out how to capture messages generated by Python/NumPy intractive shell when running my script. I would like to log all generated by console messages (errors, warnings) to same file as defined in my code log messages with time stamps:
def LogToFile():
global logger
logger = logging.getLogger('MyApp')
logger.setLevel(logging.DEBUG)
file_log_handler = RotatingFileHandler('logfile.log', maxBytes=1024, backupCount=5)
logger.addHandler(file_log_handler)
stderr_log_handler = logging.StreamHandler()
logger.addHandler(stderr_log_handler)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
file_log_handler.setFormatter(formatter)
stderr_log_handler.setFormatter(formatter)
return logger
Afaik, you'd have to specify this in basicConfig, not in your logger:
logging.basicConfig(filename=LOG_FILE,
level=logging.DEBUG)
before you do
logger = logging.getLogger('MyApp')
logger.setLevel(logging.DEBUG)
I'm setting a logging feature in my test script, but when I run it all messages are appearing on the console(stdout), the log file is written ok, how can I avoid this behaviour? I only want the messages are in the log file. Thanks.
import logging
import logging.handlers
LOG_FILE = "/var/log/mylog.log"
logger = logging.getLogger('mylog')
logger.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
file_handler = logging.handlers.RotatingFileHandler(LOG_FILE, maxBytes=10*1024*1024, backupCount=5)
file_handler.setFormatter(formatter)
file_handler.setLevel(logging.DEBUG)
logger.addHandler(file_handler)
console_handler = logging.StreamHandler()
console_handler.setFormatter(formatter)
console_handler.setLevel(logging.DEBUG)
logger.addHandler(console_handler)
....
As #alecxe said, you should change your console_handler code to:
console_handler = logging.StreamHandler()
console_handler.setFormatter(formatter)
console_handler.setLevel(logging.ERROR)
logger.addHandler(console_handler)
You're setting it yourself here:
console_handler = logging.StreamHandler()
console_handler.setFormatter(formatter)
console_handler.setLevel(logging.DEBUG)
logger.addHandler(console_handler)
Check out the docs: https://docs.python.org/2.7/library/logging.handlers.html
I've read a few posts on this but I'm still confused. I have this logging setup:
import logging
class MongoHandler(logging.Handler):
def __init__(self):
logging.Handler.__init__(self)
from pymongo import Connection
self.db = Connection('db_server').db_name
def emit(self, record):
try:
self.db.Logging.save(record.__dict__)
except:
print 'Logging Error: Unable to save log entry to db'
mh = MongoHandler()
sh = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)s - %(threadName)s - %(levelname)s - %(message)s')
sh.setFormatter(formatter)
log = logging.getLogger('DeviceMonitor_%s' % hostname)
log.addHandler(mh)
log.addHandler(sh)
log.setLevel(logging.INFO)
I want to be able to set a different level for the StreamHandler and the MongoHandler. Is that possible or do I need to have a second Logger obj?
You can set a different logging level for each logging handler but it seems you will have to set the logger's level to the "lowest". In the example below I set the logger to DEBUG, the stream handler to INFO and the TimedRotatingFileHandler to DEBUG. So the file has DEBUG entries and the stream outputs only INFO. You can't direct only DEBUG to one and only INFO to another handler. For that you'll need another logger.
logger = logging.getLogger("mylog")
formatter = logging.Formatter(
'%(asctime)s | %(name)s | %(levelname)s: %(message)s')
logger.setLevel(logging.DEBUG)
stream_handler = logging.StreamHandler()
stream_handler.setLevel(logging.INFO)
stream_handler.setFormatter(formatter)
logFilePath = "my.log"
file_handler = logging.handlers.TimedRotatingFileHandler(
filename=logFilePath, when='midnight', backupCount=30)
file_handler.setFormatter(formatter)
file_handler.setLevel(logging.DEBUG)
logger.addHandler(file_handler)
logger.addHandler(stream_handler)
logger.info("Started");
try:
x = 14
y = 0
z = x / y
except Exception as ex:
logger.error("Operation failed.")
logger.debug(
"Encountered {0} when trying to perform calculation.".format(ex))
logger.info("Ended");
I needed a time to understand the point
Set the general logger below your subloggers (handlers) (your result of logging.getLogger())
Set your subloggers levels on an equal or superior level to your general logger
In addition to GrantVS's answer:
I had to use
logging.basicConfig(level=logging.DEBUG)
in order for it to work. Otherwise great answer, thanks!
Had the same problem but the solution didn't work for iPython as the QtConsole automatically creates a handler with no level set:
import logging
root = logging.getLogger()
root.handlers
Out: [<StreamHandler <stderr> (NOTSET)>]
As a result iPython printed both DEBUG and INFO to console in spite of having different levels for my file handler and stream handler.
This thread pointed out this issue for me: Logging module does not print in IPython
I made a helper module (helped greatly by this stack thread!) called custom_logging.py to make logging more convenient in other modules:
import logging
from pathlib import Path
import sys
def _add_stream_handler(logger: logging.Logger):
stream_handler = logging.StreamHandler()
formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
stream_handler.setFormatter(formatter)
stream_handler.setLevel(logging.INFO)
logger.addHandler(stream_handler)
return logger
def _add_file_handler(logger: logging.Logger, log_path: Path):
file_handler = logging.FileHandler(log_path, mode='w')
formatter = logging.Formatter(
fmt='%(asctime)s %(name)-12s %(levelname)-8s %(message)s', datefmt='%m-%d %H:%M')
file_handler.setFormatter(formatter)
file_handler.setLevel(logging.DEBUG)
logger.addHandler(file_handler)
return logger
def create_logger(root_dir: Path, caller: str) -> logging.Logger:
log_path = root_dir / 'logs' / f'{caller}.log'
logger = logging.getLogger(caller)
root = logging.getLogger()
logger.setLevel(logging.DEBUG)
# If haven't already launched handlers...
if not len(logger.handlers):
_add_file_handler(logger=logger, log_path=log_path)
_add_stream_handler(logger=logger)
logger.info('Logging started.')
# Delete the Qtconsole stderr handler
# ... as it automatically logs both DEBUG & INFO to stderr
if root.handlers:
root.handlers = []
return logger
def log_dataframe(df, logger: logging.Logger, name: str = "DataFrame") -> None:
logger.debug(
f'''{name} head:\n {df.head()}\n----------\n''')
def log_dataframes(*args, logger: logging.Logger) -> None:
for gdf in args:
logger.debug(
f'''DataFrame head:\n {gdf.head()}\n----------\n''')
Can use its functions via:
from custom_logging import create_logger, log_dataframe
Or import custom_logging and custom_logging.create_logger() etc.
Also see sections 'Multiple handlers and formatters' and 'Logging to multiple destinations' in the official logging cookbook at:
https://docs.python.org/3/howto/logging-cookbook.html#logging-cookbook