I created a logging.conf file that works pretty well for one process. It logs everything correctly on the right file:
[loggers]
keys=root
[logger_root]
handlers=screen,file
level=DEBUG
[formatters]
keys=simple,complex
[formatter_simple]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
[formatter_complex]
format=%(asctime)s - %(name)s - %(levelname)s - %(module)s : %(lineno)d - %(message)s
[handlers]
keys=file,screen
[handler_file]
class=handlers.TimedRotatingFileHandler
interval=midnight
backupCount=7
formatter=complex
level=DEBUG
args=('../logs/my_log.log',)
[handler_screen]
class=StreamHandler
formatter=simple
level=DEBUG
args=(sys.stdout,)
And thats how I get my logger object inside my script:
# Create logger
logging.config.fileConfig('../logging.conf')
logger = logging.getLogger()
The problem is - Now I have more than one process using the same script, and both are using the same log file, what is causing a concurrency error.
How can I set my log file name from inside of my script? There is a way to set the args=('../logs/my_log.log',) attribute of the [handler_file] dynamically?
You can create the file handler at runtime in the script
filehandler = logging.FileHandler("process1.log")
filehandler.setFormatter( ... )
logger.addHandler(filehandler)
See https://docs.python.org/2/howto/logging.html#logging-advanced-tutorial for detailed documentation
Related
How do I get imported modules to show up in my filehandler without specifying each one?
When I run the "main.py", the 'sLogger' information is specified in the "log.txt" file, but not the 'module_a' information. The 'module_a' info only shows up in the console window.
I know I can fix this by adding a separate logger for 'module_a', but don't want to do this for every single module I want to get logging information from.
I thought setting 'disable_existing_loggers=False' to False would resolve this, but it isn't.
Am I doing something incorrectly there?
I also don't want to push everything with the root logger.
If I add fileHandler to the root I get everything, but I don't want to get info from the root logger.
main.py
import logging
from logging import config
logging.config.fileConfig("logger_config.txt", defaults={'logfilename': "test.log"}, disable_existing_loggers=False)
logger = logging.getLogger("sLogger")
logger.info("test")
logging.info("test")
import module_a
module_A.py:
logger = loging.getLogger(__name__)
logger.info("Test Log Message")
file_config.txt
[loggers]
keys=root, sLogger
[handlers]
keys=consoleHandler,fileHandler
[formatters]
keys=fileFormatter,consoleFormatter
[logger_root]
level=DEBUG
handlers=consoleHandler
[logger_sLogger]
level=DEBUG
handlers=consoleHandler,fileHandler
qualname=sLogger
propagate=0
[logger_parser]
level=DEBUG
handlers=consoleHandler,fileHandler
qualname=sLogger
propagate=1
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=consoleFormatter
args=(sys.stdout,)
[handler_fileHandler]
class=FileHandler
level=INFO
formatter=fileFormatter
args=('%(logfilename)s',)
[formatter_fileFormatter]
format=%(asctime)s - %(name)s - %(levelname)s: %(message)s
datefmt=
[formatter_consoleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s: %(message)s
datefmt=
It can be practical to have two or more distinct log files. For example for a Rest service, have one log file for general failures and another for faults in the content.
I tried doing this using INI-file but for some reason all the logs would go to both files. So...
How would the logging.ini look like if I want all logs from logger1 to go to logger1.log and all logs from logger2 to go to logger2.log:
logging.config.fileConfig('logging.ini')
logger1 = logging.getLogger('name1')
logger2 = logging.getLogger('name2')
This works:
python_logging.py
import logging
from logging import config
logging.config.fileConfig('logging.ini')
logger1 = logging.getLogger('name1')
logger2 = logging.getLogger('name2')
logger1.debug('This is logger1')
logger2.info('This is logger2')
logger1.warning('This is logger1')
logger1.error('This is logger1')
logger2.warning('This is logger2')
logger2.error('This is logger2')
logging.ini:
[loggers]
keys=root,name1,name2
[handlers]
keys=console_handler,file_handler_name1,file_handler_name2
[formatters]
keys=console_formatter,file_formatter
[logger_root]
level=INFO
handlers=
[logger_name1]
level=INFO
handlers=console_handler,file_handler_name1
qualname=name1
[logger_name2]
level=INFO
handlers=console_handler,file_handler_name2
qualname=name2
[handler_console_handler]
class=StreamHandler
formatter=console_formatter
args=(sys.stdout,)
[handler_file_handler_name1]
class=handlers.RotatingFileHandler
formatter=file_formatter
args=('name1.log','a',1000000,100)
[handler_file_handler_name2]
class=handlers.RotatingFileHandler
formatter=file_formatter
args=('name2.log','a',1000000,100)
[formatter_console_formatter]
format=%(asctime)s %(levelname)s | %(name)s | %(message)s'
datefmt='%d-%m-%Y %H:%M:%S
[formatter_file_formatter]
format=%(asctime)s %(levelname)s | %(name)s | %(message)s'
datefmt='%d-%m-%Y %H:%M:%S
I'm having a bit of an issue here. I have set up a scratch py file just to test out my logging. Neither the consoleHandler nor the fileHandler are getting the right output (any output). Can anyone see if they can eyeball any issues? Here is my log config file
[loggers]
keys=root
[handlers]
keys=fileHandler, consoleHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=DEBUG
handlers=fileHandler, consoleHandler
formatter=simpleFormatter
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout, )
[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=simpleFormatter
args=(os.path.join(os.getcwd(), 'logging.log'), 'w')
[formatter_simpleFormatter]
format=%(asctime)s - %(levelname)s - %(message)s
datefmt="%Y-%m-%d %H:%M:%S"
Here is the scratch python file,
import logging
from logging import config
LOGGER_NAME = 'Work'
logger = logging.getLogger(LOGGER_NAME)
logging.config.fileConfig('C:\\Users\\cschuma1\\PycharmProjects\\workstuff\\config\\logging.conf')
addition = 'add' + 'ition'
logger.debug('print addition %s', addition)
I have looked closely at other's configs on stackoverflow and I believe that I have all of the necessary levels set, and formatter/handler objects, etc. for my task. Does anyone know what I am missing?
It turns out that at least in python 3, the line
logging.config.fileConfig('C:\\Users\\cschuma1\\PycharmProjects\\workstuff\\config\\logging.conf')
needs to come before,
LOGGER_NAME = 'Work'
logger = logging.getLogger(LOGGER_NAME)
This is the fixed sample program I had above,
import logging
from logging import config
logging.config.fileConfig('C:\\Users\\cschuma1\\PycharmProjects\\workstuff\\config\\logging.conf')
LOGGER_NAME = 'Work'
logger = logging.getLogger(LOGGER_NAME)
addition = 'add' + 'ition'
logger.debug('print addition %s', addition)
Why in python logger.info("print something") does not output. I have seen questions asked before, but solution doesnt exist. I do not want to use logger.debug or logger.warning to see text.
Simply logger.info should print the text, otherwise whats the use of this?
logging.conf file as below
[loggers]
keys=root
[handlers]
keys=stream
[formatters]
keys=formatter
[logger_root]
level=INFO
handlers=stream
[handler_stream]
class=StreamHandler
level=INFO
formatter=formatter
args=(sys.stderr,)
[formatter_formatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
Demo code that access logger:
import logging
logger = logging.getLogger()
if __name__ == '__main__':
logger.info("logger")
print("print")
Output is only print, not the logger. So logger.info does not work.
By default, the root logger (the one you use when you say logger.info) is set at a level of WARN.
You can either do:
logging.basicConfig(level=logging.INFO)
or logging.getLogger().setLevel(logging.INFO)
Seems you do not load your configuration file. You should add this:
logging.config.fileConfig('path_to_logging.conf')
before logger = logging.getLogger()
because right now you are using the default WARNING level.
EDIT: in order to use logging.config, you have to import it too:
import logging.config
So the complete code should be:
import logging
import logging.config
logging.config.fileConfig('path_to_logging.conf')
logger = logging.getLogger()
if __name__ == '__main__':
logger.info("logger")
print("print")
The code above, with the following logging.conf (same as you except I removed the sentry parts):
[loggers]
keys=root
[handlers]
keys=stream
[formatters]
keys=formatter
[logger_root]
level=INFO
handlers=stream
[handler_stream]
class=StreamHandler
level=INFO
formatter=formatter
args=(sys.stderr,)
[formatter_formatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
does work:
$ ./test_script3.py
2016-05-23 15:37:40,437 - root - INFO - logger
print
I want to use a configuration file to create two loggers which will log in two distinct files. My configuration file looks like:
[loggers]
keys=root,main,zipper
[handlers]
keys=main,zip
[formatters]
keys=fmt
[logger_root]
level=DEBUG
handlers=main
[logger_main]
level=DEBUG
handlers=main
qualname=MAIN
[logger_zipper]
level=DEBUG
qualname=UPLOAD
handlers=zip
[handler_zip]
class=FileHandler
level=DEBUG
formatter=fmt
args=('zipper.log','a+')
[handler_main]
class=FileHandler
level=DEBUG
formatter=fmt
args=('main.log','a+')
[formatter_fmt]
format=%(asctime)s - %(levelname)s - %(name)s - %(message)s
I try to use this configuration file like this:
import logging
import logging.config
logging.config.fileConfig("logging.conf")
# Logs to the first file
log = logging.getLogger("")
log.debug("unspec - debug")
log.error("unspec - error")
# Logs to the first file
log_r = logging.getLogger("main")
log_r.debug("main - debug")
log_r.error("main - error")
# Also logs to the first file :(
log_z = logging.getLogger("zipper")
log_z.debug("zipper - debug")
log_z.error("zipper - error")
For some reason I don't understand, all logging messages go to the first file, when I expect the last two to be written to 'zipper.log'. What am I missing ?
The problem is that the qualified name used in the configuration file:
[logger_zipper]
level=DEBUG
qualname=UPLOAD
handlers=zip
doesn't match the one used in the code:
log_z = logging.getLogger("zipper")
Use any of these combinations:
qualname=zipper and logging.getLogger("zipper")
qualname=UPLOAD and logging.getLogger("UPLOAD")