Python - logging and configuration change - python

Ok, it's a little hard to describe my problem, I will try as much as possible to explain it simply.
When my application starts up, it creates Logger from logging lib.
Before loading configuration file, the only handler for logger is stdout.
After several printed logs, application finally loads configuration file which include Logger configuration options, such as file for store logs. I modify my Logger options to use file handler for this file, simultaneously with stdout handler. But i need to store all previous logs generated by my script in this file, using formatting from configuration. I was thinking about MemoryHandler, running simultaneously with stdout handler, and after loading configuration write all logs from memory to file - before creating a file handler.
The problem is that MemoryHandler is not well documented, and this way to solve the problem is not pretty looks for me. So, in brief - I looking for way to save log into file from MemoryHandler, or better way to solve this problem.

Related

How to see log messages from a logger module in a .pyx (cython) file?

I'm trying to better understand how a particular C wrapper library is working, so I want to see the log messages in its associated *.pyx file.
In the file, I see they've already set up a logger - I just don't know how, when I'm running the library's *.py examples, to access the log messages that the *.pyx file is generating.
In the pyx file they have
import logging
logger = logging.getLogger(__name__)
Then, of course, throughout the logic of the file there are useful values being sent, like
logger.debug('io map size: {}'.format(ret_val) and so on.
What do I need to do to see these messages? Thanks for any help you can provide!

How to use logger with one basic config across app in Python

I want to improve my understanding of how to use logging correctly in Python. I want to use .ini file to configure it and what I want to do:
define basic logger config through .fileConfig(...) in some .py file
import logger, call logger = logging.getLogger(__name__) across the app and be sure that it uses my config file that I was loaded recently in different .py file
I read few resources over Internet ofc but they are describing tricks of how to configure it etc, but want I to understand is that .fileConfig works across all app or works only for file/module where it was declared.
Looks like I missed some small tip or smth like that.
It works across the whole app. Be sure to configure the correct loggers in the config. logger = logging.getLogger(__name__) works well if you know how to handle having a different logger in every module, otherwise you might be happier just calling logger = logging.getLogger("mylogger") which always gives you the same logger. If you only configure the root logger you might even skip that and simply use logging.info("message") directly.

How do I define a different logger for an imported module in Python?

I'm using Advanced Python Scheduler in a Python script. The main program defines a log by calling logging.basicConfig with the file name of the log that I want. This log is also set to "DEBUG" as the logging level, since that's what I need at present for my script.
Unfortunately, because logging.basicConfig has been set up in this manner, apscheduler writes its log entries to the same log file. There are an awful lot of these, especially since I have one scheduled task that runs every minute.
Is there any way to redirect apscheduler's log output to another log file (without changing apscheduler's code) while using my log file for my own script? I.e. is there a way to change the file name for each module's output within my script?
I tried reading the module page and the HOWTO for logging, but could not find an answer to this.
Set the logger level for apscheduler to your desired value (e.g. WARNING to avoid seeing DEBUG and INFO messages from apscheduler like this:
logging.getLogger('apscheduler').setLevel(logging.WARNING)
You will still get messages for WARNING and higher severities. To direct messages from apscheduler into a separate file, use
aplogger = logging.getLogger('apscheduler')
aplogger.propagate = False
aplogger.setLevel(logging.WARNING) # or whatever
aphandler = logging.FileHandler(...) # as per what you want
aplogger.addHandler(aphandler)
Ensure the above code is only called once (otherwise you will add multiple FileHandler instances - probably not what you want).
maybe you want to call logging.getLogger("apscheduler") and setup its log file in there? see this answer https://stackoverflow.com/a/2031557/782168

How can I mirror Django logging to a second file temporarily?

I want to create task-specific log files that reflect what's happening during a particular operation, but I want these logging messages to also go to the primary Django log.
My current solution, which seems to work fine at first glance, is something like:
logger = getLogger("%s:%s" % (__name__, task_id))
handler = FileHandler(task_log_file)
logger.addHandler(handler)
# Work
logger.removeHandler(handler)
As I said, this works, but the main issue that occurs to me is that this logger isn't really temporary -- from what I've read of logging.Manager each logger will just hang around indefinitely until shutdown. In this case, when I'm done I know I won't use the logger again (okay, technically I might, but that will be rare), and assuming the system is stable this could be running through hundreds of thousands of tasks.
Is there a "right" way to do this?
You could have a way to mark-and-sweep your logging resource files, or just use a singleton pattern.

Python logging help needed for newbie

Help me understand please. I'm new in python. I'm writing WSGI app. And I want to turn loggining on. I do:
logger = logging.getLogger(__name__)
then I want to print something in log:
logger.exception("Some exception...")
Where it all stores? Only in console output or there is a file somewhere on the server? If so, where I can find this file?
Actually, you have to specify the file yourself, e.g. with logging.basicConfig():
logging.basicConfig(filename="/path/to/your/logfile.log")
Logging can be quite a complex matter sometimes, I'd suggest you read the HOWTO.
import logging
logging.error("Error") # Prints to console
This how to write a log to a file
logging.basicConfig(filename='Error.log',level=logging.ERROR)
logging.error('Error')
Opening the Error.log you will see
ERROR:root:Error
More info here

Categories

Resources