I used to Python logging, it works fine. The logging.basicConfig(...) set in one module (a some.py file), then we can use logging every where. Obviously, logging is global.
The question is how logging find it's settings, when we not call the module where basicConfig(...) appeared (in some.py file )? Is logging scan all the packages?
Even the logging.basicConfig(...) put into an any.py and the module (any.py) never get imported, or not used anywhere, the logging setting take effect!
To understand logging you have dive into Python's standard library sources.
Here is the trick:
#/usr/lib/python3.2/logging/__init__.py
...
root = RootLogger(WARNING)
Logger.root = root
Logger.manager = Manager(Logger.root)
...
# and
def basicConfig(**kwargs):
...
hdlr = StreamHandler(stream)
fs = kwargs.get("format", BASIC_FORMAT)
dfs = kwargs.get("datefmt", None)
style = kwargs.get("style", '%')
fmt = Formatter(fs, dfs, style)
hdlr.setFormatter(fmt)
root.addHandler(hdlr)
So, when you call basicconfig() with certain parameters, root logger is set.
Finally getLogger:
def getLogger(name=None):
"""
Return a logger with the specified name, creating it if necessary.
If no name is specified, return the root logger.
"""
if name:
return Logger.manager.getLogger(name)
else:
return root
I think there is no magic scanning here.
Try to test it this way in a separate test directory:
test/main.py:
import logging
logging.info('test')
test/any.py:
import logging
logging.basicConfig(filename='test.log', level=logging.INFO)
python main.py
Result: NO test.log file.
Now let's update the test:
test/main.py:
import logging
import any
logging.info('test')
python main.py
Result: new test.log file with INFO:root:test string inside.
So I guess that any.py in your case is imported somehow,
despite your expectations.
You may find the way any.py is imported easily,
just add few lines there:
test/any.py:
from traceback import print_stack
print_stack()
...
python main.py
Result:
File "main.py", line 2, in
import any
File "any.py", line 2, in
print_stack()
This stack shows that any.py is imported from main.py.
I hope you will find where it is imported from in your case.
Related
Write your code with a nice logger
import logging
def init_logging():
logFormatter = logging.Formatter("[%(asctime)s] %(levelname)s::%(module)s::%(funcName)s() %(message)s")
rootLogger = logging.getLogger()
LOG_DIR = os.getcwd() + '/' + 'logs'
if not os.path.exists(LOG_DIR):
os.makedirs(LOG_DIR)
fileHandler = logging.FileHandler("{0}/{1}.log".format(LOG_DIR, "g2"))
fileHandler.setFormatter(logFormatter)
rootLogger.addHandler(fileHandler)
rootLogger.setLevel(logging.DEBUG)
consoleHandler = logging.StreamHandler()
consoleHandler.setFormatter(logFormatter)
rootLogger.addHandler(consoleHandler)
return rootLogger
logger = init_logging()
works as expected. Logging using logger.debug("Hello! :)") logs to file and console.
In a second step you want to import an external module which is also logging using logging module:
Install it using pip3 install pymisp (or any other external module)
Import it using from pymisp import PyMISP (or any other external module)
Create an object of it using self.pymisp = PyMISP(self.ds_model.api_url, self.ds_model.api_key, False, 'json') (or any other...)
What now happens is, that every debug log output from the imported module is getting logged to the log file and the console. The question now is, how to set a different (higher) log level for the imported module.
As Meet Sinoja and anishtain4 pointed out in the comments, the best and most generic method is to retrieve the logger by the name of the imported module as follows:
import logging
import some_module_with_logging
logging.getLogger("some_module_with_logging").setLevel(logging.WARNING)
Another option (though not recommended if the generic method above works) is to extract the module's logger variable and customize it to your needs. Most third-party modules store it in a module-level variable called logger or _log. In your case:
import logging
import pymisp
pymisp.logger.setLevel(logging.INFO)
# code of module goes here
A colleague of mine helped with this question:
Get a named logger yourLogger = logging.getLogger('your_logger')
Add a filter to each handler prevents them to print/save other logs than yours
for handler in logging.root.handlers:
handler.addFilter(logging.Filter('your_logger'))
I just started Python and I am struggling to use Logger. I have two python files: app.py and liba.py. I want to setup logging at app.py and use it for liba.py (and another libraries). Do you have any good ideas or can you share any references?
file structure
entry.py
lib/liba.py
app.py
#! /usr/bin/env python3
import logging
logger = logging.getLogger(__name__)
from lib import liba
handler = logging.FileHandler('/tmp/app.log', 'a+')
logger.addHandler(handler)
logger.warn('sample')
liba.out()
lib/liba.py
#! /usr/bin/env python3
import logging
logger = logging.getLogger(__name__)
def out():
logger.warn('liba')
run python
$ python3 app.py
liba
app.py output log to the logfile. liba.py does not output the log into the file. I want to save logs in the same file.
Do like so:
app.py
#! /usr/bin/env python3
import logging
logger = logging.getLogger()
handler = logging.FileHandler('/tmp/app.log', 'a+')
logger.addHandler(handler)
logger.warn('sample')
from lib import liba
liba.out()
lib/liba.py
#! /usr/bin/env python3
import logging
def out():
logging.warn('liba')
You don't need to instantiate the logging, unless you want to configure handlers, which you only do in your main script. Then all logging will go to the root logger which is what you get when instantiating with no specific name logging.getLogger(). I like to use it this way as you don't need to match names across all your modules for it to work. In your modules you just send log messages out by using logging.warn('blabla'). You further need to make sure you define all your handlers before any call to logging.warn is made, otherwise some default handler will take its place.
Im trying to import files on Flask app in base of url route. I started to coding python few days ago so i havent idea if i doing it well. I write this on :
#app.route('/<file>')
def call(file):
__import__('controller.'+file)
hello = Example('Hello world')
return hello.msg
And i have other file called example.py into a controller folder that contains this:
class Example:
def __init__(self, msg):
self.msg = msg
So i start from terminal the app and i try to enter to localhost:5000/example.
Im trying to show in screen Hello world but give me the next error:
NameError: global name 'Example' is not defined
Thanks for all!
__import__ returns the newly imported module; names from that module are not added to your globals, so you need to get the Example class as an attribute from the returned module:
module = __import__('controller.'+file)
hello = module.Example('Hello world')
__import__ is rather low-level, you probably want to use importlib.import_module() instead:
import importlib
module = importlib.import_module('controller.'+file)
hello = module.Example('Hello world')
If you need to dynamically get the classname too, use getattr():
class_name = 'Example'
hello_class = getattr(module, class_name)
hello = hello_class('Hello world')
The Werkzeug package (used by Flask) offers a helpful functions here: werkzeug.utils.import_string() imports an object dynamically:
from werkzeug.utils import import_string
object_name = 'controller.{}:Example'.format(file)
hello_class = import_string(object_name)
This encapsulates the above process.
You'll need to be extremely careful with accepting names from web requests and using those as module names. Please do sanitise the file argument and only allow alphanumerics to prevent relative imports from being used.
You could use the werkzeug.utils.find_modules() function to limit the possible values for file here:
from werkzeug.utils import find_modules, import_string
module_name = 'controller.{}'.format(file)
if module_name not in set(find_modules('controller')):
abort(404) # no such module in the controller package
hello_class = import_string(module_name + ':Example')
I think you might not add the directory to the file, add the following code into the previous python program
# Add another directory
import sys
sys.path.insert(0, '/your_directory')
from Example import Example
There are two ways for you to do imports in Python:
import example
e = example.Example('hello world')
or
from example import Example
e = Example('hello world')
I am using standard logging configurations, set in settings.py file, and accessed in program but I get the error
error No handlers could be found for logger.
It works when run from the console but does not work when run from Eclipse.
The code is as follows:
import logging
from config import settings
logger = logging.getLogger('engine')
class ReplyUser(object):
def __init__(self):
logger.info("Initalizes ReplyUser")
def myfun(self):
logger.info("Hi")
print "hi"
I am guessing the problem is in the PATH which eclipse is using and it is unable to find settings.py as the handler information is stored in the settings.py file, hence the error.
I have a Python program that consists of several modules. The "main" module creates a file variable log_file for logging the output; all the other modules would need to write to that file as well.
However, I don't want to import the "main" module into other modules, since it would be a really weird dependency (not to mention it might not even work due to circular dependency).
Where, then, should I store the log_file variable?
EDIT:
Following #pyfunc answer - would this be ok:
--- config.py ---
# does not mention log_file
# unless it's required for syntax reasons; in which case log_file = None
# ...
--- main.py ---
from datetime import datetime
import config.py
log_filename = str(datetime.now()) + '.txt'
config.log_file = open(log_filename, 'w')
# ...
--- another_module.py ---
import config.py
# ...
config.log_file.write(some_stuff)
Put the "global" variable in a "settings" module.
settings.py
log_file = "/path/to/file"
main.py
import settings
import logging
logging.basicConfig(filename=settings.log_file,level=logging.DEBUG)
logging.debug("This should go to the log file")
other_module.py
import logging
logging.debug("This is a message from another place.")
While the logging module may solve your immediate problem and many others, the settings module pattern is useful for a lot of other things besides log file names. It is used by Django to configure just about everything.
One way is to take all that code into another module so that you could import it in main file and other modules as well.
Hope you have checked on : http://docs.python.org/library/logging.html