I have created a logger.py for formatting my python logger, which will have 2 different setup for logging error and info msgs in 2 different files.
import logging
import os
class Logger(logging.Filter):
def __init__(self):
try:
logpath = os.path.dirname(os.getcwd())+'/'
self.info_logger = self.setup_logger('Normal logger', 'Infologger111.log')
self.error_logger = self.setup_logger('Error logger', 'ErrorLogs2222.log')
except Exception as e:
print('[Logger __init__] Exception: ', str(e))
raise
def setup_logger(self, name, log_file, level=logging.DEBUG):
try:
handler = logging.FileHandler(log_file)
handler.setFormatter(logging.Formatter(
'%(asctime)s|%(levelname)s|p%(process)s|[%(pathname)s|%(name)s|%(funcName)s|%(lineno)d]|%(message)s',
datefmt="%Y-%m-%d %H:%M:%S"))
logger = logging.getLogger(name)
logger.setLevel(level)
logger.addHandler(handler)
return logger
except Exception as e:
print('[setup_logger] Exception: ', str(e))
raise
And I have a API_connector class in api_function.py file
import logging
logger = logging.Logger(__name__)
class API_connector(object):
def __init__(self, data={}):
logger.info("**********test log in API_connector")
Now I imported the Logger class as well as API_connector class both into my python code as below,
from logger import Logger
import api_function
class LogTest(Logger):
def __init__(self):
super().__init__()
def test_func(self):
print('inside test func')
self.info_logger.info('test_func log message')
self.error_logger.error('test_func error message ')
api_function.API_connector()
if __name__=='__main__':
ob=LogTest()
ob.info_logger.info('main start')
ob.test_func()
So the issue is 2 different files getting created and getting logged from my python code but the logging I kept in api_function file is not getting printed as my python code file does.
I want the same logger configuration to be impacted in the api_function file, by doing simply like this,
logger = logging.Logger(__name__)
How to make it printed in those 2 configured files with all the logger formatting I have configured?
Related
I need some advice for a problem:
Ich try to do unittests which are started by a campaign runner.
Runner: HTMLTestRunner
The tests are arranged in a package (Testcases). Each testcase is a python module with a class for it. So: -> Testmodul -> TestClass -> test_funct
Now, to extend logging (different reasons like only saving logs for specific errors and starting and ending a screenrecord) I made a decorator which should handle my logfiles. So that means:
When a decorated testcase is started, The logfile handle should be set to a temporary file
During the testcase run, the logfile should be filled with ALL log-messages from the project
When the testcase is finished, the logfile should be copied to another directory (timestamp appended to name) and then the old logfile should be deleted.
So far so good, this all works fine so far, but the content of the logfile is not correct. I tried so many things. Sometimes not all logmessages are covered, sometimes it is a doubled amount of logmessages. Sometimes the messages start in the middle of the testcase. Can you maybe help me at this issue?
Initialization of logger in My Campaign:
def main():
console = logging.StreamHandler()
console.setLevel(logging.INFO)
formatter = logging.Formatter(my_format_string)
console.setFormatter(formatter)
logging.getLogger().addHandler(console)
logger = logging.getLogger("campaign")
logger.setLevel(logging.INFO)
Decorators in other module:
def extended_logging(func):
#wraps(func)
def wrapped_func(*args, **kwargs):
root = logging.getLogger("campaign")
for hndlrs in root.handlers:
root.removeHandler(hndlrs)
console = logging.StreamHandler()
console.setLevel(logging.INFO)
formatter = logging.Formatter(my_format_string)
console.setFormatter(formatter)
root.addHandler(console)
filehandler = logging.FileHandler(config.LOG_FILEPATH, mode='w+')
formatter = logging.Formatter(my_format_string)
filehandler.setFormatter(formatter)
root.addHandler(filehandler)
logger.info("This should actually work")
try:
time.sleep(5)
func(*args, **kwargs)
except Exception as error: # *** This is just dummy error handling for the test
logger.info("An exception occured")
logger.info("This testcase is false")
raise Exception
finally:
# *** Copies and renames the temp logfile and removes it after
save_logs(None, None, "module", "1")
return wrapped_func
Definition of TC is like this:
logger=logging.getLogger(f"camapaign.{__name__}")
class TestClass(unittest.Testcase):
#extended_logging
def test_mylittlemoduletest:
logger.info("Doing my testcase now")
# here smthng will be done
I have a logger in on of my files which has a handler attached to it and it's level has been set to debug. Despite that, when running my program, the debug statement is not printed to the console. The root logger is still set to warning, but I understood that if I add a handler to the logger, the log is passed to that handler and logged before being passed to the parent loggers (which is eventually a null logger). It doesn't seem that is the case. For context here is the code in the file:
logger = logging.getLogger(__name__)
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.DEBUG)
logger.addHandler(console_handler)
class OpenBST:
app_data_folder = Path(user_data_dir(appname=lib_info.lib_name,
appauthor="HydrOffice"))
def __init__(self,
progress: CliProgress = CliProgress(use_logger=True),
app_data_path: Path = app_data_folder) -> None:
app_data_path.mkdir(exist_ok=True, parents=True)
self.progress = progress
self._prj = None
self._app_info = OpenBSTInfo(app_data_path=app_data_path)
self.current_project = None
logging.debug("App instance started")
And below is where it's called in an example script:
from pathlib import Path
from hyo2.openbst.lib.openbst import OpenBST
logging.basicConfig()
logger = logging.getLogger(__name__)
project_directory = Path(os.path.expanduser("~/Documents/openbst_projects"))
project_name = "test_project"
# Create App instance
obst = OpenBST()
Why doesn't the logger.debug('App instance started') not print out to the console?
EDIT:
The code below includes the suggestion from #Jesse R
__init__ was modified as such:
class OpenBST:
app_data_folder = Path(user_data_dir(appname=lib_info.lib_name,
appauthor="HydrOffice"))
def __init__(self,
progress: CliProgress = CliProgress(use_logger=True),
app_data_path: Path = app_data_folder) -> None:
app_data_path.mkdir(exist_ok=True, parents=True)
logger = logging.getLogger(__name__)
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.DEBUG)
logger.addHandler(console_handler)
self.progress = progress
self._prj = None
self._app_info = OpenBSTInfo(app_data_path=app_data_path)
self.current_project = None
logger.debug("App instance started")
No output is generated (exit code 0).
My understanding was a handler attached to a logger would execute before passing log up the chain (where the root is still set to warning).
You call logging.debug("App instance started"), which is not part of the logger that you declare from getLogger. You can set the debug level universally for logging with
logging.basicConfig(level=logging.DEBUG)
also calling logger = logging.getLogger(__name__) outside of the class does not inherit correctly, since you're not passing it but instead use logging. You can create a new logger by moving that declaration inside of the class.
For Example:
import logging
class SampleClass:
def __init__(self):
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
logger.info('will log')
logging.info('will not log')
SampleClass()
Running:
$ python logtest.py
INFO:__main__:will log
I am trying to automate a personal telescope and I have managed to code a camera module, a test driver to run everything and I am working on a logger module for future module debugging. I am having a very hard and frustrating time trying to get errors from the camera module to send to the logger module and then print to my desktop. Pls help.
I have tried and tried again to get any form of log from the cAmera.py folder to print onto the log sheet. There is a problem with filling the log function's 'self' requirement inside of the camera modules. So I made a work around and made logger go before self in camera and I was able to print a test case but not the cases in the if statement.
logger.py
import logging
from main.common import cAmera
from main.common.cAmera import *
class main:
# creates filehandlers
fh = logging.FileHandler('C:\\Users\\Nicholas Pepin\\Desktop\\CameraErrorLog.log')
fh.setLevel(logging.DEBUG)
# Creates console logger for higher level logging
ch = logging.StreamHandler()
ch.setLevel(logging.ERROR)
# Creates Formatters
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh.setFormatter(formatter)
ch.setFormatter(formatter)
# adds handlers to the logger
logger = logging.getLogger()
logger.addHandler(fh)
logger.addHandler(ch)
logger.setLevel(logging.DEBUG)
logger.info('Camera process has started : ' + __name__)
cAmera.Camera.log(logging, logger) # need to fulfil self
# Theoretically, it works. but in 'cAmera.Camera.log()' it says that self needs to be fulfilled
# and I have no idea how to do that. If someone could help guide me to a paper or video that would
# explain it, that would be very helpful.
cAmera.py
import os
import time
import win32com.client
import logging
class Camera:
def __init__(self, logger, output_directory):
# __all__ = ['__init__', 'Camera'] # for some reason, opens up the class and __init__ file?
# output_directory starts from user path
self.logger = logger
logger=logging.getLogger(__name__)
self.output_directory = output_directory
self.camera = win32com.client.Dispatch("MaxIm.CCDCamera") # Sets the camera connection path to the CCDCamera
try:
self.camera.LinkEnabled = True
logger.info("Camera is connected : "+__name__)
except:
logger.critical("Camera is not connected : "+__name__)
self.camera.DisableAutoShutdown = True # All of these settings are just basic camera setup settings.
self.camera.AutoDownload = True
def expose(self, exposure_time, filter, type="light"):
if type == "light":
type = 1
elif type == "dark":
type = 0
else:
print("ERROR: Invalid exposure type.")
return
self.camera.SetFullFrame()
self.camera.Expose(exposure_time, type, filter)
time.sleep(exposure_time)
while self.camera.ImageReady == False:
time.sleep(1)
if self.camera.ImageReady:
# self.camera.StartDownload
path = os.path.expanduser('~')
self.camera.SaveImage(os.path.join(path, 'Desktop', "test_pictures.fit"))
def log(logger, self):
logger.info("Camera test " + __name__)
if self.camera.LinkEnabled:
logger.info("Camera is connected : "+__name__)
elif not self.camera.LinkEnabled:
logger.critical("Camera cannot connect : "+__name__)
def set_gain(self):
pass
def set_binning(self):
pass
test_driver.py
from main.common.cAmera import *
from main.common.logger import *
#activates the camera function
camera_object = Camera("camera_work")
camera_object.expose(10, 1, type="dark")
#activates the logger function
main('camera_work')
I hope to see just an in-depth logger printout on my computer detailing the different ways the code screwed up. Also, if anyone can critique my code and provide help on making me a better programmer, that would be much appreciated.
You have logger in class init as parameter but you don't use it.
class Camera:
def __init__(self, logger, output_directory):
self.logger = logger
logger=logging.getLogger(__name__)
Remove logger=logging.getLogger(__name__)
class Camera:
def __init__(self, logger, output_directory):
self.logger = logger
and run as
Camera(logger, "/output/directory")
or assing this logger as default value in class
class Camera:
def __init__(self, output_directory, logger=None):
if logger:
self.logger = logger
else:
self.logger = logging.getLogger(__name__)
and use with existing logger
logger = ...
Camera("directory/output", logger)
or with logger created inside class
Camera("directory/output")
Now you should use self.logger in all methods in Camera
def __init__(self, logger, output_directory):
if logger:
self.logger = logger
else:
self.logger = logging.getLogger(__name__)
self.output_directory = output_directory
self.camera = win32com.client.Dispatch("MaxIm.CCDCamera")
try:
self.camera.LinkEnabled = True
self.logger.info("Camera is connected : {}".format(__name__))
except Exception as ex:
sel.logger.critical("Camera is not connected : {} ({})".format(__name__, ex))
self.camera.DisableAutoShutdown = True
self.camera.AutoDownload = True
Ther same in method `log
def log(self):
self.logger.info("Camera test : {}".format(__name__))
if self.camera.LinkEnabled:
self.logger.info("Camera is connected : {}".format(__name__))
elif not self.camera.LinkEnabled:
self.logger.critical("Camera cannot connect : {}".format(__name__))
To use Camera you have to create instance
cam = cAmera.Camera("directory/output", logger)
cam.log()
If you create instance before main (which probably should be def instead of class) then put it as arguments
camera_object = Camera("directory/output")
camera_object.expose(10, 1, type="dark")
main('camera_work', camera_object)
def main(text, camera):
cam = camera
cam.log()
or better do it after you create logger
main('camera_work')
def main(text):
logger = ...
camera_object = Camera("directory/output", logger)
camera_object.expose(10, 1, type="dark")
camera_object.log()
I am using a class to create logs for a program I am developing. in the log files I am getting duplicate lines. My code is as follows:
import logging
class Log():
def __init__(self,name=''):
self.name='name'
def InitLog(self,name):
self.logger = logging.getLogger(name)
self.hdlr = logging.FileHandler('/'+name+'.log')
self.formatter = logging.Formatter('%(message)s')
self.hdlr.setFormatter(self.formatter)
self.logger.addHandler(self.hdlr)
self.logger.setLevel(logging.INFO)
def E(self,msg):
self.logger.error(msg)
def I(self,msg):
self.logger.info(msg)
Calling the logger:
# Setup Log
log_url_thread_worker=Log()
log_url_thread_worker.InitLog(cyberlocker)
# Logging something
log_url_thread_worker.I(error)
can anyone see if im doing something stupid?
thanks
I'm trying to create a custom logger this way:
File: logger.py
import logging
class Logger(logging.getLoggerClass()):
def warning(self, msg, *args, **kwargs):
super(Logger, self).warning(msg, *args, **kwargs)
logging.setLoggerClass(Logger)
log = logging.getLogger('test')
handler = logging.StreamHandler()
formatter = logging.Formatter('%(pathname)s')
handler.setFormatter(formatter)
log.addHandler(handler)
File: test.py
from logger import log
log.warning('')
Output:
$ python test.py
/home/dario/Desktop/logging_test/logger.py
The expected output would be:
/home/dario/Desktop/logging_test/test.py
What's even weirder, if I comment the setLoggerClass line I do get test.py, but without the full path.
What am I doing wrong? Thank you!
Tested with Python 2.7.4 and 3.3.1 on Arch Linux x86_64.
From the Python Documentation - Logging - LogRecord attributes:
%(pathname)s - Full pathname of the source file where the logging call was issued (if available).
I.e. where a logging object, (or any subclass, like Logger) was initialized.
In this case logging and your instance of it, log, is both defined and initialized in the same file test.py.
Maybe it's a bit clearer if I split up the files:
logger.py: (the class)
import logging
# This is the path `logging` gets
class Logger(logging.getLoggerClass()):
def warning(self, msg, *args, **kwargs):
super(Logger, self).warning(msg, *args, **kwargs)
log.py: (a log module which has a log object of the type Logger)
import logging
from logger import Logger
logging.setLoggerClass(Logger)
log = logging.getLogger('test')
handler = logging.StreamHandler()
# Not this file's path, but where class Logger
# creates a logging object with super()
formatter = logging.Formatter('%(pathname)s')
handler.setFormatter(formatter)
log.addHandler(handler)
test.py: (main)
from log import log
log.warning('')