By default logging.Formatter('%(asctime)s') prints with the following format:
2011-06-09 10:54:40,638
where 638 is the millisecond. I need to change the comma to a dot:
2011-06-09 10:54:40.638
To format the time I can use:
logging.Formatter(fmt='%(asctime)s',datestr=date_format_str)
however the documentation doesn't specify how to format milliseconds. I've found this SO question which talks about microseconds, but a) I would prefer milliseconds and b) the following doesn't work on Python 2.6 (which I'm working on) due to the %f:
logging.Formatter(fmt='%(asctime)s',datefmt='%Y-%m-%d,%H:%M:%S.%f')
This should work too:
logging.Formatter(
fmt='%(asctime)s.%(msecs)03d',
datefmt='%Y-%m-%d,%H:%M:%S'
)
Please note Craig McDaniel's solution is clearly better.
logging.Formatter's formatTime method looks like this:
def formatTime(self, record, datefmt=None):
ct = self.converter(record.created)
if datefmt:
s = time.strftime(datefmt, ct)
else:
t = time.strftime("%Y-%m-%d %H:%M:%S", ct)
s = "%s,%03d" % (t, record.msecs)
return s
Notice the comma in "%s,%03d". This can not be fixed by specifying a datefmt because ct is a time.struct_time and these objects do not record milliseconds.
If we change the definition of ct to make it a datetime object instead of a struct_time, then (at least with modern versions of Python) we can call ct.strftime and then we can use %f to format microseconds:
import logging
import datetime as dt
class MyFormatter(logging.Formatter):
converter=dt.datetime.fromtimestamp
def formatTime(self, record, datefmt=None):
ct = self.converter(record.created)
if datefmt:
s = ct.strftime(datefmt)
else:
t = ct.strftime("%Y-%m-%d %H:%M:%S")
s = "%s,%03d" % (t, record.msecs)
return s
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
console = logging.StreamHandler()
logger.addHandler(console)
formatter = MyFormatter(fmt='%(asctime)s %(message)s',datefmt='%Y-%m-%d,%H:%M:%S.%f')
console.setFormatter(formatter)
logger.debug('Jackdaws love my big sphinx of quartz.')
# 2011-06-09,07:12:36.553554 Jackdaws love my big sphinx of quartz.
Or, to get milliseconds, change the comma to a decimal point, and omit the datefmt argument:
class MyFormatter(logging.Formatter):
converter=dt.datetime.fromtimestamp
def formatTime(self, record, datefmt=None):
ct = self.converter(record.created)
if datefmt:
s = ct.strftime(datefmt)
else:
t = ct.strftime("%Y-%m-%d %H:%M:%S")
s = "%s.%03d" % (t, record.msecs)
return s
...
formatter = MyFormatter(fmt='%(asctime)s %(message)s')
...
logger.debug('Jackdaws love my big sphinx of quartz.')
# 2011-06-09 08:14:38.343 Jackdaws love my big sphinx of quartz.
Adding msecs was the better option, Thanks.
Here is my amendment using this with Python 3.5.3 in Blender
import logging
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s.%(msecs)03d %(levelname)s:\t%(message)s',
datefmt='%Y-%m-%d %H:%M:%S'
)
log = logging.getLogger(__name__)
log.info("Logging Info")
log.debug("Logging Debug")
The simplest way I found was to override default_msec_format:
formatter = logging.Formatter('%(asctime)s')
formatter.default_msec_format = '%s.%03d'
I figured out a two-liner to get the Python logging module to output timestamps in RFC 3339 (ISO 1801 compliant) format, with both properly formatted milliseconds and timezone and without external dependencies:
import datetime
import logging
# Output timestamp, as the default format string does not include it
logging.basicConfig(format="%(asctime)s: level=%(levelname)s module=%(module)s msg=%(message)s")
# Produce RFC 3339 timestamps
logging.Formatter.formatTime = (lambda self, record, datefmt=None: datetime.datetime.fromtimestamp(record.created, datetime.timezone.utc).astimezone().isoformat())
Example:
>>> logging.getLogger().error("Hello, world!")
2021-06-03T13:20:49.417084+02:00: level=ERROR module=<stdin> msg=Hello, world!
Alternatively, that last line could be written out as follows:
def formatTime_RFC3339(self, record, datefmt=None):
return (
datetime.datetime.fromtimestamp(record.created, datetime.timezone.utc)
.astimezone()
.isoformat()
)
logging.Formatter.formatTime = formatTime_RFC3339
That method could also be used on specific formatter instances, rather than overriding at the class level, in which case you will need to remove self from the method signature.
Many outdated, over-complicated and weird answers here. The reason is that the documentation is inadequate and the simple solution is to just use basicConfig() and set it as follows:
logging.basicConfig(datefmt='%Y-%m-%d %H:%M:%S', format='{asctime}.{msecs:0<3.0f} {name} {threadName} {levelname}: {message}', style='{')
The trick here was that you have to also set the datefmt argument, as the default messes it up and is not what is (currently) shown in the how-to python docs. So rather look here.
An alternative and possibly cleaner way, would have been to override the default_msec_format variable with:
formatter = logging.Formatter('%(asctime)s')
formatter.default_msec_format = '%s.%03d'
However, that did not work for unknown reasons.
PS. I am using Python 3.8.
A simple expansion that doesn't require the datetime module and isn't handicapped like some other solutions is to use simple string replacement like so:
import logging
import time
class MyFormatter(logging.Formatter):
def formatTime(self, record, datefmt=None):
ct = self.converter(record.created)
if datefmt:
if "%F" in datefmt:
msec = "%03d" % record.msecs
datefmt = datefmt.replace("%F", msec)
s = time.strftime(datefmt, ct)
else:
t = time.strftime("%Y-%m-%d %H:%M:%S", ct)
s = "%s,%03d" % (t, record.msecs)
return s
This way a date format can be written however you want, even allowing for region differences, by using %F for milliseconds. For example:
log = logging.getLogger(__name__)
log.setLevel(logging.INFO)
sh = logging.StreamHandler()
log.addHandler(sh)
fm = MyFormatter(fmt='%(asctime)s-%(levelname)s-%(message)s',datefmt='%H:%M:%S.%F')
sh.setFormatter(fm)
log.info("Foo, Bar, Baz")
# 03:26:33.757-INFO-Foo, Bar, Baz
After instantiating a Formatter I usually set formatter.converter = gmtime. So in order for #unutbu's answer to work in this case you'll need:
class MyFormatter(logging.Formatter):
def formatTime(self, record, datefmt=None):
ct = self.converter(record.created)
if datefmt:
s = time.strftime(datefmt, ct)
else:
t = time.strftime("%Y-%m-%d %H:%M:%S", ct)
s = "%s.%03d" % (t, record.msecs)
return s
If you are using arrow or if you don't mind using arrow. You can substitute python's time formatting for arrow's one.
import logging
from arrow.arrow import Arrow
class ArrowTimeFormatter(logging.Formatter):
def formatTime(self, record, datefmt=None):
arrow_time = Arrow.fromtimestamp(record.created)
if datefmt:
arrow_time = arrow_time.format(datefmt)
return str(arrow_time)
logger = logging.getLogger(__name__)
default_handler = logging.StreamHandler()
default_handler.setFormatter(ArrowTimeFormatter(
fmt='%(asctime)s',
datefmt='YYYY-MM-DD HH:mm:ss.SSS'
))
logger.setLevel(logging.DEBUG)
logger.addHandler(default_handler)
Now you can use all of arrow's time formatting in datefmt attribute.
If you prefer to use style='{', fmt="{asctime}.{msecs:0<3.0f}" will 0-pad your microseconds to three places for consistency.
After burning some of my precious time the below hack worked for me. I just updated my formatter in settings.py and added datefmt as %y/%b/%Y %H:%M:%S and appended the milliseconds to the asctime like this {asctime}.{msecs:0<3.0f}
E.G:
'formatters': {
'verbose': {
'format': '[{asctime}.{msecs:0<3.0f}] {levelname} [{threadName:s}] {module} → {message}',
'datefmt': "%y/%b/%Y %H:%M:%S",
'style': '{',
},
}
Using this smart answer for the timezone and the chosen answer, you can construct the millisecond and timezone with your desired format:
import logging
import time
if __name__ == "__main__":
tz = time.strftime('%z')
logging.basicConfig(
format=(
"%(asctime)s.%(msecs)03d" + tz + " %(levelname)s "
"%(pathname)s:%(lineno)d[%(threadName)s]: %(message)s"
),
level=logging.DEBUG,
datefmt="%Y-%m-%dT%H:%M:%S",
)
logging.info("log example")
Personally, I like to keep all the logs in UTC but also have this explicitly in the log as a datetime without a timezone is meaningless in a multizone application:
logging.Formatter.converter = time.gmtime
logging.basicConfig(
format=(
"%(asctime)s.%(msecs)03d+0000 %(levelname)s "
"%(pathname)s:%(lineno)d[%(threadName)s]: %(message)s"
),
level=logging.DEBUG,
datefmt="%Y-%m-%dT%H:%M:%S",
)
tl;dr for folks looking here for an ISO formatted date:
instead of using something like '%Y-%m-%d %H:%M:%S.%03d%z', create your own class as #unutbu indicated. Here's one for iso date format:
import logging
from time import gmtime, strftime
class ISOFormatter(logging.Formatter):
def formatTime(self, record, datefmt=None):
t = strftime("%Y-%m-%dT%H:%M:%S", gmtime(record.created))
z = strftime("%z",gmtime(record.created))
s = "%s.%03d%s" % (t, record.msecs,z)
return s
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
console = logging.StreamHandler()
logger.addHandler(console)
formatter = ISOFormatter(fmt='%(asctime)s - %(module)s - %(levelname)s - %(message)s')
console.setFormatter(formatter)
logger.debug('Jackdaws love my big sphinx of quartz.')
#2020-10-23T17:25:48.310-0800 - <stdin> - DEBUG - Jackdaws love my big sphinx of quartz.
As of now the following works perfectly with python 3 .
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(levelname)-8s %(message)s',
datefmt='%Y/%m/%d %H:%M:%S.%03d',
filename=self.log_filepath,
filemode='w')
gives the following output
2020/01/11 18:51:19.011 INFO
Related
Python logger library -
Hey, is there a way to limit the string size and return another string if it exceeds the max limit?
I have a log file max size set, but it doesn't fit the needs, since the logger sometimes receives a base64 for log, which makes all the logging in the terminal useless.
example:
export = '<Base64 Long String>'
logger.info(f'result: {export}')
Since the code is a part of a big project, I cannot change it in the function itself, is there a way to set it on the logger level?
Use a custom logging.Formatter
import logging
class NotTooLongStringFormatter(logging.Formatter):
def __init__(self, max_length=10):
super(NotTooLongStringFormatter, self).__init__()
self.max_length = max_length
def format(self, record):
if len(record.msg) > self.max_length:
record.msg = record.msg[:self.max_length] + "..."
return super().format(record)
LOG = logging.getLogger("mylogger")
h = logging.StreamHandler()
h.setFormatter(NotTooLongStringFormatter(20))
LOG.addHandler(h)
LOG.error("a" * 10) # aaaaaaaaaa
LOG.info("a" * 100) # aaaaaaaaaaaaaaaaaaaa...
To keep a detailled log, with a specific format, just pass it to super.__init__
def __init__(self, max_length=10):
fmt = '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
super(NotTooLongStringFormatter, self).__init__(fmt)
self.max_length = max_length
2022-01-09 12:29:44,862 - mylogger - WARNING - aaaaaaaaaa
2022-01-09 12:29:44,863 - mylogger - WARNING - aaaaaaaaaaaaaaaaaaaa...
This is the code that I'm running:
import logging
logging.basicConfig(
level=logging.DEBUG,
filename="..\logs\demologs.log",
filemode="a",
format="%(asctime)s - %(levelname)s : %(message)s",
datefmt="%m/%d/%y %I:%M:%S %p",
)
class DemoLogging1:
def add_numbers(self, a, b):
return a + b
def multiply_numbers(self, a, b):
return a * b
dl = DemoLogging1()
sum_result = dl.add_numbers(3, 5)
logging.debug("debug: addition of numbers is: {}".format(sum_result))
logging.info("info: addition of numbers is: {}".format(sum_result))
logging.warning("warning: addition of numbers is: {}".format(sum_result))
logging.error("error: addition of numbers is: {}".format(sum_result))
logging.critical("critical: addition of numbers is: {}".format(sum_result))
multiply_result = dl.multiply_numbers(3, 5)
This won't output the date and time as far as the formatting next to each logging error statement. Why?
You are not getting output to the console because you are not specifying the console stream. It just goes to your file.
To output to both console and file, try this:
logging.basicConfig(
level=logging.DEBUG,
format="%(asctime)s - %(levelname)s : %(message)s",
datefmt="%m/%d/%y %I:%M:%S %p",
handlers=[
logging.StreamHandler(),
logging.FileHandler("test.log", "a"),
],
)
import os
import logging
os.chdir("C:\Users\SIT\Desktop\Powersupply")
logging.basicConfig(
filename='mylog{0}.txt'.format(i),
format='%(asctime)s %(message)s',
datefmt='%m/%d/%Y %I:%M:%S %p'
)
logging.warning(123)
As above , When I execute it once , I want the file name to generate different log names .
ex:
Once -> mylog(0)
Second->mylog(1)
third ->mylog(3)
Try this:
import datetime
now = datetime.datetime.now()
time_string = now.strftime( '%Y-%m-%d_%H%M%S' )
file_name = 'mylog_%s.txt' % time_string
I am looking for a way to change python logging module to display time spent form when the script started instead of current time.
Use %(relativeCreated)s in your format string, as indicated in the documentation.
Update: You can use normal Python format specifiers to control precision, e.g. %(relativeCreated).0f to show floating point values with zero decimal places.
You can subclass logging.Formatter and reimplement formatTime. Something like that:
start_time = datetime.now()
class MyFormatter(logging.Formatter):
def formatTime(self, record, datefmt=None):
delta = (datetime.now() - start_time).total_seconds()
return "{}".format(delta)
And then:
handler = logging.StreamHandler()
fmt = MyFormatter('%(filename)s %(levelname)-8s [%(asctime)s] %(message)s')
handler.setFormatter(fmt)
log = logging.getLogger('main')
log.addHandler(handler)
log.debug("=)")
You could try something like this.
Define a custom formatter at the beginning of your script:
import time
import logging
import datetime as dt
class MyFormatter(logging.Formatter):
def __init__(self,fmt=None,datefmt=None):
super(MyFormatter,self).__init__(fmt,datefmt)
self.reftime = dt.datetime.fromtimestamp(time.mktime(time.localtime()))
def formatTime(self, record, datefmt=None):
ctime = dt.datetime.fromtimestamp(time.mktime(self.converter(record.created)))
ct = (ctime - self.reftime).timetuple()
if datefmt:
s = time.strftime(datefmt, ct)
else:
t = time.strftime("%Y-%m-%d %H:%M:%S", ct)
s = "%s,%03d" % (t, record.msecs)
return s
Then setup your logging system:
handler = logging.StreamHandler(MyFormatter())
logger = logging.getLogger()
logger.addHandler(handler)
etc...
I am using python's log formatter to format log records and i have a fmt value of
fmt = "[%(filename)s:%(lineno)s] %(message)s"
What i would like is that "[file.py:20]" to be stretched to 10 characters wide (for example). If it was one value that would have been easy but is there any way to stretch this entire structure to a specified length?
I want something like:
tmp = "[%(filename)s:%(lineno)s]"
fmt = "%(tmp)10s %(message)s"
I would like to know if this is possible using string formatting or if I can trick python's formatter somehow to get what i want..
As an example, this Formatter ensures a fixed width "[%(filename)s:%(lineno)s]" by either truncating the filename, or right-padding (after the line number) with spaces.
class MyFormatter(logging.Formatter):
width = 10
def format(self, record):
max_filename_width = self.width - 3 - len(str(record.lineno))
filename = record.filename
if len(record.filename) > max_filename_width:
filename = record.filename[:max_filename_width]
a = "%s:%s" % (filename, record.lineno)
return "[%s] %s" % (a.ljust(self.width), record.msg)
if __name__ == '__main__':
logger = logging.getLogger('simple_example')
logger.setLevel(logging.DEBUG)
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
formatter = MyFormatter()
ch.setFormatter(formatter)
logger.addHandler(ch)
logger.debug('No one expects the spammish repetition')
EDIT:
If you want to ensure a minimum width of 10 characters, ditch the filename stuff.
def format(self, record):
a = "%s:%s" % (record.filename, record.lineno)
return "[%s] %s" % (a.ljust(self.width), record.msg)
Option 1
Start here: http://docs.python.org/library/logging.html#formatter-objects
You'll create your own customized subclass of Formatter that provides it's own unique format method.
Then you must be sure to call setFormatter() in each of your Handlers so that they use your new formatter.
Option 2
Create your own subclass of LogRecord with the additional property.
Subclass Logger and override makeRecord to create your new subclass of LogRecord.
Provide a customized format that uses this new property value.
Using #rob-cowie's answer as a basis, I've found the following useful:
class MyFormatter(logging.Formatter):
width = 24
datefmt='%Y-%m-%d %H:%M:%S'
def format(self, record):
cpath = '%s:%s:%s' % (record.module, record.funcName, record.lineno)
cpath = cpath[-self.width:].ljust(self.width)
record.message = record.getMessage()
s = "%-7s %s %s : %s" % (record.levelname, self.formatTime(record, self.datefmt), cpath, record.getMessage())
if record.exc_info:
# Cache the traceback text to avoid converting it multiple times
# (it's constant anyway)
if not record.exc_text:
record.exc_text = self.formatException(record.exc_info)
if record.exc_text:
if s[-1:] != "\n":
s = s + "\n"
s = s + record.exc_text
#if record.stack_info:
# if s[-1:] != "\n":
# s = s + "\n"
# s = s + self.formatStack(record.stack_info)
return s
logFormatter = MyFormatter()
logger = logging.getLogger("example")
logger.setFormatter(logFormatter)
Which gives output like:
WARNING 2014-03-28 16:05:09 module:function:31 : Message
WARNING 2014-03-28 16:05:09 dule:longerfunctions:140 : Message