Redirect all stdout/stderr globally to logger - python

Background
I have a very large python application that launches command-line utilities to get pieces of data it needs. I currently just redirect the python launcher script to a log file, which gives me all of the print() output, plus the output of the command-line utilities, i.e.:
python -m launcher.py &> /root/out.log
Problem
I've since implemented a proper logger via logging, which lets me format the logging statements more precisely, lets me limit log file size, etc. I've swapped out most of my print()statements with calls to my logger. However, I have a problem: none of the output from the command-line applications is appearing in my log. It instead gets dumped to the console. Also, the programs aren't all launched the same way: some are launched via popen(), some by exec(), some by os.system(), etc.
Question
Is there a way to globally redirect all stdout/stderr text to my logging function, without having to re-write/modify the code that launches these command-line tools? I tried setting setting the following which I found in another question:
sys.stderr.write = lambda s: logger.error(s)
However it fails with "sys.stderr.write is read-only".


While this is not a full answer, it may show you a redirect to adapt to your particular case. This is how I did it a while back. Although I cannot remember why I did it this way, or what the limitation was I was trying to circumvent, the following is redirecting stdout and stderr to a class for print() statements. The class subsequently writes to screen and to file:
import os
import sys
import datetime
class DebugLogger():
def __init__(self, filename):
timestamp = datetime.datetime.strftime(datetime.datetime.utcnow(),
'%Y-%m-%d-%H-%M-%S-%f')
#build up full path to filename
logfile = os.path.join(os.path.dirname(sys.executable),
filename + timestamp)
self.terminal = sys.stdout
self.log = open(logfile, 'a')
def write(self, message):
timestamp = datetime.datetime.strftime(datetime.datetime.utcnow(),
' %Y-%m-%d-%H:%M:%S.%f')
#write to screen
self.terminal.write(message)
#write to file
self.log.write(timestamp + ' - ' + message)
self.flush()
def flush(self):
self.terminal.flush()
self.log.flush()
os.fsync(self.log.fileno())
def close(self):
self.log.close()
def main(debug = False):
if debug:
filename = 'blabla'
sys.stdout = DebugLogger(filename)
sys.stderr = sys.stdout
print('test')
if __name__ == '__main__':
main(debug = True)


import sys
import io
class MyStream(io.IOBase):
def write(self, s):
logger.error(s)
sys.stderr = MyStream()
print('This is an error', stream=sys.stderr)
This make all call to sys.stderr go to the logger.
The original one is always in sys.__stderr__

Related

Output sys.stdout and sys.stderr to a file

I have a module that redirects console outputs to the Tkinter window, which I found here on StackOverflow. It works perfectly, however, I realized I also need to save the console output to a file. But everything I tried either doesn't work, breaks the module, or nothing happens.
EDIT: I need to modify the following module so it also SAVES THE CONSOLE OUTPUT to the .txt document. BUT preserving it's current functionality
Here is the code for the module:
# Code derived from Bryan Olson's source posted in this related Usenet discussion:
# https://groups.google.com/d/msg/comp.lang.python/HWPhLhXKUos/TpFeWxEE9nsJ
# https://groups.google.com/d/msg/comp.lang.python/HWPhLhXKUos/eEHYAl4dH9YJ
#
# See the comments and doc string below.
#
# Here's a module to show stderr output from console-less Python
# apps, and stay out of the way otherwise. I plan to make a ASPN
# recipe of it, but I thought I'd run it by this group first.
#
# To use it, import the module. That's it. Upon import it will
# assign sys.stderr.
#
# In the normal case, your code is perfect so nothing ever gets
# written to stderr, and the module won't do much of anything.
# Upon the first write to stderr, if any, the module will launch a
# new process, and that process will show the stderr output in a
# window. The window will live until dismissed; I hate, hate, hate
# those vanishing-consoles-with-critical-information.
#
# The code shows some arguably-cool tricks. To fit everthing in
# one file, the module runs the Python interpreter on itself; it
# uses the "if __name__ == '__main__'" idiom to behave radically
# differently upon import versus direct execution. It uses tkinter
# for the window, but that's in a new process; it does not import
# tkinter into your application.
#
# To try it out, save it to a file -- I call it "errorwindow.py" -
# - and import it into some subsequently-incorrect code. For
# example:
#
# import errorwindow
#
# a = 3 + 1 + nonesuchdefined
#
# should cause a window to appear, showing the traceback of a
# Python NameError.
#
# --
# --Bryan
# ----------------------------------------------------------------
#
# martineau - Modified to use subprocess.Popen instead of the os.popen
# which has been deprecated since Py 2.6. Changed so it
# redirects both stdout and stderr. Added numerous
# comments, and also inserted double quotes around paths
# in case they have embedded space characters in them, as
# they did on my Windows system.
#
# Recently updated it to work in both Python 2 and Python 3.
"""
Import this module into graphical Python apps to provide a
sys.stderr. No functions to call, just import it. It uses
only facilities in the Python standard distribution.
If nothing is ever written to stderr, then the module just
sits there and stays out of your face. Upon write to stderr,
it launches a new process, piping it error stream. The new
process throws up a window showing the error messages.
"""
import subprocess
import sys
try:
import thread
except ModuleNotFoundError: # Python 3
import _thread as thread
import os
EXC_INFO_FILENAME = 'exc_info.txt'
if __name__ == '__main__': # When spawned as separate process.
# create window in which to display output
# then copy stdin to the window until EOF
# will happen when output is sent to each OutputPipe created
try:
from Tkinter import BOTH, END, Frame, Text, TOP, YES
import tkFont
import Queue
except ModuleNotFoundError: # Python 3
from tkinter import BOTH, END, Frame, Text, TOP, YES
import tkinter.font as tkFont
import queue as Queue
Q_EMPTY = Queue.Empty # An exception class.
queue = Queue.Queue(1000) # FIFO
def read_stdin(app, bufsize=4096):
fd = sys.stdin.fileno() # File descriptor for os.read() calls.
read = os.read
put = queue.put
while True:
put(read(fd, bufsize))
class Application(Frame):
def __init__(self, master=None, font_size=8, text_color='#0000AA', rows=25, cols=100):
Frame.__init__(self, master)
# Create title based on the arguments passed to the spawned script:
# argv[0]: name of this script (ignored)
# argv[1]: name of script that imported this module
# argv[2]: name of redirected stream (optional)
if len(sys.argv) < 2:
title = "Output stream from unknown source"
elif len(sys.argv) < 3:
title = "Output stream from %s" % (sys.argv[1],)
else: # Assume it's a least 3.
title = "Output stream '%s' from %s" % (sys.argv[2], sys.argv[1])
self.master.title(title)
self.pack(fill=BOTH, expand=YES)
font = tkFont.Font(family='Courier', size=font_size)
width = font.measure(' ' * (cols+1))
height = font.metrics('linespace') * (rows+1)
self.configure(width=width, height=height)
self.pack_propagate(0) # Force frame to be configured size.
self.logwidget = Text(self, font=font)
self.logwidget.pack(side=TOP, fill=BOTH, expand=YES)
# Disallow key entry, but allow text copying with <Control-c>
self.logwidget.bind('<Key>', lambda x: 'break')
self.logwidget.bind('<Control-c>', lambda x: None)
self.logwidget.configure(foreground=text_color)
self.logwidget.insert(END, '==== Start of Output Stream ====\n\n')
self.logwidget.see(END)
self.after(200, self.start_thread) # Start queue polling thread.
def start_thread(self):
thread.start_new_thread(read_stdin, (self,))
self.after(200, self.check_q)
def check_q(self):
log = self.logwidget
log_insert = log.insert
log_see = log.see
queue_get_nowait = queue.get_nowait
go = True
while go:
try:
data = queue_get_nowait().decode() # Must decode for Python 3.
if not data:
data = '[EOF]'
go = False
log_insert(END, data)
log_see(END)
except Q_EMPTY:
self.after(200, self.check_q)
go = False
app = Application()
app.mainloop()
else: # when module is first imported
import traceback
class OutputPipe(object):
def __init__(self, name=''):
self.lock = thread.allocate_lock()
self.name = name
def flush(self): # no-op.
pass
def __getattr__(self, attr):
if attr == 'pipe': # Attribute doesn't exist, so create it.
# Launch this module as a separate process to display any output
# it receives.
# Note: It's important to put double quotes around everything in
# case any have embedded space characters.
command = '"%s" "%s" "%s" "%s"' % (sys.executable, # executable
__file__, # argv[0]
os.path.basename(sys.argv[0]), # argv[1]
self.name) # argv[2]
#
# Typical command and arg values on receiving end:
# C:\Python3\python[w].exe # executable
# C:\vols\Files\PythonLib\Stack Overflow\errorwindow3k.py # argv[0]
# errorwindow3k_test.py # argv[1]
# stderr # argv[2]
# Execute this script directly as __main__ with a stdin PIPE for sending
# output to it.
try:
# Had to also make stdout and stderr PIPEs too, to work with pythonw.exe
self.pipe = subprocess.Popen(command, bufsize=0,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE).stdin
except Exception:
# Output exception info to a file since this module isn't working.
exc_type, exc_value, exc_traceback = sys.exc_info()
msg = ('%r exception in %s\n' %
(exc_type.__name__, os.path.basename(__file__)))
with open(EXC_INFO_FILENAME, 'wt') as info:
info.write('fatal error occurred spawning output process')
info.write('exeception info:' + msg)
traceback.print_exc(file=info)
sys.exit('fatal error occurred')
return super(OutputPipe, self).__getattribute__(attr)
def write(self, data):
with self.lock:
data = data.encode() # Must encode for Python 3.
self.pipe.write(data) # First reference to pipe attr will cause an
# OutputPipe process for the stream to be created.
# Clean-up any left-over debugging file.
try:
os.remove(EXC_INFO_FILENAME) # Delete previous file, if any.
except Exception:
pass
# Redirect standard output streams in the process that imported this module.
sys.stderr = OutputPipe('stderr')
sys.stdout = OutputPipe('stdout')
I tried inserting this code after, before, and in between the module, but it didn't work or broke the module:
import sys path = 'output.txt' sys.stdout = open(path, 'w')
I also tried to do something like this, but it didn't work either.
f = open("output.txt", "w")
f.write(sys.stdout) # or "f.write(OutputPipe('stdout')" or " 'f.write(data)' between - 'def write(self, data):' "
f.close()

try:
sys.stdout = open('file.txt', 'w')
# some working code
finally:
# close file.txt
sys.stdout.close()
sys.stdout = sys.__stdout__
Update. You can write your custom logging class to suit your needs
import sys
class Logger:
def __init__(self, filename):
self.console = sys.stdout
self.file = open(filename, 'w')
def write(self, message):
self.console.write(message)
self.file.write(message)
def flush(self):
self.console.flush()
self.file.flush()
sys.stdout = Logger('file.txt')

I managed to solve my issue by adding a few lines to the code. I'll share here the full modified module code in the unlikely case that someone stumbles upon this same issue.
# Code derived from Bryan Olson's source posted in this related Usenet discussion:
# https://groups.google.com/d/msg/comp.lang.python/HWPhLhXKUos/TpFeWxEE9nsJ
# https://groups.google.com/d/msg/comp.lang.python/HWPhLhXKUos/eEHYAl4dH9YJ
#
# See the comments and doc string below.
#
# Here's a module to show stderr output from console-less Python
# apps, and stay out of the way otherwise. I plan to make a ASPN
# recipe of it, but I thought I'd run it by this group first.
#
# To use it, import the module. That's it. Upon import it will
# assign sys.stderr.
#
# In the normal case, your code is perfect so nothing ever gets
# written to stderr, and the module won't do much of anything.
# Upon the first write to stderr, if any, the module will launch a
# new process, and that process will show the stderr output in a
# window. The window will live until dismissed; I hate, hate, hate
# those vanishing-consoles-with-critical-information.
#
# The code shows some arguably-cool tricks. To fit everthing in
# one file, the module runs the Python interpreter on itself; it
# uses the "if __name__ == '__main__'" idiom to behave radically
# differently upon import versus direct execution. It uses tkinter
# for the window, but that's in a new process; it does not import
# tkinter into your application.
#
# To try it out, save it to a file -- I call it "errorwindow.py" -
# - and import it into some subsequently-incorrect code. For
# example:
#
# import errorwindow
#
# a = 3 + 1 + nonesuchdefined
#
# should cause a window to appear, showing the traceback of a
# Python NameError.
#
# --
# --Bryan
# ----------------------------------------------------------------
#
# martineau - Modified to use subprocess.Popen instead of the os.popen
# which has been deprecated since Py 2.6. Changed so it
# redirects both stdout and stderr. Added numerous
# comments, and also inserted double quotes around paths
# in case they have embedded space characters in them, as
# they did on my Windows system.
#
# Recently updated it to work in both Python 2 and Python 3.
"""
Import this module into graphical Python apps to provide a
sys.stderr. No functions to call, just import it. It uses
only facilities in the Python standard distribution.
If nothing is ever written to stderr, then the module just
sits there and stays out of your face. Upon write to stderr,
it launches a new process, piping it error stream. The new
process throws up a window showing the error messages.
"""
import subprocess
import sys
try:
import thread
except ModuleNotFoundError: # Python 3
import _thread as thread
import os
EXC_INFO_FILENAME = 'exc_info.txt'
if __name__ == '__main__': # When spawned as separate process.
# create window in which to display output
# then copy stdin to the window until EOF
# will happen when output is sent to each OutputPipe created
try:
from Tkinter import BOTH, END, Frame, Text, TOP, YES
import tkFont
import Queue
except ModuleNotFoundError: # Python 3
from tkinter import BOTH, END, Frame, Text, TOP, YES
import tkinter.font as tkFont
import queue as Queue
Q_EMPTY = Queue.Empty # An exception class.
queue = Queue.Queue(1000) # FIFO
def read_stdin(app, bufsize=4096):
fd = sys.stdin.fileno() # File descriptor for os.read() calls.
read = os.read
put = queue.put
while True:
put(read(fd, bufsize))
class Application(Frame):
def __init__(self, master=None, font_size=8, text_color='#0000AA', rows=25, cols=100):
Frame.__init__(self, master)
# Create title based on the arguments passed to the spawned script:
# argv[0]: name of this script (ignored)
# argv[1]: name of script that imported this module
# argv[2]: name of redirected stream (optional)
if len(sys.argv) < 2:
title = "Output stream from unknown source"
elif len(sys.argv) < 3:
title = "Output stream from %s" % (sys.argv[1],)
else: # Assume it's a least 3.
title = "Output stream '%s' from %s" % (sys.argv[2], sys.argv[1])
self.master.title(title)
self.pack(fill=BOTH, expand=YES)
font = tkFont.Font(family='Courier', size=font_size)
width = font.measure(' ' * (cols+1))
height = font.metrics('linespace') * (rows+1)
self.configure(width=width, height=height)
self.pack_propagate(0) # Force frame to be configured size.
self.logwidget = Text(self, font=font)
self.logwidget.pack(side=TOP, fill=BOTH, expand=YES)
# Disallow key entry, but allow text copying with <Control-c>
self.logwidget.bind('<Key>', lambda x: 'break')
self.logwidget.bind('<Control-c>', lambda x: None)
self.logwidget.configure(foreground=text_color)
self.logwidget.insert(END, '==== Start of Output Stream ====\n\n')
# Writes the console output to file!!! :) MOD
with open("Output.txt", "a") as f:
f.write('==== Start of Output Stream ====\n\n')
self.logwidget.see(END)
self.after(200, self.start_thread) # Start queue polling thread.
def start_thread(self):
thread.start_new_thread(read_stdin, (self,))
self.after(200, self.check_q)
def check_q(self):
log = self.logwidget
log_insert = log.insert
log_see = log.see
queue_get_nowait = queue.get_nowait
go = True
while go:
try:
data = queue_get_nowait().decode() # Must decode for Python 3.
with open("Output.txt", "a") as f: # Writes the console output to file!!! :) MOD
f.write(str(data))
if not data:
data = '[EOF]'
go = False
with open("Output.txt", "a") as f: # Writes the '[EOF]' to file!!! :) MOD
f.write(str('\n' + '============ [EOF] =============' + '\n' + '\n' + '\n'))
log_insert(END, data)
log_see(END)
except Q_EMPTY:
self.after(200, self.check_q)
go = False
app = Application()
app.mainloop()
else: # when module is first imported
import traceback
class OutputPipe(object):
def __init__(self, name=''):
self.lock = thread.allocate_lock()
self.name = name
def flush(self): # no-op.
pass
def __getattr__(self, attr):
if attr == 'pipe': # Attribute doesn't exist, so create it.
# Launch this module as a separate process to display any output
# it receives.
# Note: It's important to put double quotes around everything in
# case any have embedded space characters.
command = '"%s" "%s" "%s" "%s"' % (sys.executable, # executable
__file__, # argv[0]
os.path.basename(sys.argv[0]), # argv[1]
self.name) # argv[2]
#
# Typical command and arg values on receiving end:
# C:\Python3\python[w].exe # executable
# C:\vols\Files\PythonLib\Stack Overflow\errorwindow3k.py # argv[0]
# errorwindow3k_test.py # argv[1]
# stderr # argv[2]
# Execute this script directly as __main__ with a stdin PIPE for sending
# output to it.
try:
# Had to also make stdout and stderr PIPEs too, to work with pythonw.exe
self.pipe = subprocess.Popen(command, bufsize=0,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE).stdin
except Exception:
# Output exception info to a file since this module isn't working.
exc_type, exc_value, exc_traceback = sys.exc_info()
msg = ('%r exception in %s\n' %
(exc_type.__name__, os.path.basename(__file__)))
with open(EXC_INFO_FILENAME, 'wt') as info:
info.write('fatal error occurred spawning output process')
info.write('exeception info:' + msg)
traceback.print_exc(file=info)
sys.exit('fatal error occurred')
return super(OutputPipe, self).__getattribute__(attr)
def write(self, data):
with self.lock:
data = data.encode() # Must encode for Python 3.
self.pipe.write(data) # First reference to pipe attr will cause an
# OutputPipe process for the stream to be created.
# Clean-up any left-over debugging file.
try:
os.remove(EXC_INFO_FILENAME) # Delete previous file, if any.
except Exception:
pass
# Redirect standard output streams in the process that imported this module.
sys.stderr = OutputPipe('stderr')
sys.stdout = OutputPipe('stdout')

Save C-level stream print to file in Python 2.7

I am trying to save the terminal output of a python script to a file and print the output to the terminal. Using the logging script below I was able to successfully save the python output, but my script calls GNU Radio as a class and this output is not saved currectly. I believe the problem is that GNU Radio is compiled C code and not captured in the stdout redirect. I have tried using Eli Bendersky's script as a redirect but I am not understanding how to implement this in Python 2.7. Ideally I would be able to add this C-level redirect to the Logger class and save the GNU Radio output in the same generated log file. Thanks for the help!
import sys
class Logger(object):
def __init__(self, filename="Default.log"):
self.terminal = sys.stdout
self.log = open(filename, "a")
def write(self, message):
self.terminal.write(message)
self.log.write(message)
sys.stdout = Logger("yourlogfilename.txt")
print "Hello world !" # this is should be saved in yourlogfilename.txt

How to set custom output handlers for argparse in Python?

I have configured logger to print both onto terminal stdout and to a file so I can have an archive of logging messages that I can refer to.
That is easily accomplished by adding a FileHandler to your logging object. Easy peasy.
What I want to accomplish now is to make argparse log also to the same file along with logs to stdout when it encounters parsing errors. So far it only prints to stdout. I looked in the argparse documentation but I can't find anything about setting a different output stream or pipe for argparse.
Is it possible to do? How?

Looking at the argparse.py source code there doesn't seem to be a way to configure this behaviour.
My suggestion(s) would be:
File a bug report with a patch :)
Override/patch:
print_* method(s)
error method.
The print_* method(s) seem to take an optional file argument which defaults to _sys.stdout.
Update: Alternatively you could do something like this whereby you redirect sys.stdout temporarily while you parse arguments:
from contextlib import contextmanager
#contextmanager
def redirect_stdout_stderr(stream):
old_stdout = sys.stdout
old_stderr = sys.stderr
sys.stdout = stream
sys.stderr = stream
try:
yield
finally:
sys.stdout = old_stdout
sys.stderr = old_stderr
with redirct_stdout_stderr(logstream):
args = parser.parse_args()

There seems to be no way to do this through the API.
However, you can do the following:
class LoggingArgumentParser(argparse.ArgumentParser):
"""Custom ArgumentPaarser that overrides _print_message"""
def _print_message(self, message, file=None):
if message:
logger.write(message)

While the answer given by #James Mills is great and solves the issue, there is no need for a generator in this case. Hence, the yield is redundant. Another way of achieving the same (without the generator) would be to write your own context manager without using the inbuilt contextlib.contextmanager decorator. Like the following.
class redirect_stdout_stderr(object):
def __init__(self, stream):
# Save the old std streams
self.old_stream = sys.stdout
self.old_error_stream = sys.stderr
self.fstream = stream
def __enter__(self):
# Change the std streams to your streams when entering
sys.stdout = self.fstream
sys.stderr = self.fstream
def __exit__(self, exc_type, exc_value, exc_traceback):
# Change the std streams back to the original streams while exiting
sys.stdout = self.old_stream
sys.stderr = self.old_error_stream
In your case you can do something as follows.
with redirect_stdout_stderr(logstream):
# __enter__() is executed
args = parser.parse_args()
# __exit__() is executed
Hope this helps!

Logging realtime stdout to a file in Python?

I want to log the real time stdout from a different Class to a log file.
Here is my demo code. Code works well. But it is waiting for the sample function to finish. I want to run it in parallel. So the log file will written in same time of stdout prints.
import sys
import time
class Logger(object):
def __init__(self):
self.terminal = sys.stdout
self.log = open("log.txt", "a")
def write(self, message):
self.terminal.write(message)
self.log.write(message)
class sample:
def __init__(self):
print "TEST"
time.sleep(5)
if __name__ == "__main__":
a = sample()
sys.stdout = Logger()

Calls to file.write do not necessarily write the contents to the HD immediately. It depends on the buffer policy of the file object. If you want to force writing to disk at a certain moment in time, you can use the flush() method (see also this).
Note that sys.stdout flushing policy depends on the configuration of your installation and also on environmental variables, so if you want to guarantee "parallel" writes between standard output and the log file you must flush() both streams:
def write(self, message):
self.terminal.write(message)
self.log.write(message)
self.terminal.flush()
self.log.flush()

How to capture print output of another module?

I was wondering if this is possible in python:
# module1
def test():
print('hey')
# module2
import module1
module1.test() # prints to stdout
Without modifying module1 is there any way to wrap this in module2 so that I can capture the
print('hey') inside a variable? Apart from running module1 as a script?

I don't want to be responsible for modifying sys.stdout and then restoring it to its previous values. The above answers don't have any finally: clause, which can be dangerous integrating this into other important code.
https://docs.python.org/3/library/contextlib.html
import contextlib, io
f = io.StringIO()
with contextlib.redirect_stdout(f):
module1.test()
output = f.getvalue()
You probably want the variable output which is <class 'str'> with the redirected stdout.
Note: this code is lifted from the official docs with trivial modifications (but tested). Another version of this answer was already given to a mostly duplicated question here: https://stackoverflow.com/a/22434594/1092940
I leave the answer here because it is a much better solution than the others here IMO.

Yes, all you need is to redirect the stdout to a memory buffer that complies with the interface of stdout, you can do it with StringIO. This works for me in 2.7:
import sys
import cStringIO
stdout_ = sys.stdout #Keep track of the previous value.
stream = cStringIO.StringIO()
sys.stdout = stream
print "hello" # Here you can do whatever you want, import module1, call test
sys.stdout = stdout_ # restore the previous stdout.
variable = stream.getvalue() # This will get the "hello" string inside the variable

Yes, you can. You need to take control of sys.stdout. Something like this:
import sys
stdout_ = sys.stdout #Keep track of the previous value.
sys.stdout = open('myoutputfile.txt', 'w') # Something here that provides a write method.
# calls to print, ie import module1
sys.stdout = stdout_ # restore the previous stdout.

For Python 3:
# redirect sys.stdout to a buffer
import sys, io
stdout = sys.stdout
sys.stdout = io.StringIO()
# call module that calls print()
import module1
module1.test()
# get output and restore sys.stdout
output = sys.stdout.getvalue()
sys.stdout = stdout
print(output)

There's No need to use another module, just class object with write attribute, with one input, which you can save in another variable. for ecample
CLASS:
class ExClass:
def __init__(self):
self.st = ''
def write(self, o): #here o is the output that goes to stdout
self.st += str(o)
MAIN Program:
import sys
stdout_ = sys.stdout
var = ExClass()
sys.stdout = var
print("Hello") # these will not be pronted
print("Hello2") # instead will be written in var.st
sys.stdout = stdout_
print(var.st)
output will be
Hello
Hello2

Sending ftplib debug output to the logging module
Based on the approach taken by jimmy kumar ahalpara answer, I was able to capture ftplib's debug output into logging. ftplib was around before the logging module and uses print to emit debug messages.
I'd tried reassigning the print function to a logging method but I couldn't get that to work. The code below works for me.
I should think this will work with other modules as well but there would not be any granularity between different module's output as it's capturing everything sent to stdout to the same logger.
# convenience class to redirect stdout to logging
class SendToLog:
def __init__(self, logging_method):
self.logger = logging
_method
def write(self, o):
if str(o).strip(): # ignore empty lines
self.logger(str(o))
import logging
import sys
# code to initialise logging output and handlers ...
# ...
# get logger for ftplib and redirect it's print output to our log
ftp_logger = logging.getLogger('ftplib')
# note: logging's debug method is passed to the class, the instance then calls this method
sys.stdout = SendToLog(ftp_logger.debug)
# code to do stuff with ftplib ...
# remember to set ftplib's debug level > 0 or there will be no output
# FTP.set_debuglevel(1)
# ...
# important to finalise logging and restore stdout
logging.shutdown()
sys.stdout = sys.__stdout__
python3 stdout ftplib logging

Categories

Resources