Related
I have a Python script that is using some closed-box Python functions (i.e. I can't edit these functions) provided by my employer. When I call these functions, they are printing output to my linux terminal that I would like to suppress. I've tried redirecting stdout / stderr via;
orig_out = sys.stdout
sys.stdout = StringIO()
rogue_function()
sys.stdout = orig_out
but this fails to catch the output. I think the functions I'm calling via-Python (rogue_function() from above) are really wrappers for compiled C-code, which are actually doing the printing.
Does anyone know of a way I can do a "deep-capture" of any print handed to stdout / stderr by a function (and any sub-functions that function calls)?
UPDATE:
I ended up taking the method outlined in the selected answer below and writing a context manager to supress stdout and stderr:
# Define a context manager to suppress stdout and stderr.
class suppress_stdout_stderr(object):
'''
A context manager for doing a "deep suppression" of stdout and stderr in
Python, i.e. will suppress all print, even if the print originates in a
compiled C/Fortran sub-function.
This will not suppress raised exceptions, since exceptions are printed
to stderr just before a script exits, and after the context manager has
exited (at least, I think that is why it lets exceptions through).
'''
def __init__(self):
# Open a pair of null files
self.null_fds = [os.open(os.devnull,os.O_RDWR) for x in range(2)]
# Save the actual stdout (1) and stderr (2) file descriptors.
self.save_fds = [os.dup(1), os.dup(2)]
def __enter__(self):
# Assign the null pointers to stdout and stderr.
os.dup2(self.null_fds[0],1)
os.dup2(self.null_fds[1],2)
def __exit__(self, *_):
# Re-assign the real stdout/stderr back to (1) and (2)
os.dup2(self.save_fds[0],1)
os.dup2(self.save_fds[1],2)
# Close all file descriptors
for fd in self.null_fds + self.save_fds:
os.close(fd)
To use this you just:
with suppress_stdout_stderr():
rogue_function()
This works "pretty good". It does suppress the printout from the rogue functions that were cluttering up my script. I noticed in testing it that it lets through raised exceptions as well as some logger print, and I'm not entirely clear why. I think it has something to do with when these messages get sent to stdout / stderr (I think it happens after my context manager exits). If anyone can confirm this, I'd be interested in hearing the details ...
As of python 3.5 we can do this with minimal work using built-ins in contextlib, namely redirect_stdout and redirect_stderr. We only need to combine these two built-in context managers in a custom context manager of ours, which can be easily done using the nice pattern in Martijn's answer here. Redirecting both outputs to os.devnull should be safe and portable enough.
from contextlib import contextmanager,redirect_stderr,redirect_stdout
from os import devnull
#contextmanager
def suppress_stdout_stderr():
"""A context manager that redirects stdout and stderr to devnull"""
with open(devnull, 'w') as fnull:
with redirect_stderr(fnull) as err, redirect_stdout(fnull) as out:
yield (err, out)
Note that suppressing stderr will still give you full tracebacks when something breaks, which is a good thing:
import sys
def rogue_function():
print('spam to stdout')
print('important warning', file=sys.stderr)
1 + 'a'
return 42
with suppress_stdout_stderr():
rogue_function()
When run the above only prints
Traceback (most recent call last):
File "tmp.py", line 20, in <module>
rogue_function()
File "foo.py", line 16, in rogue_function
1 + 'a'
TypeError: unsupported operand type(s) for +: 'int' and 'str'
to the terminal. Unhandled exceptions should never go unnoticed.
This approach (found through the related sidebar) might work. It reassigns the file descriptors rather than just the wrappers to them in sys.stdout, etc.
python 3.6 working version, tested with million suppressions without any errors
import os
import sys
class suppress_stdout_stderr(object):
def __enter__(self):
self.outnull_file = open(os.devnull, 'w')
self.errnull_file = open(os.devnull, 'w')
self.old_stdout_fileno_undup = sys.stdout.fileno()
self.old_stderr_fileno_undup = sys.stderr.fileno()
self.old_stdout_fileno = os.dup ( sys.stdout.fileno() )
self.old_stderr_fileno = os.dup ( sys.stderr.fileno() )
self.old_stdout = sys.stdout
self.old_stderr = sys.stderr
os.dup2 ( self.outnull_file.fileno(), self.old_stdout_fileno_undup )
os.dup2 ( self.errnull_file.fileno(), self.old_stderr_fileno_undup )
sys.stdout = self.outnull_file
sys.stderr = self.errnull_file
return self
def __exit__(self, *_):
sys.stdout = self.old_stdout
sys.stderr = self.old_stderr
os.dup2 ( self.old_stdout_fileno, self.old_stdout_fileno_undup )
os.dup2 ( self.old_stderr_fileno, self.old_stderr_fileno_undup )
os.close ( self.old_stdout_fileno )
os.close ( self.old_stderr_fileno )
self.outnull_file.close()
self.errnull_file.close()
Did you try to redirect stderr too?
e.g.
sys.stdout = StringIO()
sys.stderr = StringIO()
foo(bar)
sys.stdout = sys.__stdout__ # These are provided by python
sys.stderr = sys.__stderr__
Also using StringIO might use extra memory. You can use a dummy device instead (e.g. http://coreygoldberg.blogspot.com/2009/05/python-redirect-or-turn-off-stdout-and.html).
My solution is similar to yours but uses contextlib and is a little shorter and easier to understand (IMHO).
import contextlib
#contextlib.contextmanager
def stdchannel_redirected(stdchannel, dest_filename):
"""
A context manager to temporarily redirect stdout or stderr
e.g.:
with stdchannel_redirected(sys.stderr, os.devnull):
if compiler.has_function('clock_gettime', libraries=['rt']):
libraries.append('rt')
"""
try:
oldstdchannel = os.dup(stdchannel.fileno())
dest_file = open(dest_filename, 'w')
os.dup2(dest_file.fileno(), stdchannel.fileno())
yield
finally:
if oldstdchannel is not None:
os.dup2(oldstdchannel, stdchannel.fileno())
if dest_file is not None:
dest_file.close()
The context for why I created this is at this blog post. Similar to yours I think.
I use it like this in a setup.py:
with stdchannel_redirected(sys.stderr, os.devnull):
if compiler.has_function('clock_gettime', libraries=['rt']):
libraries.append('rt')
Not really requested by the OP, but I needed to hide and store the output, and did like follows:
from io import StringIO
import sys
class Hider:
def __init__(self, channels=('stdout',)):
self._stomach = StringIO()
self._orig = {ch : None for ch in channels}
def __enter__(self):
for ch in self._orig:
self._orig[ch] = getattr(sys, ch)
setattr(sys, ch, self)
return self
def write(self, string):
self._stomach.write(string)
def flush(self):
pass
def autopsy(self):
return self._stomach.getvalue()
def __exit__(self, *args):
for ch in self._orig:
setattr(sys, ch, self._orig[ch])
Usage:
with Hider() as h:
spammy_function()
result = h.autopsy()
(tested only with Python 3)
EDIT: now allows to select stderr, stdout or both, as in Hider([stdout, stderr])
I use a decorator for this. It saves sys.stdout and sys.stderr references and makes these variables point to null. Then, after the function execution the original references are retrieved. It is important to note the try/except block, that allows the retrieval of the original references even when an exception is raised on the function.
def suppress_std(func):
def wrapper(*args, **kwargs):
stderr_tmp = sys.stderr
stdout_tmp = sys.stdout
null = open(os.devnull, 'w')
sys.stdout = null
sys.stderr = null
try:
result = func(*args, **kwargs)
sys.stderr = stderr_tmp
sys.stdout = stdout_tmp
return result
except:
sys.stderr = stderr_tmp
sys.stdout = stdout_tmp
raise
return wrapper
To use:
#suppress_std
def function_std_suppressed():
# code here
Just use Linux/Unix:
./myscript.py 2>/dev/null # gets rid of stderr
./myscript.py 2>/somewhere/myerror.log
If you are running this script on a linux based machine, you should be able to:
$> ./runscript.py > output.txt
I have a Python script that is using some closed-box Python functions (i.e. I can't edit these functions) provided by my employer. When I call these functions, they are printing output to my linux terminal that I would like to suppress. I've tried redirecting stdout / stderr via;
orig_out = sys.stdout
sys.stdout = StringIO()
rogue_function()
sys.stdout = orig_out
but this fails to catch the output. I think the functions I'm calling via-Python (rogue_function() from above) are really wrappers for compiled C-code, which are actually doing the printing.
Does anyone know of a way I can do a "deep-capture" of any print handed to stdout / stderr by a function (and any sub-functions that function calls)?
UPDATE:
I ended up taking the method outlined in the selected answer below and writing a context manager to supress stdout and stderr:
# Define a context manager to suppress stdout and stderr.
class suppress_stdout_stderr(object):
'''
A context manager for doing a "deep suppression" of stdout and stderr in
Python, i.e. will suppress all print, even if the print originates in a
compiled C/Fortran sub-function.
This will not suppress raised exceptions, since exceptions are printed
to stderr just before a script exits, and after the context manager has
exited (at least, I think that is why it lets exceptions through).
'''
def __init__(self):
# Open a pair of null files
self.null_fds = [os.open(os.devnull,os.O_RDWR) for x in range(2)]
# Save the actual stdout (1) and stderr (2) file descriptors.
self.save_fds = [os.dup(1), os.dup(2)]
def __enter__(self):
# Assign the null pointers to stdout and stderr.
os.dup2(self.null_fds[0],1)
os.dup2(self.null_fds[1],2)
def __exit__(self, *_):
# Re-assign the real stdout/stderr back to (1) and (2)
os.dup2(self.save_fds[0],1)
os.dup2(self.save_fds[1],2)
# Close all file descriptors
for fd in self.null_fds + self.save_fds:
os.close(fd)
To use this you just:
with suppress_stdout_stderr():
rogue_function()
This works "pretty good". It does suppress the printout from the rogue functions that were cluttering up my script. I noticed in testing it that it lets through raised exceptions as well as some logger print, and I'm not entirely clear why. I think it has something to do with when these messages get sent to stdout / stderr (I think it happens after my context manager exits). If anyone can confirm this, I'd be interested in hearing the details ...
As of python 3.5 we can do this with minimal work using built-ins in contextlib, namely redirect_stdout and redirect_stderr. We only need to combine these two built-in context managers in a custom context manager of ours, which can be easily done using the nice pattern in Martijn's answer here. Redirecting both outputs to os.devnull should be safe and portable enough.
from contextlib import contextmanager,redirect_stderr,redirect_stdout
from os import devnull
#contextmanager
def suppress_stdout_stderr():
"""A context manager that redirects stdout and stderr to devnull"""
with open(devnull, 'w') as fnull:
with redirect_stderr(fnull) as err, redirect_stdout(fnull) as out:
yield (err, out)
Note that suppressing stderr will still give you full tracebacks when something breaks, which is a good thing:
import sys
def rogue_function():
print('spam to stdout')
print('important warning', file=sys.stderr)
1 + 'a'
return 42
with suppress_stdout_stderr():
rogue_function()
When run the above only prints
Traceback (most recent call last):
File "tmp.py", line 20, in <module>
rogue_function()
File "foo.py", line 16, in rogue_function
1 + 'a'
TypeError: unsupported operand type(s) for +: 'int' and 'str'
to the terminal. Unhandled exceptions should never go unnoticed.
This approach (found through the related sidebar) might work. It reassigns the file descriptors rather than just the wrappers to them in sys.stdout, etc.
python 3.6 working version, tested with million suppressions without any errors
import os
import sys
class suppress_stdout_stderr(object):
def __enter__(self):
self.outnull_file = open(os.devnull, 'w')
self.errnull_file = open(os.devnull, 'w')
self.old_stdout_fileno_undup = sys.stdout.fileno()
self.old_stderr_fileno_undup = sys.stderr.fileno()
self.old_stdout_fileno = os.dup ( sys.stdout.fileno() )
self.old_stderr_fileno = os.dup ( sys.stderr.fileno() )
self.old_stdout = sys.stdout
self.old_stderr = sys.stderr
os.dup2 ( self.outnull_file.fileno(), self.old_stdout_fileno_undup )
os.dup2 ( self.errnull_file.fileno(), self.old_stderr_fileno_undup )
sys.stdout = self.outnull_file
sys.stderr = self.errnull_file
return self
def __exit__(self, *_):
sys.stdout = self.old_stdout
sys.stderr = self.old_stderr
os.dup2 ( self.old_stdout_fileno, self.old_stdout_fileno_undup )
os.dup2 ( self.old_stderr_fileno, self.old_stderr_fileno_undup )
os.close ( self.old_stdout_fileno )
os.close ( self.old_stderr_fileno )
self.outnull_file.close()
self.errnull_file.close()
Did you try to redirect stderr too?
e.g.
sys.stdout = StringIO()
sys.stderr = StringIO()
foo(bar)
sys.stdout = sys.__stdout__ # These are provided by python
sys.stderr = sys.__stderr__
Also using StringIO might use extra memory. You can use a dummy device instead (e.g. http://coreygoldberg.blogspot.com/2009/05/python-redirect-or-turn-off-stdout-and.html).
My solution is similar to yours but uses contextlib and is a little shorter and easier to understand (IMHO).
import contextlib
#contextlib.contextmanager
def stdchannel_redirected(stdchannel, dest_filename):
"""
A context manager to temporarily redirect stdout or stderr
e.g.:
with stdchannel_redirected(sys.stderr, os.devnull):
if compiler.has_function('clock_gettime', libraries=['rt']):
libraries.append('rt')
"""
try:
oldstdchannel = os.dup(stdchannel.fileno())
dest_file = open(dest_filename, 'w')
os.dup2(dest_file.fileno(), stdchannel.fileno())
yield
finally:
if oldstdchannel is not None:
os.dup2(oldstdchannel, stdchannel.fileno())
if dest_file is not None:
dest_file.close()
The context for why I created this is at this blog post. Similar to yours I think.
I use it like this in a setup.py:
with stdchannel_redirected(sys.stderr, os.devnull):
if compiler.has_function('clock_gettime', libraries=['rt']):
libraries.append('rt')
Not really requested by the OP, but I needed to hide and store the output, and did like follows:
from io import StringIO
import sys
class Hider:
def __init__(self, channels=('stdout',)):
self._stomach = StringIO()
self._orig = {ch : None for ch in channels}
def __enter__(self):
for ch in self._orig:
self._orig[ch] = getattr(sys, ch)
setattr(sys, ch, self)
return self
def write(self, string):
self._stomach.write(string)
def flush(self):
pass
def autopsy(self):
return self._stomach.getvalue()
def __exit__(self, *args):
for ch in self._orig:
setattr(sys, ch, self._orig[ch])
Usage:
with Hider() as h:
spammy_function()
result = h.autopsy()
(tested only with Python 3)
EDIT: now allows to select stderr, stdout or both, as in Hider([stdout, stderr])
I use a decorator for this. It saves sys.stdout and sys.stderr references and makes these variables point to null. Then, after the function execution the original references are retrieved. It is important to note the try/except block, that allows the retrieval of the original references even when an exception is raised on the function.
def suppress_std(func):
def wrapper(*args, **kwargs):
stderr_tmp = sys.stderr
stdout_tmp = sys.stdout
null = open(os.devnull, 'w')
sys.stdout = null
sys.stderr = null
try:
result = func(*args, **kwargs)
sys.stderr = stderr_tmp
sys.stdout = stdout_tmp
return result
except:
sys.stderr = stderr_tmp
sys.stdout = stdout_tmp
raise
return wrapper
To use:
#suppress_std
def function_std_suppressed():
# code here
Just use Linux/Unix:
./myscript.py 2>/dev/null # gets rid of stderr
./myscript.py 2>/somewhere/myerror.log
If you are running this script on a linux based machine, you should be able to:
$> ./runscript.py > output.txt
How do I get sys.stdout to be monitored by GObject.io_add_watch?
code:
GObject.io_add_watch(os.fdopen(self.builder), GObject.IO_IN, self.write_to_buffer)
I tried to get the stdout stream of my main GTK process via os.fdopen(self.builder),
yet it raises the exception:
TypeError: an integer is required
Could you please explain how I have to properly advise GObject.io_add_watch to
watch the sys.stdout stream?
Thanks!
The API will let you pass sys.stdout directly, since it's a file-like object with a fileno() method. It won't work right, though, because you can't read from sys.stdout. If you want to capture all attempts to write to stdout, you can replace it with a pipe:
import gobject
import sys
import os
def ok(stream, condition):
s = os.fdopen(stream)
data = s.readline()
with open("called.txt", "w") as f:
f.write("got {} ".format(data))
return False
def do_print():
print "hey there"
sys.stdout.flush()
rpipe, wpipe = os.pipe()
wpipe = os.fdopen(wpipe, "w", 0)
sys.stdout = wpipe # Replace sys.stdout
gobject.io_add_watch(rpipe, gobject.IO_IN, ok)
gobject.idle_add(do_print)
gobject.MainLoop().run()
Contents of "called.txt" after running the script:
got hey there
I have a Python script that is using some closed-box Python functions (i.e. I can't edit these functions) provided by my employer. When I call these functions, they are printing output to my linux terminal that I would like to suppress. I've tried redirecting stdout / stderr via;
orig_out = sys.stdout
sys.stdout = StringIO()
rogue_function()
sys.stdout = orig_out
but this fails to catch the output. I think the functions I'm calling via-Python (rogue_function() from above) are really wrappers for compiled C-code, which are actually doing the printing.
Does anyone know of a way I can do a "deep-capture" of any print handed to stdout / stderr by a function (and any sub-functions that function calls)?
UPDATE:
I ended up taking the method outlined in the selected answer below and writing a context manager to supress stdout and stderr:
# Define a context manager to suppress stdout and stderr.
class suppress_stdout_stderr(object):
'''
A context manager for doing a "deep suppression" of stdout and stderr in
Python, i.e. will suppress all print, even if the print originates in a
compiled C/Fortran sub-function.
This will not suppress raised exceptions, since exceptions are printed
to stderr just before a script exits, and after the context manager has
exited (at least, I think that is why it lets exceptions through).
'''
def __init__(self):
# Open a pair of null files
self.null_fds = [os.open(os.devnull,os.O_RDWR) for x in range(2)]
# Save the actual stdout (1) and stderr (2) file descriptors.
self.save_fds = [os.dup(1), os.dup(2)]
def __enter__(self):
# Assign the null pointers to stdout and stderr.
os.dup2(self.null_fds[0],1)
os.dup2(self.null_fds[1],2)
def __exit__(self, *_):
# Re-assign the real stdout/stderr back to (1) and (2)
os.dup2(self.save_fds[0],1)
os.dup2(self.save_fds[1],2)
# Close all file descriptors
for fd in self.null_fds + self.save_fds:
os.close(fd)
To use this you just:
with suppress_stdout_stderr():
rogue_function()
This works "pretty good". It does suppress the printout from the rogue functions that were cluttering up my script. I noticed in testing it that it lets through raised exceptions as well as some logger print, and I'm not entirely clear why. I think it has something to do with when these messages get sent to stdout / stderr (I think it happens after my context manager exits). If anyone can confirm this, I'd be interested in hearing the details ...
As of python 3.5 we can do this with minimal work using built-ins in contextlib, namely redirect_stdout and redirect_stderr. We only need to combine these two built-in context managers in a custom context manager of ours, which can be easily done using the nice pattern in Martijn's answer here. Redirecting both outputs to os.devnull should be safe and portable enough.
from contextlib import contextmanager,redirect_stderr,redirect_stdout
from os import devnull
#contextmanager
def suppress_stdout_stderr():
"""A context manager that redirects stdout and stderr to devnull"""
with open(devnull, 'w') as fnull:
with redirect_stderr(fnull) as err, redirect_stdout(fnull) as out:
yield (err, out)
Note that suppressing stderr will still give you full tracebacks when something breaks, which is a good thing:
import sys
def rogue_function():
print('spam to stdout')
print('important warning', file=sys.stderr)
1 + 'a'
return 42
with suppress_stdout_stderr():
rogue_function()
When run the above only prints
Traceback (most recent call last):
File "tmp.py", line 20, in <module>
rogue_function()
File "foo.py", line 16, in rogue_function
1 + 'a'
TypeError: unsupported operand type(s) for +: 'int' and 'str'
to the terminal. Unhandled exceptions should never go unnoticed.
This approach (found through the related sidebar) might work. It reassigns the file descriptors rather than just the wrappers to them in sys.stdout, etc.
python 3.6 working version, tested with million suppressions without any errors
import os
import sys
class suppress_stdout_stderr(object):
def __enter__(self):
self.outnull_file = open(os.devnull, 'w')
self.errnull_file = open(os.devnull, 'w')
self.old_stdout_fileno_undup = sys.stdout.fileno()
self.old_stderr_fileno_undup = sys.stderr.fileno()
self.old_stdout_fileno = os.dup ( sys.stdout.fileno() )
self.old_stderr_fileno = os.dup ( sys.stderr.fileno() )
self.old_stdout = sys.stdout
self.old_stderr = sys.stderr
os.dup2 ( self.outnull_file.fileno(), self.old_stdout_fileno_undup )
os.dup2 ( self.errnull_file.fileno(), self.old_stderr_fileno_undup )
sys.stdout = self.outnull_file
sys.stderr = self.errnull_file
return self
def __exit__(self, *_):
sys.stdout = self.old_stdout
sys.stderr = self.old_stderr
os.dup2 ( self.old_stdout_fileno, self.old_stdout_fileno_undup )
os.dup2 ( self.old_stderr_fileno, self.old_stderr_fileno_undup )
os.close ( self.old_stdout_fileno )
os.close ( self.old_stderr_fileno )
self.outnull_file.close()
self.errnull_file.close()
Did you try to redirect stderr too?
e.g.
sys.stdout = StringIO()
sys.stderr = StringIO()
foo(bar)
sys.stdout = sys.__stdout__ # These are provided by python
sys.stderr = sys.__stderr__
Also using StringIO might use extra memory. You can use a dummy device instead (e.g. http://coreygoldberg.blogspot.com/2009/05/python-redirect-or-turn-off-stdout-and.html).
My solution is similar to yours but uses contextlib and is a little shorter and easier to understand (IMHO).
import contextlib
#contextlib.contextmanager
def stdchannel_redirected(stdchannel, dest_filename):
"""
A context manager to temporarily redirect stdout or stderr
e.g.:
with stdchannel_redirected(sys.stderr, os.devnull):
if compiler.has_function('clock_gettime', libraries=['rt']):
libraries.append('rt')
"""
try:
oldstdchannel = os.dup(stdchannel.fileno())
dest_file = open(dest_filename, 'w')
os.dup2(dest_file.fileno(), stdchannel.fileno())
yield
finally:
if oldstdchannel is not None:
os.dup2(oldstdchannel, stdchannel.fileno())
if dest_file is not None:
dest_file.close()
The context for why I created this is at this blog post. Similar to yours I think.
I use it like this in a setup.py:
with stdchannel_redirected(sys.stderr, os.devnull):
if compiler.has_function('clock_gettime', libraries=['rt']):
libraries.append('rt')
Not really requested by the OP, but I needed to hide and store the output, and did like follows:
from io import StringIO
import sys
class Hider:
def __init__(self, channels=('stdout',)):
self._stomach = StringIO()
self._orig = {ch : None for ch in channels}
def __enter__(self):
for ch in self._orig:
self._orig[ch] = getattr(sys, ch)
setattr(sys, ch, self)
return self
def write(self, string):
self._stomach.write(string)
def flush(self):
pass
def autopsy(self):
return self._stomach.getvalue()
def __exit__(self, *args):
for ch in self._orig:
setattr(sys, ch, self._orig[ch])
Usage:
with Hider() as h:
spammy_function()
result = h.autopsy()
(tested only with Python 3)
EDIT: now allows to select stderr, stdout or both, as in Hider([stdout, stderr])
I use a decorator for this. It saves sys.stdout and sys.stderr references and makes these variables point to null. Then, after the function execution the original references are retrieved. It is important to note the try/except block, that allows the retrieval of the original references even when an exception is raised on the function.
def suppress_std(func):
def wrapper(*args, **kwargs):
stderr_tmp = sys.stderr
stdout_tmp = sys.stdout
null = open(os.devnull, 'w')
sys.stdout = null
sys.stderr = null
try:
result = func(*args, **kwargs)
sys.stderr = stderr_tmp
sys.stdout = stdout_tmp
return result
except:
sys.stderr = stderr_tmp
sys.stdout = stdout_tmp
raise
return wrapper
To use:
#suppress_std
def function_std_suppressed():
# code here
Just use Linux/Unix:
./myscript.py 2>/dev/null # gets rid of stderr
./myscript.py 2>/somewhere/myerror.log
If you are running this script on a linux based machine, you should be able to:
$> ./runscript.py > output.txt
Is it possible to temporarily redirect stdout/stderr in Python (i.e. for the duration of a method)?
Edit:
The problem with the current solutions (which I at first remembered but then forgot) is that they don't redirect; rather, they just replace the streams in their entirety. Hence, if a method has a local copy of one the variable for any reason (e.g. because the stream was passed as a parameter to something), it won't work.
Any solutions?
You can also put the redirection logic in a contextmanager.
import os
import sys
class RedirectStdStreams(object):
def __init__(self, stdout=None, stderr=None):
self._stdout = stdout or sys.stdout
self._stderr = stderr or sys.stderr
def __enter__(self):
self.old_stdout, self.old_stderr = sys.stdout, sys.stderr
self.old_stdout.flush(); self.old_stderr.flush()
sys.stdout, sys.stderr = self._stdout, self._stderr
def __exit__(self, exc_type, exc_value, traceback):
self._stdout.flush(); self._stderr.flush()
sys.stdout = self.old_stdout
sys.stderr = self.old_stderr
if __name__ == '__main__':
devnull = open(os.devnull, 'w')
print('Fubar')
with RedirectStdStreams(stdout=devnull, stderr=devnull):
print("You'll never see me")
print("I'm back!")
starting from python 3.4 there is the context manager contextlib.redirect_stdout:
from contextlib import redirect_stdout
with open('yourfile.txt', 'w') as f:
with redirect_stdout(f):
# do stuff...
to completely silence stdout this works:
from contextlib import redirect_stdout
with redirect_stdout(None):
# do stuff...
To solve the issue that some function might have cached sys.stdout stream as a local variable and therefore replacing the global sys.stdout won't work inside that function, you could redirect at a file descriptor level (sys.stdout.fileno()) e.g.:
from __future__ import print_function
import os
import sys
def some_function_with_cached_sys_stdout(stdout=sys.stdout):
print('cached stdout', file=stdout)
with stdout_redirected(to=os.devnull), merged_stderr_stdout():
print('stdout goes to devnull')
some_function_with_cached_sys_stdout()
print('stderr also goes to stdout that goes to devnull', file=sys.stderr)
print('stdout is back')
some_function_with_cached_sys_stdout()
print('stderr is back', file=sys.stderr)
stdout_redirected() redirects all output for sys.stdout.fileno() to a given filename, file object, or file descriptor (os.devnull in the example).
stdout_redirected() and merged_stderr_stdout() are defined here.
I am not sure what temporary redirection means. But, you can reassign streams like this and reset it back.
temp = sys.stdout
sys.stdout = sys.stderr
sys.stderr = temp
Also to write to sys.stderr within print stmts like this.
print >> sys.stderr, "Error in atexit._run_exitfuncs:"
Regular print will to stdout.
It's possible with a decorator such as the following:
import sys
def redirect_stderr_stdout(stderr=sys.stderr, stdout=sys.stdout):
def wrap(f):
def newf(*args, **kwargs):
old_stderr, old_stdout = sys.stderr, sys.stdout
sys.stderr = stderr
sys.stdout = stdout
try:
return f(*args, **kwargs)
finally:
sys.stderr, sys.stdout = old_stderr, old_stdout
return newf
return wrap
Use as:
#redirect_stderr_stdout(some_logging_stream, the_console):
def fun(...):
# whatever
or, if you don't want to modify the source for fun, call it directly as
redirect_stderr_stdout(some_logging_stream, the_console)(fun)
But note that this is not thread-safe.
Here's a context manager that I found useful. The nice things about this are that you can use it with the with statement and it also handles redirecting for child processes.
import contextlib
#contextlib.contextmanager
def stdchannel_redirected(stdchannel, dest_filename):
"""
A context manager to temporarily redirect stdout or stderr
e.g.:
with stdchannel_redirected(sys.stderr, os.devnull):
...
"""
try:
oldstdchannel = os.dup(stdchannel.fileno())
dest_file = open(dest_filename, 'w')
os.dup2(dest_file.fileno(), stdchannel.fileno())
yield
finally:
if oldstdchannel is not None:
os.dup2(oldstdchannel, stdchannel.fileno())
if dest_file is not None:
dest_file.close()
The context for why I created this is at this blog post.
Raymond Hettinger shows us a better way[1]:
import sys
with open(filepath + filename, "w") as f: #replace filepath & filename
with f as sys.stdout:
print("print this to file") #will be written to filename & -path
After the with block the sys.stdout will be reset
[1]: http://www.youtube.com/watch?v=OSGv2VnC0go&list=PLQZM27HgcgT-6D0w6arhnGdSHDcSmQ8r3
Look at contextlib.redirect_stdout(new_target) and contextlib.redirect_stderr(new_target). redirect_stderr is new in Python 3.5.
We'll use the PHP syntax of ob_start and ob_get_contents functions in python3, and redirect the input into a file.
The outputs are being stored in a file, any type of stream could be used as well.
from functools import partial
output_buffer = None
print_orig = print
def ob_start(fname="print.txt"):
global print
global output_buffer
print = partial(print_orig, file=output_buffer)
output_buffer = open(fname, 'w')
def ob_end():
global output_buffer
close(output_buffer)
print = print_orig
def ob_get_contents(fname="print.txt"):
return open(fname, 'r').read()
Usage:
print ("Hi John")
ob_start()
print ("Hi John")
ob_end()
print (ob_get_contents().replace("Hi", "Bye"))
Would print
Hi John
Bye John