I would like to create a function that keeps a record of every print command, storing each command's string into a new line in a file.
def log(line):
with open('file.txt', "a") as f:
f.write('\n' + line)
This is what I have, but is there any way to do what I said using Python?
Try replacing stdout with custom class:
import sys
class LoggedStdout():
def __init__(self, filename = None):
self.filename = filename
def write(self, text):
sys.__stdout__.write(text)
if not self.filename is None:
self.log(text)
def log(self, line):
with open(self.filename, "a") as f:
f.write('\n' + line)
sys.stdout = LoggedStdout('file.txt')
print 'Hello world!'
This would affect not only print, but also any other function that prints something to stdout, but it is often even better.
For production-mode logging it's much better to use something like logging module, rather than home-made hooks over standard IO streams.
Related
I created a python program, test.py, below:
import subprocess
import sys, os
FolderPath = subprocess.getoutput("cd . && pwd")
ProgramName = sys.argv[0]
LogName = ProgramName[:-3]+'_printout.txt'
ProgramFile = FolderPath+'/'+ProgramName
LogFile = FolderPath+'/'+LogName
_stdin = sys.stdin
_stdout = sys.stdout
_stderr = sys.stderr
sys.stdin = open(LogFile, 'w')
sys.stdout = open(LogFile, 'a')
sys.stderr = open(LogFile, 'a')
Prog = open(ProgramFile, 'r')
print(Prog.read())
TEST = str(input("Enter the name: \n TEST_NAME: "))
print(TEST)
sys.stdin = _stdin.flush()
sys.stdout = _stdout.flush()
sys.stderr = _stderr.flush()
After I executed on linux with command python test.py, I got the error in test_printout.txt.
Enter the name:
TEST_NAME: Traceback (most recent call last):
File "test.py", line 21, in <module>
TEST = str(input("Enter the name: \n TEST_NAME: "))
io.UnsupportedOperation: not readable
I modified the code:
import subprocess
import sys, os
FolderPath = subprocess.getoutput("cd . && pwd")
ProgramName = sys.argv[0]
LogName = ProgramName[:-3]+'_printout.txt'
ProgramFile = FolderPath+'/'+ProgramName
LogFile = FolderPath+'/'+LogName
_stdin = sys.stdin
_stdout = sys.stdout
_stderr = sys.stderr
sys.stdin = open(LogFile, 'w+')
sys.stdout = open(LogFile, 'a')
sys.stderr = open(LogFile, 'a')
Prog = open(ProgramFile, 'r')
print(Prog.read())
TEST = str(input("Enter the name: \n TEST_NAME: "))
print(TEST)
sys.stdin = _stdin.flush()
sys.stdout = _stdout.flush()
sys.stderr = _stderr.flush()
But got:
Enter the name:
TEST_NAME: import subprocess
It did not let me type anything. What I want is to let me type string and it also save to test_printout.txt.
Enter the name:
TEST_NAME: This Is What I Type And Save!
Does anyone know how to fix it?
Also, if I use w+ instead of w mode, it will take longer time to write to the test_printout.txt if I changed the program to import pandas.DataFrame and manipulate data.
Is there a way to only write simple print words to test_printout.txt without reading entire thing?
UPDATE
I modified the code as below:
import subprocess, sys, os
FolderPath = subprocess.getoutput("cd . && pwd")
ProgramName = sys.argv[0]
LogName = ProgramName[:-3]+'_printout.txt'
ProgramFile = FolderPath+'/'+ProgramName
LogFile = FolderPath+'/'+LogName
_stdin = sys.stdin
_stdout = sys.stdout
_stderr = sys.stderr
class stdout_Logger(object):
def __init__(self):
self.stdout = sys.stdout
self.log = open(LogFile, "a")
def write(self, message):
self.stdout.write(message)
self.log.write(message)
def flush(self):
#this flush method is needed for python 3 compatibility.
#this handles the flush command by doing nothing.
#you might want to specify some extra behavior here.
pass
sys.stdout = stdout_Logger()
class stderr_Logger(object):
def __init__(self):
self.stderr = sys.stderr
self.log = open("test_printout.txt", "a")
def write(self, message):
self.stderr.write(message)
self.log.write(message)
def flush(self):
#this flush method is needed for python 3 compatibility.
#this handles the flush command by doing nothing.
#you might want to specify some extra behavior here.
pass
sys.stderr = stderr_Logger()
Prog = open(ProgramFile, 'r')
print(Prog.read())
##START Program
TEST = str(input("Enter the name: \n TEST_NAME: "))
print(TEST)
#END Program
sys.stdin = _stdin.flush()
sys.stdout = _stdout.flush()
sys.stderr = _stderr.flush()
This got almost what I want. This also save my program to test_printout.txt at the top and do print(TEST) in the bottom.
However, it also prints all program to the linux terminal console which is not I desire. I only want it to print "Enter the name: \n TEST_NAME: " in linux terminal and I can type string instead of printing entire program.
I think the issue came from sys.stdin.
I think I figured it out. The problem is that when you substitute input with a file-handle in write mode you ban input() from reading it. You can get the same error if you tried this:
file = open("foo.txt",'w')
content = file.read()
The way to go around it is to log streams without redirecting them. So either you dump your console to file with python test.py > test_printout.txt or create a logger class to wrap around the streams (check out this answer: How to redirect stdout to both file and console with scripting?).
Perhaps its worth for you to look into the logging module, as I believe it handles these issues rather neatly.
EDIT:
From what you laid out in the comments, this is what you want:
import subprocess, sys, os
FolderPath = subprocess.getoutput("cd . && pwd")
ProgramName = sys.argv[0]
LogName = ProgramName[:-3]+'_printout.txt'
ProgramFile = FolderPath+'/'+ProgramName
LogFile = FolderPath+'/'+LogName
Prog = open(ProgramFile, 'r')
with open(LogFile, 'w') as logfile:
logfile.write(Prog.read())
_stdin = sys.stdin
_stdout = sys.stdout
_stderr = sys.stderr
class stdout_Logger(object):
def __init__(self):
self.stdout = sys.stdout
self.log = open(LogFile, "a")
def write(self, message):
self.stdout.write(message)
self.log.write(message)
def flush(self):
#this flush method is needed for python 3 compatibility.
#this handles the flush command by doing nothing.
#you might want to specify some extra behavior here.
pass
class stderr_Logger(object):
def __init__(self):
self.stderr = sys.stderr
self.log = open("test_printout.txt", "a")
def write(self, message):
self.stderr.write(message)
self.log.write(message)
def flush(self):
#this flush method is needed for python 3 compatibility.
#this handles the flush command by doing nothing.
#you might want to specify some extra behavior here.
pass
sys.stdout = stdout_Logger()
sys.stderr = stderr_Logger()
##START Program
TEST = str(input("Enter the name: \n TEST_NAME: "))
print(TEST)
#END Program
sys.stdin = _stdin.flush()
sys.stdout = _stdout.flush()
sys.stderr = _stderr.flush()
I want to log the script output to a file while still displaying the output to the screen.
It works fine, except for some cases where not all the content is written to the file (one or two lines can be missed, if the output is long)
Below is my code:
class Tee(object):
def __init__(self, *files):
self.files = files
def write(self, obj):
for f in self.files:
f.write(obj)
f.flush()
write_log = open("log.txt", 'a', 0)
sys.stdout = Tee(sys.stdout, write_log)
sys.stderr = Tee(sys.stderr, write_log)
Tried all the following options at the end of the code, but the result is the same:
os.fsync(write_log.fileno())
write_log.flush()
write_log.close()
Try using the with statement or use try-except and explicitly close the file.
I want to get the current console output of my program in python. There are a lot of solutions to get the console output when running an external program, however, I couldn't find any solution for getting the console output of the current program. Am I missing something? I am looking for a solution which works under windows and linux.
For example:
print "Hello world"
output = get_console_output() # Returns "Hello World\n"
Edit:
The solution should preserve the console output, so just replacing stdout won't work, as the console will be empty then
If you want to access the output you need to redirect the standard output stdout somewhere. You can use StringIO for this for example:
from cStringIO import StringIO
import sys
sys.stdout = buffer = StringIO()
print "Hello World"
# get output via: buffer.getvalue()
If you rather want the output to a file you could instead redirect directly to a file:
import sys
sys.stdout = open('output.txt', 'w')
print 'Hello World'
Edit: If you want output to be appended to log (according to comment), I suggest a custom class:
import sys
class Log(object):
def __init__(self):
self.orgstdout = sys.stdout
self.log = open("log.txt", "a")
def write(self, msg):
self.orgstdout.write(msg)
self.log.write(msg)
sys.stdout = Log()
print('Hello World')
You can overwrite sys.stdout with any file-like object:
import sys
import StringIO
sys.stdout = StringIO.StringIO()
You should also think about using the logging module instead of print. Or simply write a function that stores and prints values.
I always open and write into files using with statement:
with open('file_path', 'w') as handle:
print >>handle, my_stuff
However, there is one instance where I need to be able to be more flexible, and write to sys.stdout (or other types of streams), if that is provided instead of file path:
So, my question is this: Is there a way for using with statement both with real files and with sys.stdout?
Note that I can use the following code, but I think this defeats the purpose of using with:
if file_path != None:
outputHandle = open(file_path, 'w')
else:
outputHandle = sys.stdout
with outputHandle as handle:
print >>handle, my_stuff
You can create a context manager and use it like this
import contextlib, sys
#contextlib.contextmanager
def file_writer(file_name = None):
# Create writer object based on file_name
writer = open(file_name, "w") if file_name is not None else sys.stdout
# yield the writer object for the actual use
yield writer
# If it is file, then close the writer object
if file_name != None: writer.close()
with file_writer("Output.txt") as output:
print >>output, "Welcome"
with file_writer() as output:
print >>output, "Welcome"
If you don't pass any input to file_writer it will use sys.stdout.
Thing is, you don't need to use a context processor with stdout, because you're not opening or closing it. A less fancy way of abstracting this is:
def do_stuff(file):
# Your real code goes here. It works both with files or stdout
return file.readline()
def do_to_stdout():
return do_stuff(sys.stdout)
def do_to_file(filename):
with open(filename) as f:
return do_stuff(f)
print do_to_file(filename) if filename else do_to_stdout()
The simplest way is to simply use "old school" streamed filenames, that way your code doesn't have to change. In Unix this is "/dev/tty" or in Windows this is "con" (although there are other choices for both platforms).
if default_filename is None:
default_filename = "/dev/tty"
with open(default_filename, 'w') as handle:
handle.write("%s\n" % my_stuff)
This code tested in Python 2.7.3 and 3.3.5
With python3 optional closefd argument is recognized.
If set to False, resulting IO object won't close underlying fd:
if file_path != None:
outputHandle = open(file_path, 'w')
else:
outputHandle = open(sys.stdout.fileno(), 'w', closefd=False)
with outputHandle as handle:
print(my_stuff, file=handle)
I have a script that accepts as an argument a filename than opens it and writes some stuff.
I use the with statement:
with open(file_name, 'w') as out_file:
...
out_file.write(...)
Now what if I want to write to sys.stdout if no file_name is provided?
Do I necessarily need to wrap all actions in a function and put a condition before?
if file_name is None:
do_everything(sys.stdout)
else:
with open(file_name, 'w') as out_file:
do_everything(out_file)
from contextlib import contextmanager
#contextmanager
def file_or_stdout(file_name):
if file_name is None:
yield sys.stdout
else:
with open(file_name, 'w') as out_file:
yield out_file
Then you can do
with file_or_stdout(file_name) as wfile:
do_stuff_writing_to(wfile)
How do you handle command line arguments? If you use argparse you could use the type and default parameters of add_argument to handle this. For example, try something like the following:
import sys
import argparse
def main(argv=None):
if argv is None:
argv=sys.argv[1:]
parser = argparse.ArgumentParser()
parser.add_argument('infile', nargs='?',
type=argparse.FileType('w'),
default=sys.stdin)
args = parser.parse_args(argv)
print args.infile
return 0
if __name__=="__main__":
sys.exit(main(sys.argv[1:]))
If a file name is present as an argument the the script argparse will automatically open and close this file and args.infile will be a handle to this file. Otherwise args.infile will simply be sys.stdin.
You could write your own context manager. I'll post sample code later if noone else does
if file_name is None:
fd = sys.stdout
else:
fd = open(file_name, 'w')
# write to fd
if fd != sys.stdout:
fd.close();
Using the with ... as construct is useful to close the file automatically. This means that using it with sys.stdout, as I guess you know, would crash your program, because it would attempt at closing the system stdout!
This means something like with open(name, 'w') if name else sys.stdout as: would not work.
This make me say there isn't any simple-nice way to write your snippet better... but there are probably better ways to think on how to construct such a code!
The main point to clarify is when you need to open (and, more importantly, close) the filehandler for file_name, when file_name exists.
Personally I would simply drop the with .. as and take care of opening the file - and, more importantly, close it! - somewhere else. Mileage for that might vary depending on how your software is working.
This means you can simply do:
out_file = open(file_name, 'w') if file_name else sys.stdout
and work with out_file throughout your program.
When you close, remember to check if it's a file or not :)
And have you thought about simply using the logging module? That easily allows you to add different handlers, print to file, print to stdout...