I am currently running a script using easygui to receive the user input. The old script that ran in a command line would just print anything the user needed to know in the command line, but I've changed it to output notifications in new easygui boxes when there are required inputs.
Want I want to do is have the progress, each action inside the functions that are running, print into a text box as they complete. In command line I could just use print "text" but I can't get it to happen realtime in easygui. Currently I'm appending a list so I have a textbox that displays the results of the function once everything is complete, but I'd like for the large textbox window to pop up and print the line out whenever a process of note completes. Is this doable?
Here is how I'm appending the list:
result_list = []
record_str = "\n Polling has completed for 502."
result_list.append(record_str)
eg.textbox("Polling Status", "Daily polling completion status:", result_list)
I don't think there's any simple way of getting EasyGUI's textbox function to do what you want short of modifying the module. Since it's a function not a class, you can't even derive a subclass from it in order to easily reuse its code.
However it's completely feasible to create a separate Tkinter window that just displays lines of text as they are sent to it using an enhanced version of some code I found once in a thread on the comp.lang.python newsgroup.
The original code was designed to catch and display only stderr output from a GUI application which normally doesn't have stderr output handle, so the module was named errorwindow. However I modified it to be able redirect both stderr and stdout to such windows in one easygui-based app I developed, but I never got around to renaming it or updating the comments in it to also mention stdout redirection. ;¬)
Anyway, the module works by defining and creating two instances of a file-like class named OutputPipe when it's imported and assigns them to the sys.stdout and sys.stderr I/O stream file objects which are usually None in a Python .pyw GUI applications (on Windows). When output is first sent to either one of these, the same module is launched as separate Python process with its stdin, stdout, and stderr I/O handles piped with the original process.
There's a lot going on, but if nothing else, with a little study of it might give you some ideas about how to get easygui's textbox to do what you want. Hope this helps.
Note: The code posted is for Python 2.x, there's a modified version that will work in both Python 2 and 3 in my answer to another question, if anyone is interested.
File errorwindow.py:
# Code derived from Bryan Olson's source posted in this related Usenet discussion:
# https://groups.google.com/d/msg/comp.lang.python/HWPhLhXKUos/TpFeWxEE9nsJ
# https://groups.google.com/d/msg/comp.lang.python/HWPhLhXKUos/eEHYAl4dH9YJ
#
# Here's a module to show stderr output from console-less Python
# apps, and stay out of the way otherwise. I plan to make a ASPN
# recipe of it, but I thought I'd run it by this group first.
#
# To use it, import the module. That's it. Upon import it will
# assign sys.stderr.
#
# In the normal case, your code is perfect so nothing ever gets
# written to stderr, and the module won't do much of anything.
# Upon the first write to stderr, if any, the module will launch a
# new process, and that process will show the stderr output in a
# window. The window will live until dismissed; I hate, hate, hate
# those vanishing-consoles-with-critical-information.
#
# The code shows some arguably-cool tricks. To fit everthing in
# one file, the module runs the Python interpreter on itself; it
# uses the "if __name__ == '__main__'" idiom to behave radically
# differently upon import versus direct execution. It uses TkInter
# for the window, but that's in a new process; it does not import
# TkInter into your application.
#
# To try it out, save it to a file -- I call it "errorwindow.py" -
# - and import it into some subsequently-incorrect code. For
# example:
#
# import errorwindow
#
# a = 3 + 1 + nonesuchdefined
#
# should cause a window to appear, showing the traceback of a
# Python NameError.
#
# --
# --Bryan
# ----------------------------------------------------------------
#
# martineau - Modified to use subprocess.Popen instead of the os.popen
# which has been deprecated since Py 2.6. Changed so it
# redirects both stdout and stderr. Added numerous
# comments, and also inserted double quotes around paths
# in case they have embedded space characters in them, as
# they did on my Windows system.
"""
Import this module into graphical Python apps to provide a
sys.stderr. No functions to call, just import it. It uses
only facilities in the Python standard distribution.
If nothing is ever written to stderr, then the module just
sits there and stays out of your face. Upon write to stderr,
it launches a new process, piping it error stream. The new
process throws up a window showing the error messages.
"""
import subprocess
import sys
import thread
import os
if __name__ == '__main__': # when spawned as separate process
# create window in which to display output
# then copy stdin to the window until EOF
# will happen when output is sent to each OutputPipe created
from Tkinter import BOTH, END, Frame, Text, TOP, YES
import tkFont
import Queue
queue = Queue.Queue(100)
def read_stdin(app, bufsize=4096):
fd = sys.stdin.fileno() # gets file descriptor
read = os.read
put = queue.put
while True:
put(read(fd, bufsize))
class Application(Frame):
def __init__(self, master=None, font_size=8, text_color='#0000AA', rows=25, cols=100):
Frame.__init__(self, master)
# argv[0]: name of this script (not used)
# argv[1]: name of script that imported this module
# argv[2]: name of redirected stream (optional)
if len(sys.argv) < 3:
title = "Output Stream from %s" % (sys.argv[1],)
else:
title = "Output Stream '%s' from %s" % (sys.argv[2], sys.argv[1])
self.master.title(title)
self.pack(fill=BOTH, expand=YES)
font = tkFont.Font(family='Courier', size=font_size)
width = font.measure(' '*(cols+1))
height = font.metrics('linespace')*(rows+1)
self.configure(width=width, height=height)
self.pack_propagate(0) # force frame to be configured size
self.logwidget = Text(self, font=font)
self.logwidget.pack(side=TOP, fill=BOTH, expand=YES)
# Disallow key entry, but allow copy with <Control-c>
self.logwidget.bind('<Key>', lambda x: 'break')
self.logwidget.bind('<Control-c>', lambda x: None)
self.logwidget.configure(foreground=text_color)
#self.logwidget.insert(END, '==== Start of Output Stream ====\n\n')
#self.logwidget.see(END)
self.after(200, self.start_thread, ())
def start_thread(self, _):
thread.start_new_thread(read_stdin, (self,))
self.after(200, self.check_q, ())
def check_q(self, _):
log = self.logwidget
log_insert = log.insert
log_see = log.see
queue_get_nowait = queue.get_nowait
go = True
while go:
try:
data = queue_get_nowait()
if not data:
data = '[EOF]'
go = False
log_insert(END, data)
log_see(END)
except Queue.Empty:
self.after(200, self.check_q, ())
go = False
app = Application()
app.mainloop()
else: # when module is first imported
import traceback
class OutputPipe(object):
def __init__(self, name=''):
self.lock = thread.allocate_lock()
self.name = name
def __getattr__(self, attr):
if attr == 'pipe': # pipe attribute hasn't been created yet
# launch this module as a separate process to display any output
# it receives.
# Note: It's important to put double quotes around everything in case
# they have embedded space characters.
command = '"%s" "%s" "%s" "%s"' % (sys.executable, # command
__file__, # argv[0]
os.path.basename(sys.argv[0]), # argv[1]
self.name) # argv[2]
# sample command and arg values on receiving end:
# E:\Program Files\Python\python[w].exe # command
# H:\PythonLib\TestScripts\PyRemindWrk\errorwindow.py # argv[0]
# errorwindow.py # argv[1]
# stderr # argv[2]
# execute this script as __main__ with a stdin PIPE for sending output to it
try:
# had to make stdout and stderr PIPEs too, to make it work with pythonw.exe
self.pipe = subprocess.Popen(command, bufsize=0,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE).stdin
except Exception:
# output exception info to a file since this module isn't working
exc_type, exc_value, exc_traceback = sys.exc_info()
msg = ('%r exception in %s\n' %
(exc_type.__name__, os.path.basename(__file__)))
with open('exc_info.txt', 'wt') as info:
info.write('msg:' + msg)
traceback.print_exc(file=info)
sys.exit('fatal error occurred spawning output process')
return super(OutputPipe, self).__getattribute__(attr)
def write(self, data):
with self.lock:
self.pipe.write(data) # 1st reference to pipe attr will cause it to be created
# redirect standard output streams in the process importing the module
sys.stderr = OutputPipe('stderr')
sys.stdout = OutputPipe('stdout')
Related
I have a module that redirects console outputs to the Tkinter window, which I found here on StackOverflow. It works perfectly, however, I realized I also need to save the console output to a file. But everything I tried either doesn't work, breaks the module, or nothing happens.
EDIT: I need to modify the following module so it also SAVES THE CONSOLE OUTPUT to the .txt document. BUT preserving it's current functionality
Here is the code for the module:
# Code derived from Bryan Olson's source posted in this related Usenet discussion:
# https://groups.google.com/d/msg/comp.lang.python/HWPhLhXKUos/TpFeWxEE9nsJ
# https://groups.google.com/d/msg/comp.lang.python/HWPhLhXKUos/eEHYAl4dH9YJ
#
# See the comments and doc string below.
#
# Here's a module to show stderr output from console-less Python
# apps, and stay out of the way otherwise. I plan to make a ASPN
# recipe of it, but I thought I'd run it by this group first.
#
# To use it, import the module. That's it. Upon import it will
# assign sys.stderr.
#
# In the normal case, your code is perfect so nothing ever gets
# written to stderr, and the module won't do much of anything.
# Upon the first write to stderr, if any, the module will launch a
# new process, and that process will show the stderr output in a
# window. The window will live until dismissed; I hate, hate, hate
# those vanishing-consoles-with-critical-information.
#
# The code shows some arguably-cool tricks. To fit everthing in
# one file, the module runs the Python interpreter on itself; it
# uses the "if __name__ == '__main__'" idiom to behave radically
# differently upon import versus direct execution. It uses tkinter
# for the window, but that's in a new process; it does not import
# tkinter into your application.
#
# To try it out, save it to a file -- I call it "errorwindow.py" -
# - and import it into some subsequently-incorrect code. For
# example:
#
# import errorwindow
#
# a = 3 + 1 + nonesuchdefined
#
# should cause a window to appear, showing the traceback of a
# Python NameError.
#
# --
# --Bryan
# ----------------------------------------------------------------
#
# martineau - Modified to use subprocess.Popen instead of the os.popen
# which has been deprecated since Py 2.6. Changed so it
# redirects both stdout and stderr. Added numerous
# comments, and also inserted double quotes around paths
# in case they have embedded space characters in them, as
# they did on my Windows system.
#
# Recently updated it to work in both Python 2 and Python 3.
"""
Import this module into graphical Python apps to provide a
sys.stderr. No functions to call, just import it. It uses
only facilities in the Python standard distribution.
If nothing is ever written to stderr, then the module just
sits there and stays out of your face. Upon write to stderr,
it launches a new process, piping it error stream. The new
process throws up a window showing the error messages.
"""
import subprocess
import sys
try:
import thread
except ModuleNotFoundError: # Python 3
import _thread as thread
import os
EXC_INFO_FILENAME = 'exc_info.txt'
if __name__ == '__main__': # When spawned as separate process.
# create window in which to display output
# then copy stdin to the window until EOF
# will happen when output is sent to each OutputPipe created
try:
from Tkinter import BOTH, END, Frame, Text, TOP, YES
import tkFont
import Queue
except ModuleNotFoundError: # Python 3
from tkinter import BOTH, END, Frame, Text, TOP, YES
import tkinter.font as tkFont
import queue as Queue
Q_EMPTY = Queue.Empty # An exception class.
queue = Queue.Queue(1000) # FIFO
def read_stdin(app, bufsize=4096):
fd = sys.stdin.fileno() # File descriptor for os.read() calls.
read = os.read
put = queue.put
while True:
put(read(fd, bufsize))
class Application(Frame):
def __init__(self, master=None, font_size=8, text_color='#0000AA', rows=25, cols=100):
Frame.__init__(self, master)
# Create title based on the arguments passed to the spawned script:
# argv[0]: name of this script (ignored)
# argv[1]: name of script that imported this module
# argv[2]: name of redirected stream (optional)
if len(sys.argv) < 2:
title = "Output stream from unknown source"
elif len(sys.argv) < 3:
title = "Output stream from %s" % (sys.argv[1],)
else: # Assume it's a least 3.
title = "Output stream '%s' from %s" % (sys.argv[2], sys.argv[1])
self.master.title(title)
self.pack(fill=BOTH, expand=YES)
font = tkFont.Font(family='Courier', size=font_size)
width = font.measure(' ' * (cols+1))
height = font.metrics('linespace') * (rows+1)
self.configure(width=width, height=height)
self.pack_propagate(0) # Force frame to be configured size.
self.logwidget = Text(self, font=font)
self.logwidget.pack(side=TOP, fill=BOTH, expand=YES)
# Disallow key entry, but allow text copying with <Control-c>
self.logwidget.bind('<Key>', lambda x: 'break')
self.logwidget.bind('<Control-c>', lambda x: None)
self.logwidget.configure(foreground=text_color)
self.logwidget.insert(END, '==== Start of Output Stream ====\n\n')
self.logwidget.see(END)
self.after(200, self.start_thread) # Start queue polling thread.
def start_thread(self):
thread.start_new_thread(read_stdin, (self,))
self.after(200, self.check_q)
def check_q(self):
log = self.logwidget
log_insert = log.insert
log_see = log.see
queue_get_nowait = queue.get_nowait
go = True
while go:
try:
data = queue_get_nowait().decode() # Must decode for Python 3.
if not data:
data = '[EOF]'
go = False
log_insert(END, data)
log_see(END)
except Q_EMPTY:
self.after(200, self.check_q)
go = False
app = Application()
app.mainloop()
else: # when module is first imported
import traceback
class OutputPipe(object):
def __init__(self, name=''):
self.lock = thread.allocate_lock()
self.name = name
def flush(self): # no-op.
pass
def __getattr__(self, attr):
if attr == 'pipe': # Attribute doesn't exist, so create it.
# Launch this module as a separate process to display any output
# it receives.
# Note: It's important to put double quotes around everything in
# case any have embedded space characters.
command = '"%s" "%s" "%s" "%s"' % (sys.executable, # executable
__file__, # argv[0]
os.path.basename(sys.argv[0]), # argv[1]
self.name) # argv[2]
#
# Typical command and arg values on receiving end:
# C:\Python3\python[w].exe # executable
# C:\vols\Files\PythonLib\Stack Overflow\errorwindow3k.py # argv[0]
# errorwindow3k_test.py # argv[1]
# stderr # argv[2]
# Execute this script directly as __main__ with a stdin PIPE for sending
# output to it.
try:
# Had to also make stdout and stderr PIPEs too, to work with pythonw.exe
self.pipe = subprocess.Popen(command, bufsize=0,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE).stdin
except Exception:
# Output exception info to a file since this module isn't working.
exc_type, exc_value, exc_traceback = sys.exc_info()
msg = ('%r exception in %s\n' %
(exc_type.__name__, os.path.basename(__file__)))
with open(EXC_INFO_FILENAME, 'wt') as info:
info.write('fatal error occurred spawning output process')
info.write('exeception info:' + msg)
traceback.print_exc(file=info)
sys.exit('fatal error occurred')
return super(OutputPipe, self).__getattribute__(attr)
def write(self, data):
with self.lock:
data = data.encode() # Must encode for Python 3.
self.pipe.write(data) # First reference to pipe attr will cause an
# OutputPipe process for the stream to be created.
# Clean-up any left-over debugging file.
try:
os.remove(EXC_INFO_FILENAME) # Delete previous file, if any.
except Exception:
pass
# Redirect standard output streams in the process that imported this module.
sys.stderr = OutputPipe('stderr')
sys.stdout = OutputPipe('stdout')
I tried inserting this code after, before, and in between the module, but it didn't work or broke the module:
import sys path = 'output.txt' sys.stdout = open(path, 'w')
I also tried to do something like this, but it didn't work either.
f = open("output.txt", "w")
f.write(sys.stdout) # or "f.write(OutputPipe('stdout')" or " 'f.write(data)' between - 'def write(self, data):' "
f.close()
try:
sys.stdout = open('file.txt', 'w')
# some working code
finally:
# close file.txt
sys.stdout.close()
sys.stdout = sys.__stdout__
Update. You can write your custom logging class to suit your needs
import sys
class Logger:
def __init__(self, filename):
self.console = sys.stdout
self.file = open(filename, 'w')
def write(self, message):
self.console.write(message)
self.file.write(message)
def flush(self):
self.console.flush()
self.file.flush()
sys.stdout = Logger('file.txt')
I managed to solve my issue by adding a few lines to the code. I'll share here the full modified module code in the unlikely case that someone stumbles upon this same issue.
# Code derived from Bryan Olson's source posted in this related Usenet discussion:
# https://groups.google.com/d/msg/comp.lang.python/HWPhLhXKUos/TpFeWxEE9nsJ
# https://groups.google.com/d/msg/comp.lang.python/HWPhLhXKUos/eEHYAl4dH9YJ
#
# See the comments and doc string below.
#
# Here's a module to show stderr output from console-less Python
# apps, and stay out of the way otherwise. I plan to make a ASPN
# recipe of it, but I thought I'd run it by this group first.
#
# To use it, import the module. That's it. Upon import it will
# assign sys.stderr.
#
# In the normal case, your code is perfect so nothing ever gets
# written to stderr, and the module won't do much of anything.
# Upon the first write to stderr, if any, the module will launch a
# new process, and that process will show the stderr output in a
# window. The window will live until dismissed; I hate, hate, hate
# those vanishing-consoles-with-critical-information.
#
# The code shows some arguably-cool tricks. To fit everthing in
# one file, the module runs the Python interpreter on itself; it
# uses the "if __name__ == '__main__'" idiom to behave radically
# differently upon import versus direct execution. It uses tkinter
# for the window, but that's in a new process; it does not import
# tkinter into your application.
#
# To try it out, save it to a file -- I call it "errorwindow.py" -
# - and import it into some subsequently-incorrect code. For
# example:
#
# import errorwindow
#
# a = 3 + 1 + nonesuchdefined
#
# should cause a window to appear, showing the traceback of a
# Python NameError.
#
# --
# --Bryan
# ----------------------------------------------------------------
#
# martineau - Modified to use subprocess.Popen instead of the os.popen
# which has been deprecated since Py 2.6. Changed so it
# redirects both stdout and stderr. Added numerous
# comments, and also inserted double quotes around paths
# in case they have embedded space characters in them, as
# they did on my Windows system.
#
# Recently updated it to work in both Python 2 and Python 3.
"""
Import this module into graphical Python apps to provide a
sys.stderr. No functions to call, just import it. It uses
only facilities in the Python standard distribution.
If nothing is ever written to stderr, then the module just
sits there and stays out of your face. Upon write to stderr,
it launches a new process, piping it error stream. The new
process throws up a window showing the error messages.
"""
import subprocess
import sys
try:
import thread
except ModuleNotFoundError: # Python 3
import _thread as thread
import os
EXC_INFO_FILENAME = 'exc_info.txt'
if __name__ == '__main__': # When spawned as separate process.
# create window in which to display output
# then copy stdin to the window until EOF
# will happen when output is sent to each OutputPipe created
try:
from Tkinter import BOTH, END, Frame, Text, TOP, YES
import tkFont
import Queue
except ModuleNotFoundError: # Python 3
from tkinter import BOTH, END, Frame, Text, TOP, YES
import tkinter.font as tkFont
import queue as Queue
Q_EMPTY = Queue.Empty # An exception class.
queue = Queue.Queue(1000) # FIFO
def read_stdin(app, bufsize=4096):
fd = sys.stdin.fileno() # File descriptor for os.read() calls.
read = os.read
put = queue.put
while True:
put(read(fd, bufsize))
class Application(Frame):
def __init__(self, master=None, font_size=8, text_color='#0000AA', rows=25, cols=100):
Frame.__init__(self, master)
# Create title based on the arguments passed to the spawned script:
# argv[0]: name of this script (ignored)
# argv[1]: name of script that imported this module
# argv[2]: name of redirected stream (optional)
if len(sys.argv) < 2:
title = "Output stream from unknown source"
elif len(sys.argv) < 3:
title = "Output stream from %s" % (sys.argv[1],)
else: # Assume it's a least 3.
title = "Output stream '%s' from %s" % (sys.argv[2], sys.argv[1])
self.master.title(title)
self.pack(fill=BOTH, expand=YES)
font = tkFont.Font(family='Courier', size=font_size)
width = font.measure(' ' * (cols+1))
height = font.metrics('linespace') * (rows+1)
self.configure(width=width, height=height)
self.pack_propagate(0) # Force frame to be configured size.
self.logwidget = Text(self, font=font)
self.logwidget.pack(side=TOP, fill=BOTH, expand=YES)
# Disallow key entry, but allow text copying with <Control-c>
self.logwidget.bind('<Key>', lambda x: 'break')
self.logwidget.bind('<Control-c>', lambda x: None)
self.logwidget.configure(foreground=text_color)
self.logwidget.insert(END, '==== Start of Output Stream ====\n\n')
# Writes the console output to file!!! :) MOD
with open("Output.txt", "a") as f:
f.write('==== Start of Output Stream ====\n\n')
self.logwidget.see(END)
self.after(200, self.start_thread) # Start queue polling thread.
def start_thread(self):
thread.start_new_thread(read_stdin, (self,))
self.after(200, self.check_q)
def check_q(self):
log = self.logwidget
log_insert = log.insert
log_see = log.see
queue_get_nowait = queue.get_nowait
go = True
while go:
try:
data = queue_get_nowait().decode() # Must decode for Python 3.
with open("Output.txt", "a") as f: # Writes the console output to file!!! :) MOD
f.write(str(data))
if not data:
data = '[EOF]'
go = False
with open("Output.txt", "a") as f: # Writes the '[EOF]' to file!!! :) MOD
f.write(str('\n' + '============ [EOF] =============' + '\n' + '\n' + '\n'))
log_insert(END, data)
log_see(END)
except Q_EMPTY:
self.after(200, self.check_q)
go = False
app = Application()
app.mainloop()
else: # when module is first imported
import traceback
class OutputPipe(object):
def __init__(self, name=''):
self.lock = thread.allocate_lock()
self.name = name
def flush(self): # no-op.
pass
def __getattr__(self, attr):
if attr == 'pipe': # Attribute doesn't exist, so create it.
# Launch this module as a separate process to display any output
# it receives.
# Note: It's important to put double quotes around everything in
# case any have embedded space characters.
command = '"%s" "%s" "%s" "%s"' % (sys.executable, # executable
__file__, # argv[0]
os.path.basename(sys.argv[0]), # argv[1]
self.name) # argv[2]
#
# Typical command and arg values on receiving end:
# C:\Python3\python[w].exe # executable
# C:\vols\Files\PythonLib\Stack Overflow\errorwindow3k.py # argv[0]
# errorwindow3k_test.py # argv[1]
# stderr # argv[2]
# Execute this script directly as __main__ with a stdin PIPE for sending
# output to it.
try:
# Had to also make stdout and stderr PIPEs too, to work with pythonw.exe
self.pipe = subprocess.Popen(command, bufsize=0,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE).stdin
except Exception:
# Output exception info to a file since this module isn't working.
exc_type, exc_value, exc_traceback = sys.exc_info()
msg = ('%r exception in %s\n' %
(exc_type.__name__, os.path.basename(__file__)))
with open(EXC_INFO_FILENAME, 'wt') as info:
info.write('fatal error occurred spawning output process')
info.write('exeception info:' + msg)
traceback.print_exc(file=info)
sys.exit('fatal error occurred')
return super(OutputPipe, self).__getattribute__(attr)
def write(self, data):
with self.lock:
data = data.encode() # Must encode for Python 3.
self.pipe.write(data) # First reference to pipe attr will cause an
# OutputPipe process for the stream to be created.
# Clean-up any left-over debugging file.
try:
os.remove(EXC_INFO_FILENAME) # Delete previous file, if any.
except Exception:
pass
# Redirect standard output streams in the process that imported this module.
sys.stderr = OutputPipe('stderr')
sys.stdout = OutputPipe('stdout')
I'm writing a Python app that runs a command on an AWS remote docker container, and saves the output to a file. The command that is being run remotely is generating binary data (a database dump).
The app works great if I start the download and don't touch anything. The issue I'm having is that if I start the download, and hit Enter while it's downloading, or scroll my mouse wheel in the terminal window, my output file gets a ^M, or weird characters.
Sample Code:
#!/usr/bin/env python3
import npyscreen
import curses
import subprocess
MY_REGION=...
MY_CLUSTER=...
MY_TASK=...
MY_CONTAINER=...
class ProgressForm(npyscreen.Popup):
def create(self):
self.progress = self.add(
npyscreen.TitleSliderPercent, step=1, out_of=100, name="Progress"
)
def activate(self):
cmd = subprocess.Popen(
[
"aws",
"--region",
MY_REGION,
"ecs",
"execute-command",
"--cluster",
MY_CLUSTER,
"--task",
MY_TASK,
"--container",
MY_CONTAINER,
"--command",
"python -c 'for i in range(500_000): print(i)'",
"--interactive",
],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
bufsize=0,
)
total_size = 3889129
downloaded = 0
with open("out.log", "wb") as f:
while True:
chunk = cmd.stdout.read(1024)
if not chunk:
break
f.write(chunk)
downloaded += len(chunk)
self.progress.set_value(min(downloaded/total_size*100, 100))
self.progress.display()
self.parentApp.switchForm(None)
class MAIN(npyscreen.FormBaseNew):
def create(self):
self.items = self.add(
npyscreen.GridColTitles,
col_titles=["Column"],
select_whole_line=True,
)
self.items.add_handlers({curses.ascii.NL: self.item_chosen})
def activate(self):
for i in range(4):
self.items.values = [
["Row Data"]
]
self.edit()
def item_chosen(self, inpt):
self.parentApp.switchForm("progressForm")
class App(npyscreen.NPSAppManaged):
def onStart(self):
self.addForm("MAIN", MAIN, name="My App")
self.addForm("progressForm", ProgressForm)
if __name__ == "__main__":
app = App().run()
Hitting Enter during the download, or scrolling the mouse wheel results in this:
...
10667
10668
10669
...
and this:
...
17451
17452
17453
^[[<65;121;31M17454
17455
17456
17457
...
Why is my subprocess' stdout being littered with junk data?
Edit: The full output can be found here
When you don't specify what subprocess should do with stdin, it gets inherited from the parent process, letting the child see your enter keys, scroll-wheel data, etc.
A typical noninteractive process won't do "local echo" of input back to output; but you're using --interactive here, so the behavior is not surprising.
Set stdin=subprocess.DEVNULL to explicitly route stdin from nowhere (stdin connected to /dev/null shows up as an immediate EOF on the first attempted read; most programs that aren't written to require input will handle this correctly).
If the program requires there to be a stdin stream that isn't immediately closed, you might instead use stdin=subprocess.PIPE, and then leave cmd.stdin alone until it's time for the remote program to exit (at which point a cmd.stdin.close(), while not strictly mandatory, would not be remiss).
In python with gtk library used for creating windows, how can I redirect stdout in real time to a specified textview buffer?
self.textbuffer.set_text(sys.stdout)
I see that subprocess allows that, but it needs another process to run, I want to capture the output of the print commands already present in the rest of the program (current process not another process)
regards,
Something like:
import sys
class MyStdout:
def __init__(self, buffer):
self.buffer = buffer
def write(self, string):
if string:
# Append self.buffer
sys.stdout = MyStdout(self.textbuffer)
Basically, I am asking how to put a continuously updating program on display into tkinter's text widget.
from tkinter import Tk, Frame, Text, BOTH
class FrameApp(Frame):
def __init__(self, parent):
Frame.__init__(self, parent, background="white")
self.parent = parent
self.parent.title("Ethis")
self.pack(fill=BOTH, expand=1)
self.centerWindow()
def centerWindow(self):
w = 900
h = 450
sw = self.parent.winfo_screenwidth()
sh = self.parent.winfo_screenheight()
x = (sw - w)/2
y = (sh - h)/2
self.parent.geometry("%dx%d+%d+%d" % (w, h, x, y))
def theText(self):
w = Text ()
def main():
root=Tk()
app = FrameApp(root)
root.mainloop()
if __name__ == '__main__':
main()
This is my tkinter program. As you can see, I've centered it and set it up with a text function defined as theText(self). I have done anything with theText(self) because I don't know where to begin. This works fine alone, it starts up, as expected, in the center with it's title.
# Money Generator Mark 1
import time
t = 'true'
while t == 'true':
s = 0
x = 1
print ("You have $%s." % (s))
time.sleep(.75)
t = 'false'
while t == 'false':
s = s + (1 * x)
print ("You have $%s." % (s))
time.sleep(.75)
if s >= 100 and s < 200:
x = 2
if s >= 200:
x = 4
Here I have another program that works fine on it's own. I've dubbed it Money Generator as is akin to Cookie Clicker and Candy Box, those types of things. This also works fine in the command box, function and printing to there. I was wondering how to integrate these two separate programs so that the second program that is listed here will be displayed in tkinter's window.
Here is my new code that has a new issue. I'm receiving an error stating that 'generate_money' is not defined in the theText function. These new functions are within my frameApp class.
def theText(self):
self.w = Text()
self.t = threading.Thread(target=generate_money, args=(self.w))
self.t.daemon = True
def generate_money(textwidget):
p = subprocess.Popen([sys.executable, os.path.join('window.py', 'moneygenerator.py')],
stdout = subprocess.PIPE)
for line in p.stdout:
do_stuff_with(textwidget, line)
p.close()
This is unfortunately going to be a little trickier than you'd like.
The first part is easy: you can just use the subprocess module to run the background script and capture its output:
p = subprocess.Popen([sys.executable, os.path.join(scriptpath, 'moneygenerator.py')],
stdout = subprocess.PIPE)
for line in p.stdout:
do_stuff_with(line)
p.close()
The problem is that doing this in the middle of a Tkinter callback will block the entire program until the background program is done. So, instead of getting updated on each line, you'll just freeze up the app until the OS kills you/displays a beachball/etc.
There are two usual solutions to doing stuff without blocking: Do it on a thread—which is easy, except that you can't access Tkinter widgets from a background thread. Or check the subprocess's pipe in a non-blocking way—which would be great if there were a cross-platform way to do that without potentially blocking forever. (There are, however, third-party "async subprocess" wrappers on PyPI, which may be an alternate way to solve this problem.)
So, you need to combine both of these. Have a thread that blocks on the subprocess and posts messages on something that you can check in a non-blocking way in the main thread, like a queue.Queue.
But, instead of writing it yourself, there's a nice wrapper called mtTkinter that does most of the hard part for you. You can just write your background thread as if it were legal to access the widgets, and it will intercept that access and turn it into queue posts. Sadly, mtTkinter doesn't work in Python 3—but it looks pretty easy to fix; I slapped together a port in a few minutes. Although I haven't done much testing on it, I think this is probably the simplest way forward.
So:
from tkinter import Tk, Frame, Text, BOTH
import subprocess
import sys
import threading
# ...
def generate_money(textwidget):
p = subprocess.Popen([sys.executable, os.path.join(scriptpath, 'moneygenerator.py')],
stdout = subprocess.PIPE)
for line in p.stdout:
do_stuff_with(textwidget, line)
p.close()
# ...
class FrameApp(Frame):
# ...
def theText(self):
self.w = Text()
self.t = threading.Thread(target=generate_money, args=(self.w,))
You'll probably want to add some way to either tell self.t to shut down early, or just wait for it to finish, at quit time. (Or, alternatively, because you know that it can't leave any "dangerous garbage" behind it killed suddenly, you may be able to just set self.t.daemon = True.) But this is enough to show the basics.
I am trying to capture a single frame from the Apple iSight camera built into a Macbook Pro using Python (version 2.7 or 2.6) and the PyObjC (version 2.2).
As a starting point, I used this old StackOverflow question. To verify that it makes sense, I cross-referenced against Apple's MyRecorder example that it seems to be based on. Unfortunately, my script does not work.
My big questions are:
Am I initializing the camera correctly?
Am I starting the event loop correctly?
Was there any other setup I was supposed to do?
In the example script pasted below, the intended operation is that after calling startImageCapture(), I should start printing "Got a frame..." messages from the CaptureDelegate. However, the camera's light never turns on and the delegate's callback is never executed.
Also, there are no failures during startImageCapture(), all functions claim to succeed, and it successfully finds the iSight device. Analyzing the session object in pdb shows that it has valid input and output objects, the output has a delegate assigned, the device is not in use by another processes, and the session is marked as running after startRunning() is called.
Here's the code:
#!/usr/bin/env python2.7
import sys
import os
import time
import objc
import QTKit
import AppKit
from Foundation import NSObject
from Foundation import NSTimer
from PyObjCTools import AppHelper
objc.setVerbose(True)
class CaptureDelegate(NSObject):
def captureOutput_didOutputVideoFrame_withSampleBuffer_fromConnection_(self, captureOutput,
videoFrame, sampleBuffer,
connection):
# This should get called for every captured frame
print "Got a frame: %s" % videoFrame
class QuitClass(NSObject):
def quitMainLoop_(self, aTimer):
# Just stop the main loop.
print "Quitting main loop."
AppHelper.stopEventLoop()
def startImageCapture():
error = None
# Create a QT Capture session
session = QTKit.QTCaptureSession.alloc().init()
# Find iSight device and open it
dev = QTKit.QTCaptureDevice.defaultInputDeviceWithMediaType_(QTKit.QTMediaTypeVideo)
print "Device: %s" % dev
if not dev.open_(error):
print "Couldn't open capture device."
return
# Create an input instance with the device we found and add to session
input = QTKit.QTCaptureDeviceInput.alloc().initWithDevice_(dev)
if not session.addInput_error_(input, error):
print "Couldn't add input device."
return
# Create an output instance with a delegate for callbacks and add to session
output = QTKit.QTCaptureDecompressedVideoOutput.alloc().init()
delegate = CaptureDelegate.alloc().init()
output.setDelegate_(delegate)
if not session.addOutput_error_(output, error):
print "Failed to add output delegate."
return
# Start the capture
print "Initiating capture..."
session.startRunning()
def main():
# Open camera and start capturing frames
startImageCapture()
# Setup a timer to quit in 10 seconds (hack for now)
quitInst = QuitClass.alloc().init()
NSTimer.scheduledTimerWithTimeInterval_target_selector_userInfo_repeats_(10.0,
quitInst,
'quitMainLoop:',
None,
False)
# Start Cocoa's main event loop
AppHelper.runConsoleEventLoop(installInterrupt=True)
print "After event loop"
if __name__ == "__main__":
main()
Thanks for any help you can provide!
OK, I spent a day diving through the depths of PyObjC and got it working.
For future record, the reason the code in the question did not work: variable scope and garbage collection. The session variable was deleted when it fell out of scope, which happened before the event processor ran. Something must be done to retain it so it is not freed before it has time to run.
Moving everything into a class and making session a class variable made the callbacks start working. Additionally, the code below demonstrates getting the frame's pixel data into bitmap format and saving it via Cocoa calls, and also how to copy it back into Python's world-view as a buffer or string.
The script below will capture a single frame
#!/usr/bin/env python2.7
#
# camera.py -- by Trevor Bentley (02/04/2011)
#
# This work is licensed under a Creative Commons Attribution 3.0 Unported License.
#
# Run from the command line on an Apple laptop running OS X 10.6, this script will
# take a single frame capture using the built-in iSight camera and save it to disk
# using three methods.
#
import sys
import os
import time
import objc
import QTKit
from AppKit import *
from Foundation import NSObject
from Foundation import NSTimer
from PyObjCTools import AppHelper
class NSImageTest(NSObject):
def init(self):
self = super(NSImageTest, self).init()
if self is None:
return None
self.session = None
self.running = True
return self
def captureOutput_didOutputVideoFrame_withSampleBuffer_fromConnection_(self, captureOutput,
videoFrame, sampleBuffer,
connection):
self.session.stopRunning() # I just want one frame
# Get a bitmap representation of the frame using CoreImage and Cocoa calls
ciimage = CIImage.imageWithCVImageBuffer_(videoFrame)
rep = NSCIImageRep.imageRepWithCIImage_(ciimage)
bitrep = NSBitmapImageRep.alloc().initWithCIImage_(ciimage)
bitdata = bitrep.representationUsingType_properties_(NSBMPFileType, objc.NULL)
# Save image to disk using Cocoa
t0 = time.time()
bitdata.writeToFile_atomically_("grab.bmp", False)
t1 = time.time()
print "Cocoa saved in %.5f seconds" % (t1-t0)
# Save a read-only buffer of image to disk using Python
t0 = time.time()
bitbuf = bitdata.bytes()
f = open("python.bmp", "w")
f.write(bitbuf)
f.close()
t1 = time.time()
print "Python saved buffer in %.5f seconds" % (t1-t0)
# Save a string-copy of the buffer to disk using Python
t0 = time.time()
bitbufstr = str(bitbuf)
f = open("python2.bmp", "w")
f.write(bitbufstr)
f.close()
t1 = time.time()
print "Python saved string in %.5f seconds" % (t1-t0)
# Will exit on next execution of quitMainLoop_()
self.running = False
def quitMainLoop_(self, aTimer):
# Stop the main loop after one frame is captured. Call rapidly from timer.
if not self.running:
AppHelper.stopEventLoop()
def startImageCapture(self, aTimer):
error = None
print "Finding camera"
# Create a QT Capture session
self.session = QTKit.QTCaptureSession.alloc().init()
# Find iSight device and open it
dev = QTKit.QTCaptureDevice.defaultInputDeviceWithMediaType_(QTKit.QTMediaTypeVideo)
print "Device: %s" % dev
if not dev.open_(error):
print "Couldn't open capture device."
return
# Create an input instance with the device we found and add to session
input = QTKit.QTCaptureDeviceInput.alloc().initWithDevice_(dev)
if not self.session.addInput_error_(input, error):
print "Couldn't add input device."
return
# Create an output instance with a delegate for callbacks and add to session
output = QTKit.QTCaptureDecompressedVideoOutput.alloc().init()
output.setDelegate_(self)
if not self.session.addOutput_error_(output, error):
print "Failed to add output delegate."
return
# Start the capture
print "Initiating capture..."
self.session.startRunning()
def main(self):
# Callback that quits after a frame is captured
NSTimer.scheduledTimerWithTimeInterval_target_selector_userInfo_repeats_(0.1,
self,
'quitMainLoop:',
None,
True)
# Turn on the camera and start the capture
self.startImageCapture(None)
# Start Cocoa's main event loop
AppHelper.runConsoleEventLoop(installInterrupt=True)
print "Frame capture completed."
if __name__ == "__main__":
test = NSImageTest.alloc().init()
test.main()
QTKit is deprecated and PyObjC is a big dependency (and seems to be tricky to build if you want it in HomeBrew). Plus PyObjC did not have most of AVFoundation so I created a simple camera extension for Python that uses AVFoundation to record a video or snap a picture, it requires no dependencies (Cython intermediate files are committed to avoid the need to have Cython for most users).
It should be possible to build it like this:
pip install -e git+https://github.com/dashesy/pyavfcam.git
Then we can use it to take a picture:
import pyavfcam
# Open the default video source
cam = pyavfcam.AVFCam(sinks='image')
frame = cam.snap_picture('test.jpg') # frame is a memory buffer np.asarray(frame) can retrieve
Not related to this question, but if the AVFCam class is sub-classed, the overridden methods will be called with the result.