I'm using Python 3 to output 2 progress bars in the console like this:
100%|###############################################|
45%|###################### |
Both bars grow concurrently in separate threads.
The thread operations are fine and both progress bars are doing their job, but when I want to print them out they print on top of each other on one line in the console. I just got one line progress bar which alternates between showing these 2 progress bars.
Is there any way these progress bars can grow on separate lines concurrently?
You need a CLI framework. Curses is perfect if you are working on Unix (and there is a port for Windows which can be found here : https://stackoverflow.com/a/19851287/1741450 )
import curses
import time
import threading
def show_progress(win,X_line,sleeping_time):
# This is to move the progress bar per iteration.
pos = 10
# Random number I chose for demonstration.
for i in range(15):
# Add '.' for each iteration.
win.addstr(X_line,pos,".")
# Refresh or we'll never see it.
win.refresh()
# Here is where you can customize for data/percentage.
time.sleep(sleeping_time)
# Need to move up or we'll just redraw the same cell!
pos += 1
# Current text: Progress ............... Done!
win.addstr(X_line,26,"Done!")
# Gotta show our changes.
win.refresh()
# Without this the bar fades too quickly for this example.
time.sleep(0.5)
def show_progress_A(win):
show_progress( win, 1, 0.1)
def show_progress_B(win):
show_progress( win, 4 , 0.5)
if __name__ == '__main__':
curses.initscr()
win = curses.newwin(6,32,14,10)
win.border(0)
win.addstr(1,1,"Progress ")
win.addstr(4,1,"Progress ")
win.refresh()
threading.Thread( target = show_progress_B, args = (win,) ).start()
time.sleep(2.0)
threading.Thread( target = show_progress_A, args = (win,)).start()
Related
I am new to PySimpleGui and wonder if the following concept can be applied.
I have to work with two scripts, one related to a GUI and a second one that it is a "Do Something Script".
I am interested in printing the status of an external loop in a window of my GUI Script
The GUIScript that I have coded as an example is the following:
import PySimpleGUI as sg
import MainScript
### GUI Script ###
layout = [
[sg.Text('My text 1 is: '), sg.Text('{}'.format('foo1'), key='-text1-')],
[sg.Text('My text 2 is: '), sg.Text('{}'.format('foo2'), key='-text2-')],
[sg.Text('Sweep number: '), sg.Text('{}'.format('0'), key='-SWEEP-')],
[sg.ProgressBar(max_value=40, orientation='horizontal', size=(50,20), key='-PRO_BAR-')],
[sg.Button("OK"), sg.Button("Cancel")],
]
window =sg.Window("Progress Bar",layout)
while True:
event, values = window.read()
if event == 'Cancel' or event == sg.WIN_CLOSED:
break
if event == "OK":
for s in range(50):
event, values = window.read(1000)
if event == 'Cancel':
break
window['-PRO_BAR-'].update(s+1)
window['-text1-'].update('my new text1')
window['-text2-'].update('my new text2')
window['-SWEEP-'].update(s)
window.refresh()
window.close()
The previous script works as "what I would like to have". I mean that the window['-PRO_BAR-'].update(s+1) and window['-SWEEP-'].update(s) update according to the s value of its own for-loop.
However, I would like to pick up the variable=i from the MainScript (which is the following), and use it back in the GUIScript.
##### MainScript ####
SWEEPS = 50
SWEEPS_LEN = list(range(1, SWEEPS+1))
for i in enumerate(SWEEPS_LEN):
print('sweep: {}'.format(i))
# Do something
# Save Data
Here, it is obvious that i=0, i=1, i=2.... I want to use these variables (that change in the loop) them as:
window['-PRO_BAR-'].update(i+1) and window['-SWEEP-'].update(i), such as the values update.
The GUIScript would show:
Sweep: 1
then i=2, updates and shows
Sweep: 2
then updates and...
Thanks.
Say we have multiple subprocesses like the following which has some results printed in real time to sys.stdout or sys.stderr.
proc1 = subprocess.Popen(['cmd1'],
env=venv1,
stdout=sys.stdout,
stderr=sys.stderr,
)
proc2 = subprocess.Popen(['cmd2'],
env=venv2,
stdout=sys.stdout,
stderr=sys.stderr,
)
However, after executing this script in the terminal, while looking at what is being printed, it is not easy to distinguish which print is from the first process and which is from the second.
Is there a solution for this to see the stdout of each process separately, like if the terminal screen could be partitioned and each partition would have shown the printing results from each process?
I have written for you a curses application which will do what you request: divide the terminal window into a number of partitions and then watch the different output streams in the different partitions.
The function watch_fd_in_panes will take a list of lists, where the sub-lists specify which file descriptors to watch inside each partition.
Here is what your example calling code will look like:
import subprocess
from watcher import watch_fds_in_panes
proc1 = subprocess.Popen('for i in `seq 30`; do date; sleep 1 ; done',
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
# this process also writes something on stderr
proc2 = subprocess.Popen('ls -l /asdf; for i in `seq 20`; do echo $i; sleep 0.5; done',
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
proc3 = subprocess.Popen(['echo', 'hello'],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
try:
watch_fds_in_panes([[proc1.stdout.fileno(), proc1.stderr.fileno()],
[proc2.stdout.fileno(), proc2.stderr.fileno()],
[proc3.stdout.fileno(), proc3.stderr.fileno()]],
sleep_at_end=3.)
except KeyboardInterrupt:
print("interrupted")
proc1.kill()
proc2.kill()
proc3.kill()
To run it you will need these two files:
panes.py
import curses
class Panes:
"""
curses-based app that divides the screen into a number of scrollable
panes and lets the caller write text into them
"""
def start(self, num_panes):
"set up the panes and initialise the app"
# curses init
self.num = num_panes
self.stdscr = curses.initscr()
curses.noecho()
curses.cbreak()
# split the screen into number of panes stacked vertically,
# drawing some horizontal separator lines
scr_height, scr_width = self.stdscr.getmaxyx()
div_ys = [scr_height * i // self.num for i in range(1, self.num)]
for y in div_ys:
self.stdscr.addstr(y, 0, '-' * scr_width)
self.stdscr.refresh()
# 'boundaries' contains y coords of separator lines including notional
# separator lines above and below everything, and then the panes
# occupy the spaces between these
boundaries = [-1] + div_ys + [scr_height]
self.panes = []
for i in range(self.num):
top = boundaries[i] + 1
bottom = boundaries[i + 1] - 1
height = bottom - top + 1
width = scr_width
# create a scrollable pad for this pane, of height at least
# 'height' (could be more to retain some scrollback history)
pad = curses.newpad(height, width)
pad.scrollok(True)
self.panes.append({'pad': pad,
'coords': [top, 0, bottom, width],
'height': height})
def write(self, pane_num, text):
"write text to the specified pane number (from 0 to num_panes-1)"
pane = self.panes[pane_num]
pad = pane['pad']
y, x = pad.getyx()
pad.addstr(y, x, text)
y, x = pad.getyx()
view_top = max(y - pane['height'], 0)
pad.refresh(view_top, 0, *pane['coords'])
def end(self):
"restore the original terminal behaviour"
curses.nocbreak()
self.stdscr.keypad(0)
curses.echo()
curses.endwin()
and watcher.py
import os
import select
import time
from panes import Panes
def watch_fds_in_panes(fds_by_pane, sleep_at_end=0):
"""
Use panes to watch output from a number of fds that are writing data.
fds_by_pane contains a list of lists of fds to watch in each pane.
"""
panes = Panes()
npane = len(fds_by_pane)
panes.start(npane)
pane_num_for_fd = {}
active_fds = []
data_tmpl = {}
for pane_num, pane_fds in enumerate(fds_by_pane):
for fd in pane_fds:
active_fds.append(fd)
pane_num_for_fd[fd] = pane_num
data_tmpl[fd] = bytes()
try:
while active_fds:
all_data = data_tmpl.copy()
timeout = None
while True:
fds_read, _, _ = select.select(active_fds, [], [], timeout)
timeout = 0
if fds_read:
for fd in fds_read:
data = os.read(fd, 1)
if data:
all_data[fd] += data
else:
active_fds.remove(fd) # saw EOF
else:
# no more data ready to read
break
for fd, data in all_data.items():
if data:
strng = data.decode('utf-8')
panes.write(pane_num_for_fd[fd], strng)
except KeyboardInterrupt:
panes.end()
raise
time.sleep(sleep_at_end)
panes.end()
Finally, here is a screenshot of the above code in action:
In this example, we are monitoring both stdout and stderr of each process in the relevant partition. In the screenshot, the line that proc2 wrote to stderr before the start of the loop (regarding /asdf) has appeared after the first line that proc2 wrote to stdout during the first iteration of the loop (i.e. the 1 which has since scrolled off the top of the partition), but this is cannot be controlled because they were written to different pipes.
I am designing a new time/score keeper for an air hockey table using a PyBoard as a base. My plan is to use a TM1627 (4x7seg) for time display, rotary encoder w/ button to set the time, IR and a couple 7segs for scoring, IR reflector sensors for goallines, and a relay to control the fan.
I'm getting hung up trying to separate the clock into its own thread while focusing on reading the sensors. Figured I could use uasyncio to split everything up nicely, but I can't figure out where to put the directives to spin off a thread for the clock and eventually the sensors.
On execution right now, it appears the rotary encoder is assigned the default value, no timer is started, the encoder doesn't set the time, and the program returns control to REPL rather quickly.
Prior to trying to async everything, I had the rotary encoder and timer working well. Now, not so much.
from rotary_irq_pyb import RotaryIRQ
from machine import Pin
import tm1637
import utime
import uasyncio
async def countdown(cntr):
# just init min/sec to any int > 0
min = sec = 99
enableColon = True
while True:
# update the 4x7seg with the time remaining
min = abs(int((cntr - utime.time()) / 60))
sec = (cntr - utime.time()) % 60
#print(str(), str(sec), sep=':' )
enableColon = not enableColon # alternately blink the colon
tm.numbers(min, sec, colon = enableColon)
if(min + sec == 0): # once both reach zero, break
break
await uasyncio.sleep(500)
X1 = pyb.Pin.board.X1
X2 = pyb.Pin.board.X2
Y1 = pyb.Pin.board.Y1
Y2 = pyb.Pin.board.Y2
button = pyb.Pin(pyb.Pin.board.X3, pyb.Pin.IN)
r = RotaryIRQ(pin_num_clk=X1,
pin_num_dt=X2,
min_val=3,
max_val=10,
reverse=False,
range_mode=RotaryIRQ.RANGE_BOUNDED)
tm = tm1637.TM1637(clk = Y1, dio = Y2)
val_old = val_new = 0
while True:
val_new = r.value()
if(val_old != val_new):
val_old = val_new
print(str(val_new))
if(button.value()): # save value as minutes
loop = uasyncio.get_event_loop()
endTime = utime.time() + (60 * val_new)
loop.create_task(countdown(endTime))
r.close() # Turn off Rotary Encoder
break
#loop = uasyncio.get_event_loop()
#loop.create_task(countdown(et))
#loop.run_until_complete(countdown(et))
I'm sure it's something simple, but this is the first non-CLI python script I've done, so I'm sure there are a bunch of silly mistakes. Any assistance would be appreciated.
First of all, I'm a newby in python. Had to take a course of it in college and got hooked by its efficiency.
I have this sticky problem where the Windows 7 prompt becomes unresponsive after using a curses window. In Windows 10 it works well. Note that I'm using the Win7 terminal with its default settings. In my code I create a curses window to show 2 simultaneous progress bars, each for a file download. I implemented this by passing the curses window to a FileDownload class (one class instance for each download) that handles its progress bar inside this window. Oddly, in Windows 7 when the downloads are done and the control returns to the prompt, it becomes unresponsive to the keyboard. I worked around this by invoking curses.endwin() after using the window, but this causes the prompt to display all the way down the screen buffer, what hides the curses window.
Here is my code. Any ideas are greatly appreciated. Thanks!
# Skeleton version for simulations.
# Downloads 2 files simultaneously and shows a progress bar for each.
# Each file download is a FileDownload object that interacts with a
# common curses window passed as an argument.
import requests, math, threading, curses, datetime
class FileDownload:
def __init__(self, y_pos, window, url):
# Y position of the progress bar in the download queue window.
self.__bar_pos = int(y_pos)
self.__progress_window = window
self.__download_url = url
# Status of the file download object.
self.__status = "queued"
t = threading.Thread(target=self.__file_downloader)
t.start()
# Downloads selected file and handles its progress bar.
def __file_downloader(self):
file = requests.get(self.__download_url, stream=True)
self.__status = "downloading"
self.__progress_window.addstr(self.__bar_pos + 1, 1, "0%" + " " * 60 + "100%")
size = int(file.headers.get('content-length'))
win_prompt = "Downloading " + format(size, ",d") + " Bytes:"
self.__progress_window.addstr(self.__bar_pos, 1, win_prompt)
file_name = str(datetime.datetime.now().strftime("%Y-%m-%d_%H.%M.%d"))
dump = open(file_name, "wb")
# Progress bar length.
bar_space = 58
# Same as an index.
current_iteration = 0
# Beginning position of the progress bar.
progress_position = 4
# How many iterations will be needed (in chunks of 1 MB).
iterations = math.ceil(size / 1024 ** 2)
# Downloads the file in 1MB chunks.
for block in file.iter_content(1024 ** 2):
dump.write(block)
# Progress bar controller.
current_iteration += 1
step = math.floor(bar_space / iterations)
if current_iteration > 1:
progress_position += step
if current_iteration == iterations:
step = bar_space - step * (current_iteration - 1)
# Updates the progress bar.
self.__progress_window.addstr(self.__bar_pos + 1, progress_position,
"#" * step)
dump.close()
self.__status = "downloaded"
# Returns the current status of the file download ("queued", "downloading" or
# "downloaded").
def get_status(self):
return self.__status
# Instantiates each file download.
def files_downloader():
# Creates curses window.
curses.initscr()
win = curses.newwin(8, 70)
win.border(0)
win.immedok(True)
# Download URLs.
urls = ["http://ipv4.download.thinkbroadband.com/10MB.zip",
"http://ipv4.download.thinkbroadband.com/5MB.zip"]
downloads_dct = {}
for n in range(len(urls)):
# Progress bar position in the window for the file.
y_pos = n * 4 + 1
downloads_dct[n + 1] = FileDownload(y_pos, win, urls[n])
# Waits for all files to be downloaded before passing control of the terminal
# to the user.
all_downloaded = False
while not all_downloaded:
all_downloaded = True
for key, file_download in downloads_dct.items():
if file_download.get_status() != "downloaded":
all_downloaded = False
# Prevents the prompt from returning inside the curses window.
win.addstr(7, 1, "-")
# This solves the unresponsive prompt issue but hides the curses window if the screen buffer
# is higher than the window size.
# curses.endwin()
while input("\nEnter to continue: ") == "":
files_downloader()
Perhaps you're using cygwin (and ncurses): ncurses (like any other curses implementation) changes the terminal I/O mode when it is running. The changes that you probably are seeing is that
input characters are not echoed
you have to type controlJ to end an input line, rather than just Enter
output is not flushed automatically at the end of each line
It makes those changes to allow it to read single characters and also to use the terminal more efficiently.
To change back to the terminal's normal I/O mode, you would use the endwin function. The reset_shell_mode function also would be useful.
Further reading:
endwin (ncurses manual)
reset_shell_mode (ncurses manual)
While running program through the terminal we can stop the program by pressing 'Ctrl+c' and it will show the message as 'KeyboardInterrupt' . So, is there any way to do the sane thing by clicking the push-button in PyQt.
If your program is running a loop, you can call processEvents periodically to allow the gui time to update (which should allow you to click a button to close the application):
count = 0
while True:
count += 1
if not count % 50:
QtGui.qApp.processEvents()
# do stuff...
In my script to interrupt an infinite loop I also used QtGui.qApp.processEvents() and it worked out fine. The infinite loop writes to and reads data from a serial port and the user can interrupt the loop with a push button (1.condition).
def Move_Right(self):
# move the slide right
cmdPack = struct.pack(cmdStruct, Address, Rotate_Right, 0, Motor5, Speed5)
dataByte = bytearray(cmdPack)
checksumInt = sum(dataByte[:]) % 256
msgPack = struct.pack(msgStruct, Address, Rotate_Right, 0, Motor5, Speed5, checksumInt)
ser0.flushOutput() # Clear output buffer
ser0.write(msgPack)
# read the switch status
cmdPack = struct.pack(cmdStruct, Address, Command.GAP, 10, Motor5, 0)
dataByte = bytearray(cmdPack)
checksumInt = sum(dataByte[:]) % 256
msgPack = struct.pack(msgStruct, Address, Command.GAP, 10, Motor5, 0, checksumInt)
ser0.flushOutput() # Clear output buffer
# check the switch status with an infinite write/read loop with two break out conditions
while True:
QtGui.qApp.processEvents() # 1. condition: interrupt with push button
ser0.write(msgPack)
reply = ser0.read(9)
answer = struct.unpack('>BBBBlB', reply)
value = answer[4]
command = answer[3]
if (command == 6) and (value == 1): # 2. condition: interrupt with limit switch
print 'end of line'
Stop_Motor5()
break