To implement a status bar like below:
[========== ] 45%
[================ ] 60%
[==========================] 100%
I want to this to be printed out to stdout, and keep refreshing it, not print to another line. How to do this?
The '\r' character (carriage return) resets the cursor to the beginning of the line and allows you to write over what was previously on the line.
from time import sleep
import sys
for i in range(21):
sys.stdout.write('\r')
# the exact output you're looking for:
sys.stdout.write("[%-20s] %d%%" % ('='*i, 5*i))
sys.stdout.flush()
sleep(0.25)
I'm not 100% sure if this is completely portable across all systems, but it works on Linux and OSX at the least.
There's a Python module that you can get from PyPI called progressbar that implements such functionality. If you don't mind adding a dependency, it's a good solution. Otherwise, go with one of the other answers.
A simple example of how to use it:
import progressbar
from time import sleep
bar = progressbar.ProgressBar(maxval=20, \
widgets=[progressbar.Bar('=', '[', ']'), ' ', progressbar.Percentage()])
bar.start()
for i in xrange(20):
bar.update(i+1)
sleep(0.1)
bar.finish()
To install it, you can use easy_install progressbar, or pip install progressbar if you prefer pip.
I found useful library tqdm (https://github.com/tqdm/tqdm/, previously: https://github.com/noamraph/tqdm). It automatically estimates time of completion and can be used as iterator.
Usage:
import tqdm
import time
for i in tqdm.tqdm(range(1000)):
time.sleep(0.01)
# or other long operations
Results in:
|####------| 450/1000 45% [elapsed: 00:04 left: 00:05, 99.15 iters/sec]
tqdm can wrap any iterable.
You can use \r (carriage return). Demo:
import sys
total = 10000000
point = total / 100
increment = total / 20
for i in xrange(total):
if(i % (5 * point) == 0):
sys.stdout.write("\r[" + "=" * (i / increment) + " " * ((total - i)/ increment) + "]" + str(i / point) + "%")
sys.stdout.flush()
Here you can use following code as a function:
def drawProgressBar(percent, barLen = 20):
sys.stdout.write("\r")
progress = ""
for i in range(barLen):
if i < int(barLen * percent):
progress += "="
else:
progress += " "
sys.stdout.write("[ %s ] %.2f%%" % (progress, percent * 100))
sys.stdout.flush()
With use of .format:
def drawProgressBar(percent, barLen = 20):
# percent float from 0 to 1.
sys.stdout.write("\r")
sys.stdout.write("[{:<{}}] {:.0f}%".format("=" * int(barLen * percent), barLen, percent * 100))
sys.stdout.flush()
Try this function using only the built-in sys:
import sys
def print_progress_bar(index, total, label):
n_bar = 50 # Progress bar width
progress = index / total
sys.stdout.write('\r')
sys.stdout.write(f"[{'=' * int(n_bar * progress):{n_bar}s}] {int(100 * progress)}% {label}")
sys.stdout.flush()
Usage:
foo_list = ["a", "b", "c", "d"]
total = len(foo_list)
for index, item in enumerate(foo_list):
print_progress_bar(index, total, "foo bar")
sleep(0.5)
enumerate(foo_list) gives you access to the index value during a loop.
Output:
[================================================ ] 96% foo bar
None of the answers posted completely addressed my needs. So I wrote my own as shown above. The features I needed:
Pass only the step number and total number of steps and it does the difficult job of calculating percentage complete.
Using 60 characters, divide them into 480 "ticks" to yield 0.21 % per tick. Without ticks, each character would only be 1.67 %.
Support for prepending a title.
Optional percentage complete at end of line.
Variable length progress bar that defaults to 60 characters or 480 "ticks".
Set progress bar color, the default is green.
How to Call the Progress Display
Calling the progress display is pretty straight forward. For the above sample .gif the function was called using:
percent_complete(step, total_steps, title="Convert Markdown")
The total_steps was about 2,500 for len(rows) in Stack Exchange Data Dump in CSV format. The step was the current row number as each Stack Exchange Markdown Q&A was converted to Kramdown (for GitHub Pages).
Python Code
The code is straight forward, but a bit longer than the other answers:
def percent_complete(step, total_steps, bar_width=60, title="", print_perc=True):
import sys
# UTF-8 left blocks: 1, 1/8, 1/4, 3/8, 1/2, 5/8, 3/4, 7/8
utf_8s = ["█", "▏", "▎", "▍", "▌", "▋", "▊", "█"]
perc = 100 * float(step) / float(total_steps)
max_ticks = bar_width * 8
num_ticks = int(round(perc / 100 * max_ticks))
full_ticks = num_ticks / 8 # Number of full blocks
part_ticks = num_ticks % 8 # Size of partial block (array index)
disp = bar = "" # Blank out variables
bar += utf_8s[0] * int(full_ticks) # Add full blocks into Progress Bar
# If part_ticks is zero, then no partial block, else append part char
if part_ticks > 0:
bar += utf_8s[part_ticks]
# Pad Progress Bar with fill character
bar += "▒" * int((max_ticks/8 - float(num_ticks)/8.0))
if len(title) > 0:
disp = title + ": " # Optional title to progress display
# Print progress bar in green: https://stackoverflow.com/a/21786287/6929343
disp += "\x1b[0;32m" # Color Green
disp += bar # Progress bar to progress display
disp += "\x1b[0m" # Color Reset
if print_perc:
# If requested, append percentage complete to progress display
if perc > 100.0:
perc = 100.0 # Fix "100.04 %" rounding error
disp += " {:6.2f}".format(perc) + " %"
# Output to terminal repetitively over the same line using '\r'.
sys.stdout.write("\r" + disp)
sys.stdout.flush()
Python Code Notes
A few points:
The [ .... ] bracket placeholders requirement in the question are not necessary because there is the fill characters that serve the same purpose. This saves two extra characters to make the progress bar wider.
The bar_width keyword parameter can be used depending on screen width. The default of 60 seems a good fit for most purposes.
The print_perc=True keyword parameter default can be overridden by passing print_perc=False when calling the function. This would allow a longer progress bar.
The title="" keyword parameter defaults to no title. Should you desire one use title="My Title" and : will automatically be added to it.
When your program finishes remember to call sys.stdout.write("\r") followed by sys.stdout.flush() to clear the progress display line.
Summary
This answer is a bit longer than the others but it's important to note it's a full solution, not part of a solution that you need to add more code to.
Another point is this solution has no dependencies and nothing extra to install. The UTF-8 character set is supported by Python and gnome-terminal was no extra setup required. If you are using Python 2.7 you might require # -*- coding: utf-8 -*- after the shebang. IE as the second line of your program.
The function could be converted to a class with separate init, update, pause (for printing debug stuff to the screen), resume and close methods.
This function was converted from a bash script:
How to add a progress bar to a shell script?
The bash script would display Sony TV volume with libnotify-bin (pop-up bubble message) whenever TV volume was changed. If you are interested in a bash progress bar, please visit the Stack Overflow link.
Edit January 30, 2022
Change from 4 ticks to 8 ticks per character.
Remove breaks between full blocks.
Add color support.
def printProgressBar(value,label):
n_bar = 40 #size of progress bar
max = 100
j= value/max
sys.stdout.write('\r')
bar = '█' * int(n_bar * j)
bar = bar + '-' * int(n_bar * (1-j))
sys.stdout.write(f"{label.ljust(10)} | [{bar:{n_bar}s}] {int(100 * j)}% ")
sys.stdout.flush()
call:
printProgressBar(30,"IP")
IP | [████████████----------------------------] 30%
based on the above answers and other similar questions about CLI progress bar, I think I got a general common answer to all of them. Check it at https://stackoverflow.com/a/15860757/2254146
Here is a copy of the function, but modified to fit your style:
import time, sys
# update_progress() : Displays or updates a console progress bar
## Accepts a float between 0 and 1. Any int will be converted to a float.
## A value under 0 represents a 'halt'.
## A value at 1 or bigger represents 100%
def update_progress(progress):
barLength = 20 # Modify this to change the length of the progress bar
status = ""
if isinstance(progress, int):
progress = float(progress)
if not isinstance(progress, float):
progress = 0
status = "error: progress var must be float\r\n"
if progress < 0:
progress = 0
status = "Halt...\r\n"
if progress >= 1:
progress = 1
status = "Done...\r\n"
block = int(round(barLength*progress))
text = "\rPercent: [{0}] {1}% {2}".format( "="*block + " "*(barLength-block), progress*100, status)
sys.stdout.write(text)
sys.stdout.flush()
Looks like
Percent: [====================] 99.0%
If you are developing a command line interface, I suggest you to take a look at click which is very nice:
import click
import time
for filename in range(3):
with click.progressbar(range(100), fill_char='=', empty_char=' ') as bar:
for user in bar:
time.sleep(0.01)
Here the output you get:
$ python test.py
[====================================] 100%
[====================================] 100%
[========= ] 27%
As described in Mark Rushakoff's solution, you can output the carriage return character, sys.stdout.write('\r'), to reset the cursor to the beginning of the line. To generalize that solution, while also implementing Python 3's f-Strings, you could use
from time import sleep
import sys
n_bar = 50
iterable = range(33) # for demo purposes
n_iter = len(iterable)
for i, item in enumerate(iterable):
j = (i + 1) / n_iter
sys.stdout.write('\r')
sys.stdout.write(f"[{'=' * int(n_bar * j):{n_bar}s}] {int(100 * j)}%")
sys.stdout.flush()
sleep(0.05)
# do something with <item> here
To be pure python and not make system calls:
from time import sleep
for i in range(21):
spaces = " " * (20 - i)
percentage = 5*i
print(f"\r[{'='*i}{spaces}]{percentage}%", flush=True, end="")
sleep(0.25)
I came upon this thread today and after having tried out this solution from Mark Rushakoff
from time import sleep
import sys
for i in range(21):
sys.stdout.write('\r')
# the exact output you're looking for:
sys.stdout.write("[%-20s] %d%%" % ('='*i, 5*i))
sys.stdout.flush()
sleep(0.25)
I can say that this works fine on W7-64 with python 3.4.3 64-bit but only in the native console. However when using the built-in console of spyder 3.0.0dev, the line breaks are still/again present. As this took me some time to figure out, I'd like to report this observation here.
Easiest is still
import sys
total_records = 1000
for i in range (total_records):
sys.stdout.write('\rUpdated record: ' + str(i) + ' of ' + str(total_records))
sys.stdout.flush()
Key is to convert the integer type to string.
Building on some of the answers here and elsewhere, I've written this simple function which displays a progress bar and elapsed/estimated remaining time. Should work on most unix-based machines.
import time
import sys
percent = 50.0
start = time.time()
draw_progress_bar(percent, start)
def draw_progress_bar(percent, start, barLen=20):
sys.stdout.write("\r")
progress = ""
for i in range(barLen):
if i < int(barLen * percent):
progress += "="
else:
progress += " "
elapsedTime = time.time() - start;
estimatedRemaining = int(elapsedTime * (1.0/percent) - elapsedTime)
if (percent == 1.0):
sys.stdout.write("[ %s ] %.1f%% Elapsed: %im %02is ETA: Done!\n" %
(progress, percent * 100, int(elapsedTime)/60, int(elapsedTime)%60))
sys.stdout.flush()
return
else:
sys.stdout.write("[ %s ] %.1f%% Elapsed: %im %02is ETA: %im%02is " %
(progress, percent * 100, int(elapsedTime)/60, int(elapsedTime)%60,
estimatedRemaining/60, estimatedRemaining%60))
sys.stdout.flush()
return
This is quite a simple approach can be used with any loop.
#!/usr/bin/python
for i in range(100001):
s = ((i/5000)*'#')+str(i)+(' %')
print ('\r'+s),
Using #Mark-Rushakoff answer, I worked out a simpler approach, no need to call the sys library. It works with Python 3. Tested in Windows:
from time import sleep
for i in range(21):
# the exact output you're looking for:
print ("\r[%-20s] %d%%" % ('='*i, 5*i), end='')
sleep(0.25)
Try PyProg. PyProg is an open-source library for Python to create super customizable progress indicators & bars.
It is currently at version 1.0.2; it is hosted on Github and available on PyPI (Links down below). It is compatible with Python 3 & 2 and it can also be used with Qt Console.
It is really easy to use. The following code:
import pyprog
from time import sleep
# Create Object
prog = pyprog.ProgressBar(" ", " ", total=34, bar_length=26, complete_symbol="=", not_complete_symbol=" ", wrap_bar_prefix=" [", wrap_bar_suffix="] ", progress_explain="", progress_loc=pyprog.ProgressBar.PROGRESS_LOC_END)
# Update Progress Bar
prog.update()
for i in range(34):
# Do something
sleep(0.1)
# Set current status
prog.set_stat(i + 1)
# Update Progress Bar again
prog.update()
# Make the Progress Bar final
prog.end()
will produce exactly what you want (even the bar length!):
[=========== ] 45%
[=============== ] 60%
[==========================] 100%
For more options to customize the progress bar, go to the Github page of this website.
I actually made PyProg because I needed a simple but super customizable progress bar library. You can easily install it with: pip install pyprog.
PyProg Github: https://github.com/Bill13579/pyprog
PyPI: https://pypi.python.org/pypi/pyprog/
Here is something I have made using the solution by #Mark-Rushakoff. To adaptively adjust to the terminal width.
from time import sleep
import os
import sys
from math import ceil
l = list(map(int,os.popen('stty size','r').read().split()))
col = l[1]
col = col - 6
for i in range(col):
sys.stdout.write('\r')
getStr = "[%s " % ('='*i)
sys.stdout.write(getStr.ljust(col)+"]"+"%d%%" % (ceil((100/col)*i)))
sys.stdout.flush()
sleep(0.25)
print("")
Per Steven C. Howell's comment on Mark Rushakoff's answer
j = (i + 1) / n
stdout.write('\r')
stdout.write('[%-20s] %d%%' % ('='*int(20*j), 100*j))
stdout.flush()
where i is the current item and n is the total number of items
For Python 3.6 the following works for me to update the output inline:
for current_epoch in range(10):
for current_step) in range(100):
print("Train epoch %s: Step %s" % (current_epoch, current_step), end="\r")
print()
import progressbar
import time
# Function to create
def animated_marker():
widgets = ['Loading: ', progressbar.Bar('=', '[', ']', '-'), progressbar.Percentage()]
bar = progressbar.ProgressBar(max_value=200,widgets=widgets).start()
for i in range(200):
time.sleep(0.1)
bar.update(i+1)
bar.finish()
# Driver's code
animated_marker()
Here Is a simple progress bar code with 0 imports
#!/usr/bin/python3
def progressbar(current_value,total_value,bar_lengh,progress_char):
percentage = int((current_value/total_value)*100) # Percent Completed Calculation
progress = int((bar_lengh * current_value ) / total_value) # Progress Done Calculation
loadbar = "Progress: [{:{len}}]{}%".format(progress*progress_char,percentage,len = bar_lengh) # Progress Bar String
print(loadbar, end='\r') # Progress Bar Output
if __name__ == "__main__":
the_list = range(1,301)
for i in the_list:
progressbar(i,len(the_list),30,'■')
print("\n")
Progress: [■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■]100%
I am writing a code for an information retrieval project, which reads Wikipedia pages in XML format from a file, processes the string (I've omitted this part for the sake of simplicity), tokenizes the strings and build positional indexes for the terms found on the pages. Then it saves the indexes to a file using pickle once, and then reads it from that file for the next usages for less processing time (I've included the code for that parts, but they're commented)
After that, I need to fill a 1572 * ~97000 matrix (1572 is the number of Wiki pages, and 97000 is the number of terms found in them. Like each Wiki page is a vector of words, and vectors[i][j] is, the number of occurrences of the i'th word of the word set in the j'th Wiki Page. (Again it's been simplified but it doesn't matter)
The problem is that it takes way too much memory to run the code, and even then, from a point between the 350th and 400th row of the matrix beyond, it doesn't proceed to run the code (it doesn't stop either). I thought the problem was with memory, because when its usage exceeded my 7.7GiB RAM and 1.7GiB swap, it stopped and printed:
Process finished with exit code 137 (interrupted by signal 9: SIGKILL)
But when I added a 6GiB memory by making a swap file for Python3.7 (using the script recommended here, the program didn't run out of memory, but instead got stuck when it had 7.7GiB RAM + 3.9GiB swap memory occupied) as I said, at a point between the 350th and 400th iteration of i in the loop at the bottom. Instead of Ubuntu 18.04,I tried it on Windows 10, the screen simply went black. I tried this on Windows 7, again to no avail.
Next I thought it was a PyCharm issue, so I ran the python file using the python3 file.py command, and it got stuck at the very point it had with PyCharm. I even used the numpy.float16 datatype to save memory, but it had no effect. I asked a colleague about their matrix dimensions, they were similar to mine, but they weren't having problems with it. Is it malware or a memory leak? Or is it something am I doing something wrong here?
import pickle
from hazm import *
from collections import defaultdict
import numpy as np
'''For each word there's one of these. it stores the word's frequency, and the positions it has occurred in on each wiki page'''
class Positional_Index:
def __init__(self):
self.freq = 0
self.title = defaultdict(list)
self.text = defaultdict(list)
'''Here I tokenize words and construct indexes for them'''
# tree = ET.parse('Wiki.xml')
# root = tree.getroot()
# index_dict = defaultdict(Positional_Index)
# all_content = root.findall('{http://www.mediawiki.org/xml/export-0.10/}page')
#
# for page_index, pg in enumerate(all_content):
# title = pg.find('{http://www.mediawiki.org/xml/export-0.10/}title').text
# txt = pg.find('{http://www.mediawiki.org/xml/export-0.10/}revision') \
# .find('{http://www.mediawiki.org/xml/export-0.10/}text').text
#
# title_arr = word_tokenize(title)
# txt_arr = word_tokenize(txt)
#
# for term_index, term in enumerate(title_arr):
# index_dict[term].freq += 1
# index_dict[term].title[page_index] += [term_index]
#
# for term_index, term in enumerate(txt_arr):
# index_dict[term].freq += 1
# index_dict[term].text[page_index] += [term_index]
#
# with open('texts/indices.txt', 'wb') as f:
# pickle.dump(index_dict, f)
with open('texts/indices.txt', 'rb') as file:
data = pickle.load(file)
'''Here I'm trying to keep the number of occurrences of each word on each page'''
page_count = 1572
vectors = np.array([[0 for j in range(len(data.keys()))] for i in range(page_count)], dtype=np.float16)
words = list(data.keys())
word_count = len(words)
const_log_of_d = np.log10(1572)
""" :( """
for i in range(page_count):
for j in range(word_count):
vectors[i][j] = (len(data[words[j]].title[i]) + len(data[words[j]].text[i]))
if i % 50 == 0:
print("i:", i)
Update : I tried this on a friend's computer, this time it killed the process at someplace between the 1350th-1400th iteration.
I want to parallelize a task (progresser()) for a range of input parameters (L). The progress of each task should be monitored by an individual progress bar in the terminal. I'm using the tqdm package for the progress bars. The following code works on my Mac for up to 23 progress bars (L = list(range(23)) and below), but produces chaotic jumping of the progress bars starting at L = list(range(24)). Has anyone an idea how to fix this?
from time import sleep
import random
from tqdm import tqdm
from multiprocessing import Pool, freeze_support, RLock
L = list(range(24)) # works until 23, breaks starting at 24
def progresser(n):
text = f'#{n}'
sampling_counts = 10
with tqdm(total=sampling_counts, desc=text, position=n+1) as pbar:
for i in range(sampling_counts):
sleep(random.uniform(0, 1))
pbar.update(1)
if __name__ == '__main__':
freeze_support()
p = Pool(processes=None,
initargs=(RLock(),), initializer=tqdm.set_lock
)
p.map(progresser, L)
print('\n' * (len(L) + 1))
As an example of how it should look like in general, I provide a screenshot for L = list(range(16)) below.
versions: python==3.7.3, tqdm==4.32.1
I'm not getting any jumping when I set the size to 30. Maybe you have more processors and can have more workers running.
However, if n grows large you will start to see jumps because of the nature of the chunksize.
I.e
p.map will split your input into chunksizes and give each process a chunk. So as n grows larger, so does your chunksize, and so does your ....... yup position (pos=n+1)!
Note: Although map preserves the order of the results returned. The order its computed is arbitrary.
As n grows large I would suggest using processor id as the position to view progress on a per process basis.
from time import sleep
import random
from tqdm import tqdm
from multiprocessing import Pool, freeze_support, RLock
from multiprocessing import current_process
def progresser(n):
text = f'#{n}'
sampling_counts = 10
current = current_process()
pos = current._identity[0]-1
with tqdm(total=sampling_counts, desc=text, position=pos) as pbar:
for i in range(sampling_counts):
sleep(random.uniform(0, 1))
pbar.update(1)
if __name__ == '__main__':
freeze_support()
L = list(range(30)) # works until 23, breaks starting at 24
# p = Pool(processes=None,
# initargs=(RLock(),), initializer=tqdm.set_lock
# )
with Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),)) as p:
p.map(progresser, L)
print('\n' * (len(L) + 1))