Develop Reference

Customize tqdm bar in Python

I'm trying to change the display of 10000.000000001317/10000 to only two decimals.
Finished: 100%|████████████████████████████| 10000.000000001317/10000 [00:01<00:00, 9330.68it/s]
This is my code right now;
def create_dataframe(size):
df = pd.DataFrame()
pbar = tqdm(total=size)
#Increment is equal to the total number of records to be generated divided by the fields to be created
#divided by total (this being the necessary iterations for each field)
increment = size / 5 / size
df["ID"] = range(size)
pbar.set_description("Generating Names")
df["Name"], _ = zip(*[(fake.name(), pbar.update(increment)) for _ in range(size)])
pbar.set_description("Generating Emails")
df["Email"], _ = zip(*[(fake.free_email(), pbar.update(increment)) for _ in range(size)])
pbar.set_description("Generating Addresses")
df["Address"], _ = zip(*[(fake.address(), pbar.update(increment)) for _ in range(size)])
pbar.set_description("Generating Phones")
df["Phone"], _ = zip(*[(fake.phone_number(), pbar.update(increment)) for _ in range(size)])
pbar.set_description("Generating Comments")
df["Comment"], _ = zip(*[(fake.text(), pbar.update(increment)) for _ in range(size)])
pbar.set_description("Finished")
pbar.close()
return df
According to the docs, or at least what I've understood, this would be the default formatting to the argument bar_format;
pbar = tqdm(total=size, bar_format='{desc}{percentage:3.0f}%|{bar}| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, ''{rate_fmt}{postfix}]')
I tried:
Setting .2f in n_ftm, the output of this results in an error.
pbar = tqdm(total=size, bar_format='{desc}{percentage:3.0f}%|{bar}| {n_fmt:.2f}/{total_fmt} [{elapsed}<{remaining}, ''{rate_fmt}{postfix}]')
Or formatting the set_description,
pbar.set_description("Finished: {:.2f}/{:.2f}".format(pbar.n, size))
This prints another x/total before the actual bar;
Finished: 10000.00/10000.00: 100%|██████████| 10000.000000001317/10000 [00:01<00:00, 9702.65it/s]
Plus it would be ideal once the bar finished having 10000/10000 no decimals.

You cannot specify the number of floating points because n_fmt is a string. You can however pass unit_scale=True:
pbar = tqdm(total=size,
bar_format='{desc}{percentage:3.0f}%|{bar}| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, ''{rate_fmt}{postfix}]',
unit_scale=True)
According to the doc:
If 1 or True, the number of iterations will be reduced/scaled
automatically and a metric prefix following the
International System of Units standard will be added
(kilo, mega, etc.)
You'll get the following output for size=10000:
Finished: 100%|█████████████████████████████████████████████████████████████████████| 10.0k/10.0k [00:03<00:00, 3.31kit/s]

Python/ I dont understand one line for loops in a list comprehension

below codes purpose is how many times a specific string occurs consecutively in a given string. But I could not understand the logic of [sum(1 for _ in group)+1 for label, group in groups if label==''][0] . I am looking for an explanation. I am writing what I understood so that you can correct me. Any help and explanation is highly appreciated thank you for your time.
from sum(1 for _ in group)+1 : Sum 1s for anything that is in group but I think like group is not defined, I don't know if it is something that comes with the library but it is not colored.
from [sum(1 for _ in group)+1 for label, group in groups if label==''][0] I basically can not follow, if label is a empty string but I don't know about [0] at the end.
from itertools import groupby
checkstr = ['AGATC', 'AATG', 'TATC']
s = 'GCTAAATTTGTTCAGCCAGATGTAGGCTTACAAATCAAGCTGTCCGCTCGGCACGGCCTACACACGTCGTGTAACTACAACAGCTAGTTAATCTGGATATCACCATGACCGAATCATAGATTTCGCCTTAAGGAGCTTTACCATGGCTTGGGATCCAATACTAAGGGCTCGACCTAGGCGAATGAGTTTCAGGTTGGCAATCAGCAACGCTCGCCATCCGGACGACGGCTTACAGTTAGTAGCATAGTACGCGATTTTCGGGAAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGTATCTATCTATCTATCTATCT'
for c in checkstr:
groups = groupby(s.split(c))
try:
print(c,[sum(1 for _ in group)+1 for label, group in groups if label==''][0])
except IndexError:
print(c,0)
print(sum(1 for _ in group)+1)

I have broken down the list comprehension into a few steps to make the program flow clear.
Make sure that you comment out your method when using my method.For some odd reason I couldn't get both methods to work together.
from itertools import groupby
checkstr = ['AGATC', 'AATG', 'TATC']
s = 'GCTAAATTTGTTCAGCCAGATGTAGGCTTACAAATCAAGCTGTCCGCTCGGCACGGCCTACACACGTCGTGTAACTACAACAGCTAGTTAATCTGGATATCACCATGACCGAATCATAGATTTCGCCTTAAGGAGCTTTACCATGGCTTGGGATCCAATACTAAGGGCTCGACCTAGGCGAATGAGTTTCAGGTTGGCAATCAGCAACGCTCGCCATCCGGACGACGGCTTACAGTTAGTAGCATAGTACGCGATTTTCGGGAAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGTATCTATCTATCTATCTATCT'
for c in checkstr:
groups = groupby(s.split(c))
try:
"""
print(c,[sum(1 for _ in group)+1 for label, group in groups if label==''][0])
"""
#same as
my_list = []
for label, group in groups:
if label == '':
for _ in group:
my_list.append(1)
print(c,sum(my_list)+1)
except IndexError:
print(c,0)
#print(sum(1 for _ in group)+1)
I get almost the same output.
But my method gives 1 as the output for 'AGATC'.
I can't get it to break from try and get it into the except.I tried few other methods too.This was the best way i could structure it to make what happens in list comprehension clear.
Hope this helps you clear your doubt.
EDIT
The accuracy of the code kept bothering me because the code you posted in your question returns two words less.This code works perfectly fine.And I have used my analogous form of list comprehension.
from itertools import groupby
checkstr = ['AGATC', 'AATG', 'TATC']
s = 'GCTAAATTTGTTCAGCCAGATGTAGGCTTACAAATCAAGCTGTCCGCTCGGCACGGCCTACACACGTCGTGTAACTACAACAGCTAGTTAATCTGGATATCACCATGACCGAATCATAGATTTCGCCTTAAGGAGCTTTACCATGGCTTGGGATCCAATACTAAGGGCTCGACCTAGGCGAATGAGTTTCAGGTTGGCAATCAGCAACGCTCGCCATCCGGACGACGGCTTACAGTTAGTAGCATAGTACGCGATTTTCGGGAAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGTATCTATCTATCTATCTATCT'
"""
for c in checkstr:
groups = groupby(s.split(c))
try:
print(c,[sum(1 for _ in group)+1 for label, group in groups if label==''][0])
except IndexError:
print(c,0)
print(sum(1 for _ in group)+1)
"""
for c in checkstr:
groups = groupby(s.split(c))
"""
print(c,[sum(1 for _ in group)+1 for label, group in groups if label==''][0])
"""
#same as
my_list = []
for label, group in groups:
if label == '':
for _ in group:
my_list.append(1)
x= sum(my_list)
if x == 0:
print(c,0)
else:
print(c,x+2)
OUTPUT
AGATC 0
AATG 44
TATC 6

You might want to read about list comprehensions.
Lets break it down:
sum(1 for _ in group)+1 is the same as len(group)+1 if group had a __len__ attribute. If we assume we could do len(group) then we could rewrite this comprehension as: [len(group)+1 for label, group in groups if label==''][0]
Lets look at this: [len(group) for label, group in groups], which is just a list of the sizes of every single group in groups. With if label=='' we basically delete all entries from that list, that have an empty string as label. The [0] selects only the first entry.
I words: The size (+1) of the first group that has an empty label.

You can write out the code to make clear what exactly happens:
from itertools import groupby
checkstr = ['AGATC', 'AATG', 'TATC']
s = 'GCTAAATTTGTTCAGCCAGATGTAGGCTTACAAATCAAGCTGTCCGCTCGGCACGGCCTACACACGTCGTGTAACTACAACAGCTAGTTAATCTGGATATCACCATGACCGAATCATAGATTTCGCCTTAAGGAGCTTTACCATGGCTTGGGATCCAATACTAAGGGCTCGACCTAGGCGAATGAGTTTCAGGTTGGCAATCAGCAACGCTCGCCATCCGGACGACGGCTTACAGTTAGTAGCATAGTACGCGATTTTCGGGAAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGAATGTATCTATCTATCTATCTATCT'
for c in checkstr:
slist = s.split(c)
groups = groupby(slist)
try:
# print(c,[sum(1 for _ in group)+1 for label, group in groups if label==''][0])
nameless_list = []
for label, group in groups:
if label=='':
nameless_list.append(sum(1 for _ in group)+1)
print(c, nameless_list[0])
except IndexError:
print(c,0)
print(sum(1 for _ in groups)+1)
The list comprehension creates a list. When label is not empty, you are left with an empty list. Then the first element of the list (hence [0] at the end) is printed. This results in an index-error if the list is empty. This error is caught by the exeption handler which prints a 0 instead of the first list element.

Python Sets vs Hash table Performance for List Comparison Algorithm

I was under the impression that sets in Python were implemented using hash tables. However, in the code below, the sets-based version performs much better than the hash-table version. I guess the sets implementation is quite different to what I've done in count_common_elements_hash_table(). Can anyone please explain the difference in performance?
import random
import time
import decimal
def count_common_elements_sets(l1, l2):
common_elements = set(l1).intersection(l2)
return len(common_elements)
def count_common_elements_hash_table(l1, l2):
table = {}
common_elements = []
for v in l1:
table[v] = True
count = 0
for w in l2:
if table.get(w): # Avoid KeyError that would arise with table[w]
common_elements.append(w)
count += 1
return count
l1 = [random.randint(0, 99) for _ in range(1000)]
l2 = [random.randint(0, 99) for _ in range(1000)]
# Time execution
start = time.perf_counter()
count_common_elements_hash_table(l1, l2)
end = time.perf_counter()
print("Hash table version: {:.7f}".format(end - start))
# Time execution
start = time.perf_counter()
count_common_elements_sets(l1, l2)
end = time.perf_counter()
print("Sets version: {:.7f}".format(end - start))

Distributing numbers in SuDoku

I am creating a SuDuko Generator, and am trying to distribute my 1's to every single box. To do this, I have created a list with two items (acting a coordinates). After the function has run, it returns a list such as [a, 1]. I then want a quick way to assign a variable a1 to 1 (a1 would be the variable I would want if [a, 1] was returned and 1 because I'm trying to distribute my 1's). Is there a way to do this? If so how? Here is the source code:
import random
def func():
rows1 = ['a', 'b', 'c']
rows2 = ['d', 'e', 'f']
rows3 = ['g', 'h', 'i']
columns1 = [1, 2, 3]
columns2 = [4, 5, 6]
columns3 = [7, 8, 9]
letter1 = random.choice(rows1)
number1 = random.choice(columns1)
random1 = [letter1, number1]
rows1.remove(letter1)
columns1.remove(number1)
letter2 = random.choice(rows1)
number2 = random.choice(columns2)
random2 = [letter2, number2]
rows1.remove(letter2)
columns2.remove(number2)
letter3 = random.choice(rows1)
number3 = random.choice(columns3)
random3 = [letter3, number3]
columns3.remove(number3)
letter4 = random.choice(rows2)
random4 = [letter4, number4]
rows2.remove(letter4)
columns1.remove(number4)
letter5 = random.choice(rows2)
number5 = random.choice(columns2)
random5 = [letter5, number5]
rows2.remove(letter5)
columns2.remove(number5)
letter6 = random.choice(rows2)
number6 = random.choice(columns3)
random6 = [letter6, number6]
columns3.remove(number6)
letter7 = random.choice(rows3)
number7 = random.choice(columns1)
random7 = [letter7, number7]
rows3.remove(letter7)
letter8 = random.choice(rows3)
number8 = random.choice(columns2)
random8 = [letter8, number8]
rows3.remove(letter8)
letter9 = random.choice(rows3)
number9 = random.choice(columns3)
random9 = [letter9, number9]
return (random1, random2, random3) #etc
Thanks in advance

I then want a quick way to assign a variable a1 to 1 (a1 would be the variable I would want if [a, 1] was returned and 1 because I'm trying to distribute my 1's). Is there a way to do this?
Yes, there is a way to do this, but no, you do not want to. The right thing to do is this:
Instead of creating 81 different variables, create one data structure, which can be indexed.
The most obvious choice is a dict that uses (row, col) pairs as keys.
First, for a dict:
cells = {}
cells['a', '1'] = 1
The downside here is that the cells don't exist until you fill them in. You can pre-fill them like this:
cells = {(letter, digit): None for letter in 'abcdefghi' for digit in '123456789'}
Or you can use a defaultdict, or use the setdefault function in place of [].
You could also use some kind of explicit 2D object, like a list of lists, or a numpy.array, etc., but then you'll need some way to convert a letter and digit into a pair of array coordinates:
cells = [[None for _ in range(9)] for _ in range(9)]
def getcell(letter, digit):
return cells[letter - ord('a')][int(digit) - 1]
def setcell(letter, digit, value):
cells[letter - ord('a')][int(digit) - 1] = value
       
You may realize that this is starting to look complex enough that it might be worth wrapping up in a class. If so, you're right. Something like this:
class Board(object):
def __init__(self):
self.cells = [[None for _ in range(9)] for _ in range(9)]
def map_coords(self, letter, digit):
return letter - ord('a'), int(digit) - 1
def getcell(self, letter, digit):
row, col = self.map_coords(letter, digit)
return self.cells[row][col]
    def setcell(self, letter, digit, value):
row, col = self.map_coords(letter, digit)
self.cells[row][col] = value
Now you can just do this:
board = Board()
board.setcell('a', '1') = 1

editing a list of lists python3

I have the following code
class Board:
def __init__(self, size=7):
self._size = size
self._list, self._llist =[],[]
for i in range (self._size):
self._list.append('_ ')
for j in range(self._size):
self._llist.append(self._list)
def printboard(self):
for i in range(self._size):
for j in range(self._size):
print(self._llist[i][j], end = ' ')
print('\n')
def updateboard(self,x,y,letter):
self._llist[x][y]=letter
self.printboard()
board = Board(3)
board.updateboard(0,0,'c')
and this prints
c _ _
c _ _
c _ _
instead of
c _ _
_ _ _
_ _ _
I can't see what is going wrong. Also, is there a simpler way to create the list of lists dynamically?
Thanks!

You are creating llist with the same list object, repeated multiple times. If you want each list in llist to be a separate, independent object (so that when you modify the contents only one list is changed) then you need to append a different copy to each. The easiest way to do this is to change:
self._llist.append(self._list)
to
self._llist.append(list(self._list))
Simpler code would be:
self._list = ['_ '] * self._size
self._llist = [list(self._list) for i in range(self._size)]