Develop Reference

Trying to add a progress bar as my python program runs

I am a beginner writing a Python code, where the computer generates a random number between 1 and 10, 1 and 100, 1 and 1000, 1 and 10000, 1 and 100000 and so on. The computer itself will guess the random number a number of times (a user input number), and every time there is a count of how many times the computer took to guess the random number. A mean of the count over the number of times will be calculated and put in an array, where matplotlib will generate a graph of x=log10(the upper bounds of the random number range, i.e. 10, 100, 1000,...)
At the moment, I print the log10 of each bound as it is processed, and that has been acting as my progress tracker. But I am thinking of adding my progress bar, and I don't know where to put it so that I can see how much of the overall program has run.
I have added tqdm.tqdm in all sorts of different places to no avail. I am expecting a progress bar increasing as the program runs.
My program is as shown.
# Importing the modules needed
import random
import time
import timeit
import numpy as np
import matplotlib.pyplot as plt
import tqdm
# Function for making the computer guess the number it itself has generated and seeing how many times it takes for it to guess the number
def computer_guess(x):
# Telling program that value "low" exists and it's 0
low = 0
# Telling program that value "high" exists and it's the arbitrary parameter x
high = x
# Storing random number with lower limit "low" and upper limit "high" as "ranno" for the while-loop later
ranno = random.randint(low, high)
# Setting initial value of "guess" for iteration
guess = -1
# Setting initial value of "count" for iteration
count = 1
# While-loop for all guessing conditions
while guess != ranno:
# Condition: As long as values of "low" and "high" aren't the same, keep guessing until the values are the same, in which case "guess" is same as "low" (or "high" becuase they are the same anyway)
if low != high:
guess = random.randint(low, high)
else:
guess = low
# Condition: If the guess if bigger than the random number, lower the "high" value to one less than 1, and add 1 to the guess count
if guess > ranno:
high = guess - 1
count += 1
# Condition: If the guess if smaller than the random number, increase the "low" value to one more than 1, and add 1 to the guess count
elif guess < ranno:
low = guess + 1
count += 1
# Condition: If it's not either of the above, i.e. the computer has guessed the number, return the guess count for this function
else:
return count
# Setting up a global array "upper_bounds" of the range of range of random numbers as a log array from 10^1 to 10^50
upper_bounds = np.logspace(1, 50, 50, 10)
def guess_avg(no_of_guesses):
# Empty array for all averages
list_of_averages = []
# For every value in the "upper_bounds" array,
for bound in upper_bounds:
# choose random number, "ranx", between 0 and the bound in the array
ranx = random.randint(0, bound)
# make an empty Numpy array, "guess_array", to insert the guesses into
guess_array = np.array([])
# For every value in whatever the no_of_guesses is when function called,
for i in np.arange(no_of_guesses):
# assign value, "guess", as calling function with value "ranx"
guess = computer_guess(ranx)
# stuff each resultant guess into the "guess_array" array
guess_array = np.append(guess_array, guess)
# Print log10 of each value in "upper_bound"
print(int(np.log10(bound)))
# Finding the mean of each value of the array of all guesses for the order of magnitude
average_of_guesses = np.mean(guess_array)
# Stuff the averages of guesses into the array the empty array made before
list_of_averages.append(average_of_guesses)
# Save the average of all averages in the list of averages into a single variable
average_of_averages = np.mean(list_of_averages)
# Print the list of averages
print(f"Your list of averages: {list_of_averages}")
# Print the average of averages
print(f"Average of averages: {average_of_averages}")
return list_of_averages
# Repeat the "guess_avg" function as long as the program is running
while True:
# Ask user to input a number for how many guesses they want the computer to make for each order of magnitude, and use that number for calling the function "guess_avg()"
resultant_average_numbers = guess_avg(
int(input("Input the number of guesses you want the computer to make: ")))
# Plot a graph with log10 of the order of magnitudes on the horizontal and the returned number of average of guesses
plt.plot(np.log10(upper_bounds), resultant_average_numbers)
# Show plot
plt.show()
I apologise if this is badly explained, it's my first time using Stackoverflow.

You can define the following progress_bar function, which you will call from wherever you want to monitor the advancement in you code:
import colorama
def progress_bar(progress, total, color=colorama.Fore.YELLOW):
percent = 100 * (progress / float(total))
bar = '█' * int(percent) + '-' * (100 - int(percent))
print(color + f'\r|{bar}| {percent:.2f}%', end='\r')
if progress == total:
print(colorama.Fore.GREEN + f'\r|{bar}| {percent:.2f}%', end='\r')
Hope this helps

You can also call tqdm by hand and then update it manually.
progress_bar = tqdm.tqdm(total=100)
progress_bar.update()
When you are finished, you can call progress_bar.clear() to start again.

You probably want two progress bars in the guess_avg() function. One to track the ranges and another to track the guesses.
In this example I've used the Enlighten progress bar library, but you can accomplish similar behavior with other libraries that support nested progress bars. One advantage Enlighten is going to have over others is you can print whatever you want while the progress bar is running, good for debugging.
You can make this simpler by using context managers and auto-updating counters, but I didn't do that here to make it clearer what's happening. You can also customize the template used for the progress bar.
import enlighten
def guess_avg(no_of_guesses):
# Empty array for all averages
list_of_averages = []
# For every value in the "upper_bounds" array,
# Create a progress bar manager manager
manager = enlighten.get_manager(leave=False)
# Create main progress bar for ranges
pbar_bounds = manager.counter(total=len(upper_bounds), desc='Bound ranges', unit='ranges')
for bound in upper_bounds:
# choose random number, "ranx", between 0 and the bound in the array
ranx = random.randint(0, bound)
# make an empty Numpy array, "guess_array", to insert the guesses into
guess_array = np.array([])
# For every value in whatever the no_of_guesses is when function called,
# Create nested progress bar for guesses, leave removes progress bar on close
pbar_guesses = manager.counter(total=no_of_guesses, desc='Guessing', unit='guesses', leave=False)
for i in np.arange(no_of_guesses):
# assign value, "guess", as calling function with value "ranx"
guess = computer_guess(ranx)
# stuff each resultant guess into the "guess_array" array
guess_array = np.append(guess_array, guess)
pbar_guesses.update() # Update nested progress bar
pbar_guesses.close() # Close nested progress bar
# Print log10 of each value in "upper_bound"
print(int(np.log10(bound))) # You can remove this now if you want
pbar_bounds.update() # Update main progress bar
# Finding the mean of each value of the array of all guesses for the order of magnitude
average_of_guesses = np.mean(guess_array)
# Stuff the averages of guesses into the array the empty array made before
list_of_averages.append(average_of_guesses)
manager.stop() # Stop the progress bar manager
# Save the average of all averages in the list of averages into a single variable
average_of_averages = np.mean(list_of_averages)
# Print the list of averages
print(f"Your list of averages: {list_of_averages}")
# Print the average of averages
print(f"Average of averages: {average_of_averages}")
return list_of_averages

Segmenting a list of data; python

I am trying to write a code that takes a list flow_rate, changes it into a segmented list list_segmented of length segment_len. Then with that segmented list, I take each index and make it a list of data_segment.
I am getting stuck trying to figure out how to make each list_segmented[i] = data_segment. The last part of the code calls another function for data_segment in which I have previously written and can import it.
Appreciate your help.
def flow_rate_to_disorder_status(flow_rate,segment_len,interval,threshold):
inlist = flow_rate[:]
list_segmented = []
disorder_status = []
while inlist:
list_segmented.append(inlist[0 : segment_len])
inlist[0 : segment_len] = []
for i in range(0, len(list_segmented)):
data_segment = list_segmented[i]
condition = sym.has_symptom(data_segment, interval, threshold)
disorder_status.append(condition)
Initial function:
def has_symptom(data_segment,interval,threshold):
max_ratio = 1 # maximum ratio allowed when dividing
# data points in interval by len data_segment
# for our example it is 1
# NOTE: max_ratio can NOT be less than threshold
# to define the range of the given interval:
min_interval = interval[0]
max_interval = interval[1]
# create an empty list to add to data points that fall in the interval
symptom_yes = []
# create a loop function to read every point in data_segment
# and compare wether or not it falls in the interval
for i in range(0, len(data_segment)):
if min_interval <= data_segment[i] <= max_interval:
# if the data falls in interval, add it to list symptom_yes
symptom_yes.append(data_segment[i])
# to get the fraction ration between interval points and total data points
fraction_ratio = len(symptom_yes) / len(data_segment)
# if the ratio of data points that fall in interval to total points in
# data segments is more than or equal to threshold and less than or equal
# to max_ratio (1 in our case) then apply condition
if threshold <= fraction_ratio <= max_ratio:
condition = True # entire segment has the symptom
else:
condition = False # entire segment does NOT have the symptom
return condition

You nearly did it:
for i in range(0, len(data_segment)): # <-- looping thru data_segment
# data_segment = list_segmented[i] <-- this was back to front.
list_segmented[i] = data_segment # <-- this will work
note: there are cleaner ways of doing this in python (like list comprehension).
Anyway, good question. Hope that helps.

It looks like the lines
condition = sym.has_symptom(data_segment, interval, threshold)
disorder_status.append(condition)
should each be indented by one more level to be inside the for loop, so that they are executed for each data segment.
You presumably also want to return disorder_status at the end of the function.

Trying to calculate EMA using python and i cant figure out why my code is always producing the same result

I am trying to calculate an exponential moving average of bitcoin in python2.7 but my result is always the same value and I have no idea why.
def calcSMA(data,counter,timeframe):
closesum = 0
for i in range(timeframe):
closesum = float(closesum) + float(data[counter-i])
return float(closesum / timeframe)
def calcEMA(price,timeframe,prevema):
multiplier = float(2/(timeframe+1))
ema = ((float(price) - float(prevema))*multiplier) + float(prevema)
return float(ema)
counter = 0
closeprice = [7242.4,7240,7242.8,7253.8,7250.6,7255.7,7254.9,7251.4,7234.3,7237.4
,7240.7,7232,7230.2,7232.2,7236.1,7230.5,7230.5,7230.4,7236.4]
while counter < len(closeprice):
if counter == 3:
movingaverage = calcSMA(closeprice,counter,3)
print movingaverage
if counter > 3:
movingaverage = calcEMA(closeprice[counter],3,movingaverage)
print movingaverage
counter +=1
This is how to calculate the EMA:
{Close - EMA(previous day)} x multiplier + EMA(previous day)
you seed the formula with a simple moving average.
Doing this in Excel works so might it be my use of variables?
I would be really glad if someone could tell me what I am doing wrong because I have failed on this simple problem for hours and can't figure it out I've tried storing my previous ema in a separate variable and I even stored all of them in a list but I am always getting the same values at every timestep.

The expression 2/(timeframe+1) is always zero, because the components are all integers and therefore Python 2 uses integer division. Wrapping that result in float() does no good; you just get 0.0 instead of 0.
Try 2.0/(timeframe+1) instead.

Execute if statement only once in a period of time in Python?

Creating a python script which receives data from an arduino distance sensor. I am receiving a value each nanosecond. Whenever this value is higher than 50, I want to warn the user. (will eventually do this with a notification program, but for now I am just printing warning). I have the following:
while 1: # Keep receiving data
data = ser.readline() # Getting data from arduino
value = [int(s) for s in data.split() if s.isdigit()] # Converting the Arduino ouput to only get the integer value
if value: # Avoid empty values
distance = value[0] # List to int
if distance > 50: # If bigger than 50 cm, warn user.
warn_user()
I only want to execute the warn_user() function once in 30 seconds, after that, the if statement shouldn't trigger anymore, only when the values drop under 50 and THEN > 50 again. I tried working with True/False statements, timer sleeps but this did not work. Any tips? thanks.

You just have to add some more logical conditions to control the program's flow. Something like this would work:
from time import time
warning_enabled = True
time_of_warning = 0
while 1:
data = ser.readline()
value = [int(s) for s in data.split() if s.isdigit()]
if value:
distance = value[0]
if distance > 50 and warning_enabled:
warn_user()
warning_enabled = False
time_of_warning = time()
if time() - time_of_warning > 30 and not warning_enabled and distance < 50:
warning_enabled = True
What this does is that it keeps track of the last time when the warning was fired and uses the warning_enable flag to make the second if only fire when possible.
Cheers

You need only track what you're looking for to accomplish your goal: the timestamp of the last warning and whether the distance was below the value you're tracking.
import time
distance_was_safe = True # tracks if distance went below specified range
last_warned = 0 # tracks timestamp since last warning
safe_distance = 50 # 50 cm
warn_interval = 30 # 30 sec, warn no more than once every interval
while True:
# Get data from Arduino.
data = ser.readline()
# Convert the Arduino output to only get the integer values.
value = [int(s) for s in data.split() if s.isdigit()]
# Avoid empty output.
if value:
# Get the first integer.
distance = value[0]
# If greater than safe zone, potentially warn the user.
if distance > safe_distance:
# Warn the user if distance was below range,
# and it has been enough time since the last warning.
if distance_was_safe and time.time() - last_warned > warn_interval:
distance_was_safe = False
last_warned = time.time()
warn_user()
else:
# Distance was less than warning distance, reset the flag.
distance_was_safe = True

How to avoid division by zero error, when performing calculations on parsed xml data

please be kind with your answers I have been coding now for 10 days. I am having trouble with performing loops in my code, but I am fairly certain this is because I am getting a traceback.
I parse an xml file obtained from a url, using the following code:
pattern4 = re.compile('title=\'Naps posted: (.*) Winners:')
pattern5 = re.compile('Winners: (.*)\'><img src=')
for row in xmlload1['rows']:
cell = row["cell"]
##### defining the Keys (key is the area from which data is pulled in the XML) for use in the pattern finding/regex
user_delimiter = cell['username']
selection_delimiter = cell['race_horse']
##### the use of the float here is to make sure the result of the strike rate calculations returns as a decimal, otherwise python 2 rounds to the nearest integer!
user_numberofselections = float(re.findall(pattern4, user_delimiter)[0])
user_numberofwinners = float(re.findall(pattern5, user_delimiter)[0])
strikeratecalc1 = user_numberofwinners/user_numberofselections
strikeratecalc2 = strikeratecalc1*100
##### Printing the results of the code at hand
print "number of selections = ",user_numberofselections
print "number of winners = ",user_numberofwinners
print "Strike rate = ",strikeratecalc2,"%"
print ""
getData()
This code with the rest of the code returns:
number of selections = 112.0
number of winners = 21.0
Strike rate = 18.75 %
number of selections = 146.0
number of winners = 21.0
Strike rate = 14.3835616438 %
number of selections = 163.0
number of winners = 55.0
Strike rate = 33.7423312883 %
Now the results of this xmlload suggest that there is only three users to parse however there is a 4th whos data would read
number of selections = 0
number of winners = 0
Strike rate = 0
for my purposes it is not necessary to pull in user stat's for those with no track record, how do i make the code either skip users with 0 selections or at least make it so that the division by zero error doesn't affect the ability of the code to run in a loop?
Kind regards!

Just use a continue when you find a 0.
user_numberofwinners = float(re.findall(pattern5, user_delimiter)[0])
# if the number of winners is 0, go to the next row to avoid division by 0
if user_numberofwinners == 0.0 : continue;