Good Evening,
I am trying to estimate the remaining time to the end of a loop; I've used:
start = datetime.now()
progress = 0
for i in range(1000):
#do a few calculations
progress += 1
stop = datetime.now()
execution_time = stop-start
remaining = execution_time * ( 1000 - progress )
print("Progress:", progress, "%, estimated", remaining, "time remaining")
But it does not seem to work properly, since it goes up to minutes, even though the loop would take 20 seconds in total, and decrease quickly when reaching the end.
How can I try to forecast the remaining time of a loop efficiently and correctly?
Simply use tqdm package:
from tqdm import tqdm
for i in tqdm(range(10000)):
dosomthing()
It will print everything for you:
76%|█████████████ | 7568/10000 [00:33<00:10, 229.00it/s]
Rather than using datetime.datetime.now() for this sort of thing you can use time.perf_counter(), which is available in Python 3.3+. From the docs:
Return the value (in fractional seconds) of a performance counter,
i.e. a clock with the highest available resolution to measure a short
duration. It does include time elapsed during sleep and is
system-wide. The reference point of the returned value is undefined,
so that only the difference between the results of consecutive calls
is valid.
Also, you can print using a carriage return instead of a newline so that the progress reports are printed on a single line. Here's a brief demo derived from your code.
from time import sleep, perf_counter
fmt = " Progress: {:>3}% estimated {:>3}s remaining"
num = 1000
start = perf_counter()
for i in range(1, num + 1):
# Simulate doing a few calculations
sleep(0.01)
stop = perf_counter()
remaining = round((stop - start) * (num / i - 1))
print(fmt.format(100 * i // num, remaining), end='\r')
print()
Depending on your terminal (and Python version) you may also need to add the flush=True keyword arg to the print call in order to get the progress reports to print as they are issued.
I think that in this line:
remaining = execution_time * ( 1000 - progress )
you should divide execution_time/progress, because you want to know how long it takes to complete one percent of progress.
remaining = execution_time/progress * ( 1000 - progress )
Your calculation for time remaining is wrong. If it takes execution_time for progress steps. Then how much does it take for 1000 steps ?
Simple cross multiply gives you the total time. Subtract it from the time already elapsed and that will give you the time remaining.
remaining_time = execution_time * 1000 / progress - execution_time
percent_complete = (progress / 1000) * 100 #You can simplify this if you like
print("Progress:", percent_complete , "%, Estimated", remaining_time, "time remaining")
Also your variable execution_time_1 is never defined
Related
I am quite new to coding and i am wondering how to make it more efficient. I am running it on a raspberry pi which will have other tasks so i want this to be as easy to run as possible. The code will use a magnetic sensor to record passes made by a magnet mounted on a wheel and from there determine the speed of the outside diameter of the wheel. It would be useful to implement something that takes the five last speed outputs and gives sort of an average but only if it does not affect the complexity of the code much. Would be really greatful for any help!
from gpiozero import Button
import time
global t0
t0 = time.clock()
raduis = 300
button = Button (21)
from signal import pause
def calculate_speed(radius):
global t0
t1 = time.clock
interval = t1 - t0
speed = radius/interval
print (speed, 'mm/sek')
y = True
while y == True:
button.when_pressed = calculate_speed(radius)
time.sleep(0.2)
#used to prevent one pass of the magnet from recording many passes
You should store five last speed outputs in an array (list) and then you can calculate the average speed
speed_records = []
def calculate_speed(radius):
global t0
t1 = time.clock
interval = t1 - t0
speed = radius/interval
print (speed, 'mm/sek')
speed_records.append(speed) # Adds one speed record to the list
if len(speed_records) >= 5: # checks if there are 5 five records available
last_five_records = speed_records[-5:] # Seperates five last records
average = sum(last_five_records) / 5
print('Average Speed:',average) # Prints out the average
if len(speed_records) > 10: # Free Up some memory
speed_records = list(set(speed_records) - set(speed_records[:5])) #removes the first five records
The following code uses modular arithmetic to iterate through a single list, adding and overwriting values, and printing the averaged speed. Adjust iterations to control how many passes to average over.
from gpiozero import Button
from signal import pause
import time
radius = 300
button = Button (21)
iterations = 5
speeds = [0] * iterations
speed_idx = 0
def calculate_speed(radius):
global speeds, speed_idx
t1 = time.time()
speeds[speed_idx] = radius / (t1- t0)
print (sum(speeds) / iterations, 'mm/sek')
speed_idx += 1
speed_idx %= iterations
t0 = time.time()
while True:
button.when_pressed = calculate_speed(radius)
time.sleep(0.2)
t0 = time.time()
Note this takes 5 measurements to "ramp up" in a sense. If you want- you could add an if statement to avoid printing out the first 4 recordings.
Additionally, if you wanted a smoother measurement of speeds, it occurred to me that you could use a single value to hold the sum of the speeds of the last N passes, and each time subtract off the average (assuming N sums), and add the new speed. It would a few extra passes to stabilize, but afterwards it should smooth the reported speeds a bit.
For example if I have:
something=something+1
I want to know how many times in a minute, this line is executed in order to create another variable with this result?
I suppose you are trying to do some basic benchmarking, in this case it would go like this:
import time
start = int(round(time.time() * 1000))
something = 0
while something < 1000000:
something = something + 1
delta = int(round(time.time() * 1000)) - start
print "loop ran 1000000 times in {0:0d} milliseconds".format(delta)
If you are willing to wait a full minute (which usually is not the case) you could do something like
import time
start = time.time()
operationsPerMinute = 0
while (time.time() - start < 60):
operationsPerMinute = operationsPerMinute + 1
print(operationsPerMinute)
In which case operationsPerMinute holds the value you want.
Alternatively, you could run in a much smaller time frame and use a bit of math to estimate it in a whole minute time frame.
For benchmarking you would probably ask for per second timings.
To count events in the last minute, here is a class remembering event timestamps for a given period:
import bisect
import time
class TimedCounter:
def __init__(self, period=60.0):
self._timestamps = []
self._period = period
def _cleanup(self, now):
del self._timestamps[:bisect.bisect_left(self._timestamps, now - self._period)]
def increment(self):
now = time.time()
self._timestamps.append(now)
self._cleanup(now) # optimization: not necessary to call every time
def get_count(self):
self._cleanup(time.time())
return len(self._timestamps)
and an example:
tc = TimedCounter(1.0)
for i in range(7, 27):
tc.increment()
print("{} time(s) in the last second".format(tc.get_count()))
time.sleep(7/i)
I'm trying to generate some random seeded times to tell my script when to fire each of the scripts from within a main script.
I want to set a time frame of:
START_TIME = "02:00"
END_TIME = "03:00"
When it reaches the start time, it needs to look at how many scripts we have to run:
script1.do_proc()
script2.alter()
script3.noneex()
In this case there are 3 to run, so it needs to generate 3 randomized times to start those scripts with a minimum separation of 5 mins between each script but the times must be within the time set in START_TIME and END_TIME
But, it also needs to know that script1.main is ALWAYS the first script to fire, other scripts can be shuffled around (random)
So we could potentially have script1 running at 01:43 and then script3 running at 01:55 and then script2 might run at 02:59
We could also potentially have script1 running at 01:35 and then script3 running at 01:45 and then script2 might run at 01:45 which is also fine.
My script so far can be found below:
import random
import pytz
from time import sleep
from datetime import datetime
import script1
import script2
import script3
START_TIME = "01:21"
END_TIME = "03:00"
while 1:
try:
# Set current time & dates for GMT, London
CURRENT_GMTTIME = datetime.now(pytz.timezone('Europe/London')).strftime("%H%M")
CURRENT_GMTDAY = datetime.now(pytz.timezone('Europe/London')).strftime("%d%m%Y")
sleep(5)
# Grab old day for comparisons
try:
with open("DATECHECK.txt", 'rb') as DATECHECK:
OLD_DAY = DATECHECK.read()
except IOError:
with open("DATECHECK.txt", 'wb') as DATECHECK:
DATECHECK.write("0")
OLD_DAY = 0
# Check for new day, if it's a new day do more
if int(CURRENT_GMTDAY) != int(OLD_DAY):
print "New Day"
# Check that we are in the correct period of time to start running
if int(CURRENT_GMTTIME) <= int(START_TIME.replace(":", "")) and int(CURRENT_GMTTIME) >= int(END_TIME.replace(":", "")):
print "Correct time, starting"
# Unsure how to seed the start times for the scripts below
script1.do_proc()
script2.alter()
script3.noneex()
# Unsure how to seed the start times for above
# Save the current day to prevent it from running again today.
with open("DATECHECK.txt", 'wb') as DATECHECK:
DATECHECK.write(CURRENT_GMTDAY)
print "Completed"
else:
pass
else:
pass
except Exception:
print "Error..."
sleep(60)
EDIT 31/03/2016
Let's say I add the following
SCRIPTS = ["script1.test()", "script2.test()", "script3.test()"]
MAIN_SCRIPT = "script1.test()"
TIME_DIFFERENCE = datetime.strptime(END_TIME, "%H:%M") - datetime.strptime(START_TIME, "%H:%M")
TIME_DIFFERENCE = TIME_DIFFERENCE.seconds
We now have the the number of scripts to run
We have the list of the script to run.
We have the name of the main script, the one to run first.
We have the time in seconds to show how much time we have in total to run all the scripts within.
Surely there is a way we can just plug some sort of loop to make it do it all..
for i in range(len(SCRIPTS)), which is 3 times
Generate 3 seeds, making sure the minimum time is of 300 and all together the 3 seeds must not exceed TIME_DIFFERENCE
Create the start time based on RUN_TIME = START_TIME and then RUN_TIME = RUN_TIME + SEED[i]
First loop would check that that MAIN_SCRIPT exists within SCRIPTS, if it does then it would run that script first, delete itself from SCRIPTS and then on next loops, as it doesn't exist in SCRIPTS it would switch to randomly calling one of the other scripts.
Seeding the times
The following appears to work, there might be an easier way of doing this though.
CALCULATE_SEEDS = 0
NEW_SEED = 0
SEEDS_SUCESSS = False
SEEDS = []
while SEEDS_SUCESSS == False:
# Generate a new seed number
NEW_SEED = random.randrange(0, TIME_DIFFERENCE)
# Make sure the seed is above the minimum number
if NEW_SEED > 300:
SEEDS.append(NEW_SEED)
# Make sure we have the same amount of seeds as scripts before continuing.
if len(SEEDS) == len(SCRIPTS):
# Calculate all of the seeds together
for SEED in SEEDS:
CALCULATE_SEEDS += SEED
# Make sure the calculated seeds added together is smaller than the total time difference
if CALCULATE_SEEDS >= TIME_DIFFERENCE:
# Reset and try again if it's not below the number
SEEDS = []
else:
# Exit while loop if we have a correct amount of seeds with minimum times.
SEEDS_SUCESSS = True
Use datetime.timedelta to compute time differences. This code assumes all three processes run on the same day
from datetime import datetime, timedelta
from random import randint
YR, MO, DY = 2016, 3, 30
START_TIME = datetime( YR, MO, DY, 1, 21, 00 ) # "01:21"
END_TIME = datetime( YR, MO, DY, 3, 0, 0 ) # "3:00"
duration_all = (END_TIME - START_TIME).seconds
d1 = ( duration_all - 600 ) // 3
#
rnd1 = randint(0,d1)
rnd2 = rnd1 + 300 + randint(0,d1)
rnd3 = rnd2 + 300 + randint(0,d1)
#
time1 = START_TIME + timedelta(seconds=rnd1)
time2 = START_TIME + timedelta(seconds=rnd2)
time3 = START_TIME + timedelta(seconds=rnd3)
#
print (time1)
print (time2)
print (time3)
Values of rnd1, rnd2and rnd3 are at least 5 minutes (300 seconds) apart.
Values of rnd3 cannot be greater than the total time interval (3 * d1 + 600). So all three times occur inside the interval.
NB You did not specify how much time each script runs. That is why I did not use time.sleep. A possible option would be threading.Timer (see python documentation).
Assume you store all the method.func() in an array and, as u described, subsequent scripts must be at least 5 mins after script1. They can be executed randomly, so we can launch multiple processes and let them sleep for a period before they can automatically start. (Timing is in seconds)
from multiprocessing import Process
import os
import random
import time
#store all scripts you want to execute here
eval_scripts = ["script1.test()","script2.test()", "script3.test()"]
#run job on different processes. non-blocking
def run_job(eval_string,time_sleep):
#print out script + time to test
print eval_string + " " + str(time_sleep)
time.sleep(time_sleep) #wait to be executed
#time to start
eval(eval_string)
def do_my_jobs():
start_time = []
#assume the duration between start_time and end_time is 60 mins, leave some time for other jobs after the first job (5-10 mins). This is just to be careful in case random.randrange returns the largest number
#adjust this according to the duration between start_time and end_time since calculating (end_time - star_time) is trivial.
proc1_start_time = random.randrange(60*60 - 10*60)
start_time.append(proc1_start_time)
#randomize timing for other procs != first script
for i in range(len(eval_scripts)-1):
#randomize time from (proc1_start_time + 5 mins) to (end_time - star_time)
start_time.append(random.randint(proc1_start_time+5*60, 60*60))
for i in range(len(eval_scripts)):
p_t = Process(target = run_job, args = (eval_scripts[i],start_time[i],))
p_t.start()
p_t.join()
Now all you need to do is to call do_my_jobs() only ONCE at START_TIME every day.
I am required to display the time it took to run two different algorithms using functions available in the time library. I'm assuming I have to use the timeit() function however I'm not familiar as to how to incorporate that into the code. So far this is what I have:
import time
def time2Algorithms(sound):
# normalize(sound)
largest = 0
for s in getSamples(sound):
largest = max(largest,getSampleValue(s) )
multiplier = 32767.0 / largest
for s in getSamples(sound):
louder = multiplier * getSampleValue(s)
setSampleValue(s,louder)
explore(sound)
# onlyMaximize(sound)
for sample in getSamples(sound):
value = getSampleValue(sample)
if value >= 0:
setSampleValue(sample,32767)
if value < 0:
setSampleValue(sample,-32768)
explore(sound)
My goal is to display the run times of both the normalize and maximize algorithms after they execute.
Thanks.
The time module (which you are required to use) does not include timeit (different module).
Just add a
start = time.time()
just before the part you want to time, and e.g
print(time.time() - start)
just after said part -- this will display the elapsed time in seconds. Ornament and format that as required, of course:-)
You can use timeit like this
import timeit
start_time = timeit.default_timer()
# Your algo goes here
elapsed = timeit.default_timer() - start_time
and also time module which is easy
import time
start_time = time.time()
# Your algo goes here
elapsed = time.time() - start_time
So I have a script where it measures how fast a person can press the keyboard 100 times. I have used the time module to set the start and the end of the measuring:
import time
import os
start = time.time()
pause() * 100 #defined the definition to os.system("pause")
end = time.time()
How do I make it so python can compare the elapsed time so if the time taken is >20seconds, it performs commands, and if it is equal or less than 20 seconds then preforms other commands?
You mean like this?
elapsed_time = end - start
if elapsed_time > 20:
# code
else:
# other code