I have multiple functions I repeatedly want to measure execution time for using the builtin timeit library. (Say fun1, fun2 and fun3 are all depending on a couple subroutines, some of which I am trying to optimize. After every iteration, I want to know how fast my 3 top-level functions are executing)
The thing is, I am not sure in advance how long the functions are going to run, I just have a rough estimate. Using timeit.repeat(...) with a sufficient amount of repetitions/number of execution gives me a good estimate, but sometimes it takes very long because I accidentally slowed down one of the subroutines. It would be very handy to have a tqdm-like progress bar for the timing routine so I can estimate in advance for how long I have to wait until timing is done. I did not find any such feature in the timeit library, so here is the question:
Is it possible to show a (tqdm-like) progress bar when timing functions using timeit.repeat or timeit.timeit?
You can create your subclass of timeit.Timer that uses tqdm to track the total iterations performed.
from timeit import Timer, default_number
from tqdm import tqdm
import itertools
import gc
class ProgressTimer(Timer):
def timeit(self, number=default_number):
"""Time 'number' executions of the main statement.
To be precise, this executes the setup statement once, and
then returns the time it takes to execute the main statement
a number of times, as a float measured in seconds. The
argument is the number of times through the loop, defaulting
to one million. The main statement, the setup statement and
the timer function to be used are passed to the constructor.
"""
# wrap the iterator in tqdm
it = tqdm(itertools.repeat(None, number), total=number)
gcold = gc.isenabled()
gc.disable()
try:
timing = self.inner(it, self.timer)
finally:
if gcold:
gc.enable()
# the tqdm bar sometimes doesn't flush on short timers, so print an empty line
print()
return timing
To use this object, we just need to pass in the script we want to run. You can either define it as a string (like below) or you can simply open the file for reading and read to a variable.
py_setup = 'import numpy as np'
py_script = """
x = np.random.rand(1000)
x.sum()
"""
pt = ProgressTimer(py_script, setup=py_setup)
pt.timeit()
# prints / returns:
100%|███████████████████████████████████████████████| 1000000/1000000 [00:13<00:00, 76749.68it/s]
13.02982600001269
Looking at the source code of timeit, there is a template that gets executed when any timing is done. One could simply change that template to include a progress indicator:
import timeit
timeit.template = """
def inner(_it, _timer{init}):
from tqdm import tqdm
{setup}
_t0 = _timer()
for _i in tqdm(_it, total=_it.__length_hint__()):
{stmt}
_t1 = _timer()
return _t1 - _t0
"""
# some timeit test:
timeit.timeit(lambda: "-".join(map(str, range(100))), number=1000000)
Of course, this will influence the result, because the tqdm-calls are inside the _t0 and _t1 measurements. tqdm's documentation claims, that the overhead is only 60ns per iteration, though.
Related
When I run code without lru_cache I get this result. Which is understandable
with multiprocessing
time took 0.4375
without multiprocessing
time took8.8125
But when I run using lru_cache this is the result:
Test1
with multiprocessing
time took 0.34375
without multiprocessing
time took 0.3125
Test2
with multiprocessing
time took 3.234375
without multiprocessing
time took 3.046875
He we can clearly see without multiprocessing is almost equal or little faster than multiprocessing method. What's the reason for this? I understand that creating process is overhead but work list is very huge (10 million) so I guess chunk size is not too small. Or am I doing this wrong way?
Code explanation:
oddlist () take number and return the sum of all odd nums in that range
oddcount is a tuple contains 10 million random numbers
Code:
import os
from random import randint
from functools import reduce
from operator import add
from multiprocessing import Pool
import time
from functools import lru_cache
#lru_cache(maxsize=None)
def oddlist(num):
return reduce(add,(i for i in range(num) if i&1))
if __name__ == '__main__':
oddcounts=tuple(randint(10,50) for i in range(10000000))
print('with multiporcessing')
s=time.process_time()
with Pool(12) as p:
mp=p.map(oddlist, oddcounts)
e=time.process_time()
print(f'time took {e-s}')
print('witout multiporcessing')
s=time.process_time()
z=tuple(oddlist(i) for i in oddcounts)
e=time.process_time()
print(f'time took {e-s}')
Each process has its own cache, so while using multiprocessing, caching is 1/12th as effective as it would otherwise be. There are only 40 possible input values to oddlist. In the multiprocessing case, each process computes all 40, then uses the cache. Without multiprocessing, all 40 are only computed once. So, in addition to the overhead of starting the processes, each process does more work than it would need to if caching were working as intended. Also, there is a cost to pass the work to be done in each process to it, and passing the result back.
I am new to Python multiprocessing. I have a function that returns values and is supposed to act in parallel. In the following, you can find a sample code.
import multiprocessing as mp
from tqdm import tqdm
def foo(self):
arg_triplets = [(self.loc_x[ii], self.loc_y[jj], self.arg)
for ii in np.arange(0, self.nx) for jj in np.arange(0, self.ny)]
ctx = mp.get_context('fork')
max_proc = mp.cpu_count()-1
pool = ctx.Pool(processes=max_proc)
return_values = list(tqdm(pool.imap(target_foo, arg_triplets), total=nx*ny))
pool.close()
pool.join()
So, when I run this routine once, everything works fine. The function target_foo takes triplets of arguments and returns all the output values as a list. I can monitor the status of my 8 core processor, and see 7 of them are working simultaneously. But the problem starts when I use function foo in a for loop. For example, I need to gather data for multiple foo which I do not need in parallel. So I create a for loop that calls foo sequentially. In each call of foo, the function target_foo is supposed to work in parallel. The problem is that for the first time it works on parallel, but from the second time, it does not. What am I doing wrong?
I'm using python timeit to see how long it takes for a function to run.
setup = '''from __main__ import my_func
import random
l = list(range(10000))
random.shuffle(l)'''
timeit.timeit('my_func(l)', setup=setup, number=1000)
the results I'm getting are bigger than a 'normal' check with datetime.
Does timeit also count the time the setup takes, and if so - how can I disable it?
Does my_func(l) mutate l? That could affect the timings.
timeit will run the setup once and reuse the objects created by the setup each time it calls the code that is to be timed. Also it can call the code a few times to gauge roughly how fast it runs and choose the number of iterations before the actual timed run (though not when you've specified the number of runs yourself). That would mean if there is an initial fast run it won't be included in the timed results.
For example if my_func() was a badly written quicksort function it might run quickly when you call it on a shuffled list and very very slowly when you call it again with a (now sorted) list. timeit would only measure the very slow calls.
The docs say:
The execution time of setup is excluded from the overall timed
execution run.
The Python 2.0 docs are pretty clear that the setup statement is not timed:
Time number executions of the main statement. This executes the setup
statement once, and then returns the time it takes to execute the main
statement a number of times, measured in seconds as a float.
But if you're not sure, put a big, slow process into the setup statement and test to see what difference it makes.
To time a python function, the most common approach seems to involve the 'timeit' module, which returns the time that it took to run the function, but does not return the output of the function.
Is there a module which does the same thing as timeit, but which returns the output of the function, in addition to the running time, or is it necessary to implement that manually? If implementing this manually, what's a good timing function for this purpose, which is reasonably accurate, and which doesn't have a lot of overhead (options include, os.times(), datetime.now(), etc.)?
Likely a number of approaches to this problem, but here are two you may consider:
Run the function and store its output in a variable. Print the time.clock time after the function completes, but immediately before returning the output stored at the variable. The time complexity of the return statement is negligible w/r/t the function.
The above approach may be inappropriate if you are, say, comparing several implementations for both correctness and runtime. In that case, consider returning the function's output and the time.clock output in a list, which can then be accessed, stored in a struct, etc. Again, the function itself will majorize vs. the list operations and return.
As per the comment, use time.clock to get processor time precision.
You can try using time.time():
def timefunc(func):
from time import time
then = time()
func()
print time() - then
As such:
def foo():
from sys import stdout
from time import sleep
for i in range(1, 11):
stdout.write("\r%d" % i)
stdout.flush()
sleep(0.1)
stdout.write("\n")
>>> timefunc(foo)
10
1.01269602776
>>> timefunc(foo)
10
1.00967097282
>>> timefunc(foo)
10
1.01678395271
>>>
In Matlab there is
"timeit(F), which measures the typical time (in seconds) required to run the function specified by the function handle F that takes no input argument."
This method returns the median of (I think 13) runs of the function.
Having looked at the methods time and timeit in Python, I can't quite find anything that will let me call from an IPython console, time my script (not function) a number of times, and return either an average or median.
Is there an easy way to do this? or at least time 1 execution, whereby I can make my own loop and average?
Thanks
You may want to look at this link and consider the %timeit magic from IPython
link
Example:
Say you define a function you want to test:
def logtest1(N):
tr=0.
for i in xrange(N):
T= 40. + 10.*random()
tr = tr + -log(random())/T
from timeit import Timer, timeit, repeat
runningtime = repeat("logtest1(int(10e5))", setup="from __main__ import logtest1", repeat=5, number=1)
print (runningtime)
That will run my function logtest1(int(10e5)) 1 time and store the time in the list runningtime then it will repeat the same thing 5 times and store the results in the same list. You can then take the average of the median of that list.