Is there any significant difference between:
from time import time
start = time()
# some process
print time() - start
and:
from timeit import timeit
def my_funct():
# some process
print timeit(my_funct, number=1)
For an example, I'll use Project Euler 1 (because it's really easy to understand/solve)
def pE1test1(): # using time()
from time import time
start = time()
print sum([n for n in range(1, 1000) if n%3==0 or n%5==0])
print time() - start
def pE1test2(): # using timeit
print sum([n for n in range(1, 1000) if n%3==0 or n%5==0])
from timeit import timeit
pE1test1()
print timeit(pE1test2, number=1)
This outputs:
>>>
233168
0.0090000629425
233168
0.00513921300363
What is the major difference between timeit and time?
timeit will use the best available timing function on your system. See the docs on timeit.default_timer.
Also, timeit turns off the garbage collector.
Also, I believe you're using timeit wrong. You should be passing a string as per the last example in the documentation:
print timeit("pE1test2()","from __main__ import PE1test2",number=1)
And of course, another major difference is that timeit makes it trivial to time the execution of the function for thousands of iterations (which is the only time a timing result is meaningful). This decreases the importance of a single run taking longer than the others (e.g. due to your system resources being hogged by some other program).
The purposes of the two modules are very different.
The time module provides low-level access to various time/date functions provided by the underlying system.
The timeit module is specifically for running performance tests.
As you point out, you can do simple timing using the functions in time, but there are a number of common pitfalls that people fall into when trying to do performance testing. timeit tries to mitigate those in order to get repeatable numbers that can be sensibly compared.
Related
how make a simple cpython program from scratch and measure it performance gain with its respective python program..
example if I make a program of 10! in python and the in Cpython how would I know how much it improves the performance of computation.
Take a look at : How to use timeit module
simple code to test time taken for a function:
def time_taken(func):
from time import time
start = time()
func()
end = time()
return (end - start)
def your_func():
#your code or logic
# To test
time_taken(your_func)
Use this mechanism to test time taken in both situations.
Let's say CPython took c seconds & Python took p seconds, CPython is faster than Python by: (c - p) * 100 / p
If c is 2 secs & p is 1 sec. CPython is faster than Python by (2-1)*100/1 = 100%
However, the performance keeps changing depending on the code & problem statement.
Share the output. Goodluck!
In Matlab there is
"timeit(F), which measures the typical time (in seconds) required to run the function specified by the function handle F that takes no input argument."
This method returns the median of (I think 13) runs of the function.
Having looked at the methods time and timeit in Python, I can't quite find anything that will let me call from an IPython console, time my script (not function) a number of times, and return either an average or median.
Is there an easy way to do this? or at least time 1 execution, whereby I can make my own loop and average?
Thanks
You may want to look at this link and consider the %timeit magic from IPython
link
Example:
Say you define a function you want to test:
def logtest1(N):
tr=0.
for i in xrange(N):
T= 40. + 10.*random()
tr = tr + -log(random())/T
from timeit import Timer, timeit, repeat
runningtime = repeat("logtest1(int(10e5))", setup="from __main__ import logtest1", repeat=5, number=1)
print (runningtime)
That will run my function logtest1(int(10e5)) 1 time and store the time in the list runningtime then it will repeat the same thing 5 times and store the results in the same list. You can then take the average of the median of that list.
I run test sqript. It use numpy.fft.fft(), anfft.fft() based on FFTW and pyfftw.interfaces.numpy_fft.fft() based on FFTW.
here is source of my test script:
import numpy as np
import anfft
import pyfftw
import time
a = pyfftw.n_byte_align_empty(128, 16, 'complex128')
a[:] = np.random.randn(128) + 1j*np.random.randn(128)
time0 = time.clock()
res1 = np.fft.fft(a)
time1 = time.clock()
res2 = anfft.fft(a)
time2 = time.clock()
res3 = pyfftw.interfaces.numpy_fft.fft(a,threads=50)
time3 = time.clock()
print 'Time numpy: %s' % (time1 - time0)
print 'Time anfft: %s' % (time2 - time1)
print 'Time pyfftw: %s' % (time3 - time2)
and I get these results:
Time numpy: 0.00154248116307
Time anfft: 0.0139805208195
Time pyfftw: 0.137729374893
anfft library produce more faster fft on huge data, but what about pyfftw? why it is so slowly?
In this case, spawning more threads than you have CPU cores will not give an increase in performance, and will probably make the program slower due to the overhead of switching threads. 50 threads is complete overkill.
Try benchmarking with one thread.
The problem here is the overhead in using the numpy_fft interface. Firstly, you should enable the cache with pyfftw.interfaces.cache.enable(), and then test the result with timeit. Even using the cache there is a fixed overhead of using the interfaces that is not present if you use the raw interface.
On my machine, on a 128-length array, the overhead of the interface still slows it down more than numpy.fft. As the length increases, this overhead becomes less important, so on say a 16000-length array, the numpy_fft interface is faster.
There are tweaks you can invoke to speed things up on the interfaces end, but these are unlikely to make much difference in your case.
The best way to get the fastest possible transform in all situations is to use the FFTW object directly, and the easiest way to do that is with the builders functions. In your case:
t = pyfftw.builders.fft(a)
timeit t()
With that I get pyfftw being about 15 times faster than np.fft with a 128 length array.
It might be that pyFFTW is actually spending most of its time planning the transform. Try including for example planner_effort='FFTW_ESTIMATE' in the pyfftw fft call, and see how that affects the performance.
Is there a simple way to time a Python program's execution?
clarification: Entire programs
Use timeit:
This module provides a simple way to time small bits of Python code. It has both command line as well as callable interfaces. It avoids a number of common traps for measuring execution times.
You'll need a python statement in a string; if you have a main function in your code, you could use it like this:
>>> from timeit import Timer
>>> timer = Timer('main()', 'from yourmodule import main')
>>> print timer.timeit()
The second string provides the setup, the environment for the first statement to be timed in. The second part is not being timed, and is intended for setting the stage as it were. The first string is then run through it's paces; by default a million times, to get accurate timings.
If you need more detail as to where things are slow, use one of the python profilers:
A profiler is a program that describes the run time performance of a program, providing a variety of statistics.
The easiest way to run this is by using the cProfile module from the command line:
$ python -m cProfile yourprogram.py
You might want to use built-in profiler.
Also you might want to measure function's running time by using following simple decorator:
import time
def myprof(func):
def wrapping_fun(*args):
start = time.clock()
result = func(*args)
end = time.clock()
print 'Run time of %s is %4.2fs' % (func.__name__, (end - start))
return result
return wrapping_fun
Usage:
#myprof
def myfun():
# function body
If you're on Linux/Unix/POSIX-combatible platform just use time. This way you won't interfere with you script and won't slow it down with unnecessarily detailed (for you) profiling. Naturally, you can use it for pretty much anything, not just Python scripts.
For snippets use the timeit module.
For entire programs use the cProfile module.
Use timeit
>>> import timeit
>>> t = timeit.Timer(stmt="lst = ['c'] * 100")
>>> print t.timeit()
1.10580182076
>>> t = timeit.Timer(stmt="lst = ['c' for x in xrange(100)]")
>>> print t.timeit()
7.66900897026
I would like to know that how much time a particular function has spent during the duration of the program which involves recursion, what is the best way of doing it?
Thank you
The best way would be to run some benchmark tests (to test individual functions) or Profiling (to test an entire application/program). Python comes with built-in Profilers.
Alternatively, you could go back to the very basics by simply setting a start time at the beginning of the program, and, at the end of the program, subtracting the current time from the start time. This is basically very simple Benchmarking.
Here is an implementation from the an answer from the linked question:
import time
start = time.time()
do_long_code()
print "it took", time.time() - start, "seconds."
Python has something for benchmarking included in its standard library, as well.
From the example give on the page:
def test():
"Time me"
L = []
for i in range(100):
L.append(i)
if __name__=='__main__':
from timeit import Timer
t = Timer("test()", "from __main__ import test")
print t.timeit()
Use the profiler!
python -m cProfile -o prof yourscript.py
runsnake prof
runsnake is a nice tool for looking at the profiling output. You can of course use other tools.
More on the Profiler here: http://docs.python.org/library/profile.html