Consider this code:
import matplotlib.pyplot as plt
# Returns 2^n
def pow(n):
if n == 0:
return 1
x = pow(n//2)
if n%2 == 0:
return x*x
return 2*x*x
y = [10^4, 10^5,10^6, 10^7, 10^8, 10^9, 10^10]
z = []
for n in y:
start = time.time()
pow(n)
print(n, time.time() - start) # elapsed time
z.append(time.time()-start)
plt.plot(y,z)
plt.show()
I am trying to figure out what's the time complexity of the recursive
function pow(n).
I calculated the time complexity as O(log(n)) but when using the function
time.time() the function appears to be linear. How come?
Why is the time complexity O(n) and not O(log(n))?
If you replace all appearances of x*x in your example with a constant (e.g. 1) or multiplication with addition, you will see that you indeed get O(log(n)) complexity, as your function is invoked log(n) times (though we measure really small times in this case, so the results of using time might not reflect the complexity of the function in this case). I think the conclusion is that your assumption that multiplication is O(1) is not correct (see e.g. this question), in particular as the numbers you multiply are really large and don't fit anymore in a traditional 32/64-bit representation.
Related
I tried check memory complexity for recursive and iteration program computing factorial.
I assume that solution is O(N), not interesting how implemented is multiplication.
But when I compared time of solution for two cases recursive and iteration I had different results.
Time of execution of factorial (loop) : 521.7680931091309
Time of execution (recursive) : 1214.137077331543
import time
# O(n) complexity
def factorial(n):
result = 1
for i in range(1, n+1):
result *= i
return result
# O(n) complexity
def factorial_rec(n):
if n <= 1:
return 1
else:
return n * factorial_rec(n-1)
MAX_N = 100
COUNT = 100000
t1 = time.time()
for _ in range(0, COUNT):
factorial(MAX_N)
t2 = time.time()
print(f"Time of execution of factorial (loop) : {(t2-t1)*1000} ")
t1 = time.time()
for _ in range(0, COUNT):
factorial_rec(MAX_N)
t2 = time.time()
print(f"Time of execution (recursive) : {(t2-t1)*1000} ")
Where is the differences between implementation of recursive in Python and iteration process?
The recursive solution runs slower than the nonrecursive solution. This is to be expected. The recursive solution involves a nontrivial amount of function-call overhead.
It is not true that two O(n) algorithms will have the same run-time. They have the same complexity, but the definition of O(n) involves an unspecified constant of proportionality (which in the linear case is simply the slope of the line). Not all lines have the same slope. The slope for the recursive case is just bigger than the slope for the nonrecursive case. Think of it this way. If n = 10, the recursive function requires 10 function calls. If n = 20, it requires 20 function calls, etc. Note how that goes up linearly and is thus part of the slope. There is nothing which corresponds to this cost in the nonrecursive case. Other than the function call overhead, the computation is mostly just multiplication (which is the same in both cases) -- so the recursive function is almost obviously less efficient. It incurs a cost that the other function does not.
I am trying to solve the time complexity of this function (I'm still new to solving complexity problems) and was wondering what the time complexity of this function would be:
def mystery(lis):
n = len(lis)
for index in range(n):
x = 2*index % n
lis[index],lis[x] = lis[x],lis[index]
print(lis)
I believe the answer is O(n) but I am not 100% sure as the line: x = 2*index % n is making me wonder if it is maybe O(n log n).
The operation to * two operands together is usually consider constant time in time complexity analysis. Same with %.
The fact that you have n as one of the operand doesn't make it O(n) because n is a single number. To make it O(n) you need to perform an operation n times.
def func(n):
if n == 1:
return 1
return func(n-1) + n*(n-1)
print func(5)
Getting confused. Not sure what exactly it is. Is it O(n)?
Calculating the n*(n-1) is a fixed time operation. The interesting part of the function is calling func(n-1) until n is 1. The function will make n such calls, so it's complexity is O(n).
If we assume that arithmetic operations are constant time operations (and they really are when numbers are relatively small) then time complexity is O(n):
T(n) = T(n-1) + C = T(n-2) + C + C = ... = n * C = O(n)
But the multiplication complexity in practice depends on the underlying type (and we are talking about Python where the type depends on the value). It depends on the N as N approaches infinity. Thus, strictly speaking, the complexity is equal to:
T(n) = O(n * multComplexity(n))
And this multComplexity(n) depends on a specific algorithm that is used for multiplication of huge numbers.
As described in other answers, the answer is close to O(n) for practical purposes. For a more precise analysis, if you don't want to make the approximation that multiplication is constant-time:
Calculating n*(n-1) takes O(log n * log n) (or O(log n)^1.58, depending on the algorithm Python uses, which depends on the size of the integer). See here - note that we need to take the log because the complexity is relative to the number of digits.
Adding the two terms takes O(log n), so we can ignore that.
The multiplication gets done O(n) times, so the total is O(n * log n * log n). (It might be possible to get this bound tighter, but it's certainly larger than O(n) - see the WolframAlpha plot).
In practice, the log terms won't really matter unless n gets very large.
The question is available here. My Python code is
def solution(A, B):
if len(A) == 1:
return [1]
ways = [0] * (len(A) + 1)
ways[1], ways[2] = 1, 2
for i in xrange(3, len(ways)):
ways[i] = ways[i-1] + ways[i-2]
result = [1] * len(A)
for i in xrange(len(A)):
result[i] = ways[A[i]] & ((1<<B[i]) - 1)
return result
The detected time complexity by the system is O(L^2) and I can't see why. Thank you in advance.
First, let's show that the runtime genuinely is O(L^2). I copied a section of your code, and ran it with increasing values of L:
import time
import matplotlib.pyplot as plt
def solution(L):
if L == 0:
return
ways = [0] * (L+5)
ways[1], ways[2] = 1, 2
for i in xrange(3, len(ways)):
ways[i] = ways[i-1] + ways[i-2]
points = []
for L in xrange(0, 100001, 10000):
start = time.time()
solution(L)
points.append(time.time() - start)
plt.plot(points)
plt.show()
The result graph is this:
To understand why this O(L^2) when the obvious "time complexity" calculation suggests O(L), note that "time complexity" is not a well-defined concept on its own since it depends on which basic operations you're counting. Normally the basic operations are taken for granted, but in some cases you need to be more careful. Here, if you count additions as a basic operation, then the code is O(N). However, if you count bit (or byte) operations then the code is O(N^2). Here's the reason:
You're building an array of the first L Fibonacci numbers. The length (in digits) of the i'th Fibonacci number is Theta(i). So ways[i] = ways[i-1] + ways[i-2] adds two numbers with approximately i digits, which takes O(i) time if you count bit or byte operations.
This observation gives you an O(L^2) bit operation count for this loop:
for i in xrange(3, len(ways)):
ways[i] = ways[i-1] + ways[i-2]
In the case of this program, it's quite reasonable to count bit operations: your numbers are unboundedly huge as L increases and addition of huge numbers is linear in clock time rather than O(1).
You can fix the complexity of your code by computing the Fibonacci numbers mod 2^32 -- since 2^32 is a multiple of 2^B[i]. That will keep a finite bound on the numbers you're dealing with:
for i in xrange(3, len(ways)):
ways[i] = (ways[i-1] + ways[i-2]) & ((1<<32) - 1)
There are some other issues with the code, but this will fix the slowness.
I've taken the relevant parts of the function:
def solution(A, B):
for i in xrange(3, len(A) + 1): # replaced ways for clarity
# ...
for i in xrange(len(A)):
# ...
return result
Observations:
A is an iterable object (e.g. a list)
You're iterating over the elements of A in sequence
The behavior of your function depends on the number of elements in A, making it O(A)
You're iterating over A twice, meaning 2 O(A) -> O(A)
On point 4, since 2 is a constant factor, 2 O(A) is still in O(A).
I think the page is not correct in its measurement. Had the loops been nested, then it would've been O(A²), but the loops are not nested.
This short sample is O(N²):
def process_list(my_list):
for i in range(0, len(my_list)):
for j in range(0, len(my_list)):
# do something with my_list[i] and my_list[j]
I've not seen the code the page is using to 'detect' the time complexity of the code, but my guess is that the page is counting the number of loops you're using without understanding much of the actual structure of the code.
EDIT1:
Note that, based on this answer, the time complexity of the len function is actually O(1), not O(N), so the page is not incorrectly trying to count its use for the time-complexity. If it were doing that, it would've incorrectly claimed a larger order of growth because it's used 4 separate times.
EDIT2:
As #PaulHankin notes, asymptotic analysis also depends on what's considered a "basic operation". In my analysis, I've counted additions and assignments as "basic operations" by using the uniform cost method, not the logarithmic cost method, which I did not mention at first.
Most of the time simple arithmetic operations are always treated as basic operations. This is what I see most commonly being done, unless the algorithm being analysed is for a basic operation itself (e.g. time complexity of a multiplication function), which is not the case here.
The only reason why we have different results appears to be this distinction. I think we're both correct.
EDIT3:
While an algorithm in O(N) is also in O(N²), I think it's reasonable to state that the code is still in O(N) b/c, at the level of abstraction we're using, the computational steps that seem more relevant (i.e. are more influential) are in the loop as a function of the size of the input iterable A, not the number of bits being used to represent each value.
Consider the following algorithm to compute an:
def function(a, n):
r = 1
for i in range(0, n):
r *= a
return r
Under the uniform cost method, this is in O(N), because the loop is executed n times, but under logarithmic cost method, the algorithm above turns out to be in O(N²) instead due to the time complexity of the multiplication at line r *= a being in O(N), since the number of bits to represent each number is dependent on the size of the number itself.
Codility Ladder competition is best solved in here:
It is super tricky.
We first compute the Fibonacci sequence for the first L+2 numbers. The first two numbers are used only as fillers, so we have to index the sequence as A[idx]+1 instead of A[idx]-1. The second step is to replace the modulo operation by removing all but the n lowest bits
I'm doing a course on algorithms in Python. I'm struggling a bit with Big-O notation. The problem was creating a linear function to find the k-smallest number, and improvind the function to be log linear. I've produced this:
import random
import timeit
def linear_find_smallest(values):
smallest = len(values)
for i in values:
if smallest > i != 0:
smallest = i
return smallest
def loglinear_find_smallest(values):
values.sort()
smallest = len(values)
for i in values:
if smallest > i != 0:
smallest = i
del values[values.index(i):]
return smallest
if __name__ == '__main__':
values = random.sample([i for i in range(1, 10000)], len(range(1, 10000)))
t = timeit.Timer("linear_find_smallest(%s)" % str(values),
setup="from __main__ import linear_find_smallest")
print("Lineair: found in %f seconds" % t.timeit(number=1000))
t = timeit.Timer("loglinear_find_smallest(%s)" % str(values),
setup="from __main__ import loglinear_find_smallest")
print("Log linear: found in %f seconds" % t.timeit(number=1000))
However, the linear function runs in 1.614513 seconds, and the (I supposed) log linear function in 8.463193 seconds. Can someone tell me if I'm doing something wrong here?
First of all, after you sort, you can just return the first value, which is the smallest by definition of sort.
def loglinear_find_smallest(values):
values.sort()
return values[0]
Now for the real question on complexity. If you are given an arbitrary list, then linear search (O(n)) is the best you can do to find the smallest. This is because no matter what your search strategy is, someone can craft an input to your function such that the the first n-1 places you check for the smallest value is not the smallest value.
With that said, nothing can be faster than a linear search. In particular, log linear (O(n log n)) is actually guaranteed to be slower since you are doing more operations. I'm really not sure how/why you got confused on log linear ostensibly being faster than linear. Maybe you confused it with logarthmic (O(log n))?