I coded an algorithm to determine if an input array is a monotonic array (its elements from left to right are entirely increasing or entirely decreasing) or not.
I was wondering what the space complexity of this algorithm is. I am thinking it is O(n) because the count increases as the size of the array increases. Basically I am increasing the count every time the current element is <= or >= to the next element. So the largest value of count and count2 would basically be the size of the input array if the loop goes through every element.
Could someone please explain and correct me if I am wrong?
def monotonic(array):
count = 0
count2 = 0
for i in range(len(array) - 1):
if array[i] <= array[i + 1]:
count += 1
if array[i] >= array[i + 1]:
count2 += 1
if array == []: return True
if count == len(array) - 1 or count2 == len(array) - 1:
return True
else:
return False
The time complexity is O(n).
The space complexity is O(1). The only storage used is for count and count2. There are no additional lists, nor are there recursive calls that would take up a variable amount of stack space.
When counting size complexities, the usual assumption is that if something fits in memory, then its size fits into a constant number of bits, because the size of a computer's machine word is a constant defined by its architecture.
This is a pragmatic choice, and we make different assumptions when it's appropriate. The goal when making a statement of complexity is to say something useful. We all know that asymptotic analysis doesn't technically apply to real, bounded, machines, but it's a useful tool anyway.
Related
I've written a program to benchmark two ways of finding "the longest Collatz chain for integers less than some bound".
The first way is with "backtrack memoization" which keeps track of the current chain from start till hash table collision (in a stack) and then pops all the values into the hash table (with incrementing chain length values).
The second way is with simpler memoization that only memoizes the starting value of the chain.
To my surprise and confusion, the algorithm that memoizes the entirety of the sub-chain up until the first collision is consistently slower than the algorithm which only memoizes the starting value.
I'm wondering if this is due to one of the following factors:
Is Python really slow with stacks? Enough that it offsets performance
gains
Is my code/algorithm bad?
Is it simply the case that, statistically, as integers grow large,
the time spent revisiting the non-memoized elements of previously
calculated Collatz chains/sub-chains is asymptotically minimal, to
the point that any overhead due to popping elements off a stack
simply isn't worth the gains?
In short, I'm wondering if this unexpected result is due to the language, the code, or math (i.e. the statistics of Collatz).
import time
def results(backtrackMemoization, start, maxChainValue, collatzDict):
print()
print(("with " if backtrackMemoization else "without ") + "backtracking memoization")
print("length of " + str(collatzDict[maxChainValue[0]]) + " found for n = " + str(maxChainValue[0]))
print("computed in " + str(round(time.time() - start, 3)) + " seconds")
def collatz(backtrackMemoization, start, maxChainValue, collatzDict):
for target in range(1, maxNum):
n = target
if (backtrackMemoization):
stack = []
else:
length = 0
while (n not in collatzDict):
if (backtrackMemoization):
stack.append(n)
else:
length = length + 1
if (n % 2):
n = 3 * n + 1
else:
n = n // 2
if (backtrackMemoization):
additionalLength = 1
while (len(stack) > 0):
collatzDict[stack.pop()] = collatzDict[n] + additionalLength
additionalLength = additionalLength + 1
else:
collatzDict[target] = collatzDict[n] + length
if (collatzDict[target] > collatzDict[maxChainValue[0]]):
maxChainValue[0] = target
def benchmarkAlgo(maxNum, backtrackMemoization):
start = time.time()
maxChainValue = [1]
collatzDict = {1:0}
collatz(backtrackMemoization, start, maxChainValue, collatzDict)
results(backtrackMemoization, start, maxChainValue, collatzDict)
try:
maxNum = int(input("enter upper bound> "))
print("setting upper bound to " + str(maxNum))
except:
maxNum = 100000
print("defaulting upper bound to " + str(maxNum))
benchmarkAlgo(maxNum, True)
benchmarkAlgo(maxNum, False)
There is a tradeoff in your code. Without the backtrack memoization, dictionary lookups will miss about twice as many times as when you use it. For example, if maxNum = 1,000,000 then the number of missed dictionary lookups is
without backtrack memoization: 5,226,259
with backtrack memoization: 2,168,610
On the other hand, with backtrack memoization, you are constructing a much bigger dictionary since you are collecting lengths of chains not only for the target values, but also for any value that is encountered in the middle of a chain. Here is the final length of collatzDict for maxNum = 1,000,000:
without backtrack memoization: 999,999
with backtrack memoization: 2,168,611
There is a cost of writing to this dictionary that many more times, popping all these additional values from the stack, etc. It seems that in the end, this cost outweighs the benefits of reducing dictionary lookup misses. In my tests, the code with backtrack memoization run about 20% slower.
It is possible to optimize backtrack memoization, to keep the dictionary lookup misses low while reducing the cost of constructing the dictionary:
Let the stack consist of tuples (n, i) where n is as in your code, and i is the length of the chain traversed up to this point (i.e. i is incremented at every iteration of the while loop). Such a tuple is put on the stack only if n < maxNum. In addition, keep track of how long the whole chain gets before you find a value that is already in the dictionary (i.e. of the total number of iterations of the while loop).
The information collected in this way will let you construct new dictionary entries from the tuples that were put on the stack.
The dictionary obtained in this way will be exactly the same as the one constructed without backtrack memoization, but it will be built in a more efficient way, since a key n will be added when it is first encountered. For this reason, dictionary lookup misses will be still much lower than without backtrack memoization. Here are the numbers of misses I obtained for maxNum = 1,000,000:
without backtrack memoization: 5,226,259
with backtrack memoization: 2,168,610
with optimized backtrack memoization: 2,355,035
For larger values of maxNum the optimized code should run faster than without backtrack memoization. In my tests it was about 25% faster for maxNum >= 1,000,000 .
Could anyone explain exactly what's happening under the hood to make the recursive approach in the following problem much faster and efficient in terms of time complexity?
The problem: Write a program that would take an array of integers as input and return the largest three numbers sorted in an array, without sorting the original (input) array.
For example:
Input: [22, 5, 3, 1, 8, 2]
Output: [5, 8, 22]
Even though we can simply sort the original array and return the last three elements, that would take at least O(nlog(n)) time as the fastest sorting algorithm would do just that. So the challenge is to perform better and complete the task in O(n) time.
So I was able to come up with a recursive solution:
def findThreeLargestNumbers(array, largest=[]):
if len(largest) == 3:
return largest
max = array[0]
for i in array:
if i > max:
max = i
array.remove(max)
largest.insert(0, max)
return findThreeLargestNumbers(array, largest)
In which I kept finding the largest number, removing it from the original array, appending it to my empty array, and recursively calling the function again until there are three elements in my array.
However, when I looked at the suggested iterative method, I composed this code:
def findThreeLargestNumbers(array):
sortedLargest = [None, None, None]
for num in array:
check(num, sortedLargest)
return sortedLargest
def check(num, sortedLargest):
for i in reversed(range(len(sortedLargest))):
if sortedLargest[i] is None:
sortedLargest[i] = num
return
if num > sortedLargest[i]:
shift(sortedLargest, i, num)
return
def shift(array, idx, element):
if idx == 0:
array[0] = element
return array
array[0] = array[1]
array[idx-1] = array[idx]
array[idx] = element
return array
Both codes passed successfully all the tests and I was convinced that the iterative approach is faster (even though not as clean..). However, I imported the time module and put the codes to the test by providing an array of one million random integers and calculating how long each solution would take to return back the sorted array of the largest three numbers.
The recursive approach was way much faster (about 9 times faster) than the iterative approach!
Why is that? Even though the recursive approach is traversing the huge array three times and, on top of that, every time it removes an element (which takes O(n) time as all other 999 elements would need to be shifted in the memory), whereas the iterative approach is traversing the input array only once and yes making some operations at every iteration but with a very negligible array of size 3 that wouldn't even take time at all!
I really want to be able to judge and pick the most efficient algorithm for any given problem so any explanation would tremendously help.
Advice for optimization.
Avoid function calls. Avoid creating temporary garbage. Avoid extra comparisons. Have logic that looks at elements as little as possible. Walk through how your code works by hand and look at how many steps it takes.
Your recursive code makes only 3 function calls, and as pointed out elsewhere does an average of 1.5 comparisons per call. (1 while looking for the min, 0.5 while figuring out where to remove the element.)
Your iterative code makes lots of comparisons per element, calls excess functions, and makes calls to things like sorted that create/destroy junk.
Now compare with this iterative solution:
def find_largest(array, limit=3):
if len(array) <= limit:
# Special logic not needed.
return sorted(array)
else:
# Initialize the answer to values that will be replaced.
min_val = min(array[0:limit])
answer = [min_val for _ in range(limit)]
# Now scan for smallest.
for i in array:
if answer[0] < i:
# Sift elements down until we find the right spot.
j = 1
while j < limit and answer[j] < i:
answer[j-1] = answer[j]
j = j+1
# Now insert.
answer[j-1] = i
return answer
There are no function calls. It is possible that you can make up to 6 comparisons per element (verify that answer[0] < i, verify that (j=1) < 3, verify that answer[1] < i, verify that (j=2) < 3, verify that answer[2] < i, then find that (j=3) < 3 is not true). You will hit that worst case if array is sorted. But most of the time you only do the first comparison then move to the next element. No muss, no fuss.
How does it benchmark?
Note that if you wanted the smallest 100 elements, then you'd find it worthwhile to use a smarter data structure such as a heap to avoid the bubble sort.
I am not really confortable with python, but I have a different approach to the problem for what it's worth.
As far as I saw, all solutions posted are O(NM) where N is the length of the array and M the length of the largest elements array.
Because of your specific situation whereN >> M you could say it's O(N), but the longest the inputs the more it will be O(NM)
I agree with #zvone that it seems you have more steps in the iterative solution, which sounds like an valid explanation to your different computing speeds.
Back to my proposal, implements binary search O(N*logM) with recursion:
import math
def binarySearch(arr, target, origin = 0):
"""
Recursive binary search
Args:
arr (list): List of numbers to search in
target (int): Number to search with
Returns:
int: index + 1 from inmmediate lower element to target in arr or -1 if already present or lower than the lowest in arr
"""
half = math.floor((len(arr) - 1) / 2);
if target > arr[-1]:
return origin + len(arr)
if len(arr) == 1 or target < arr[0]:
return -1
if arr[half] < target and arr[half+1] > target:
return origin + half + 1
if arr[half] == target or arr[half+1] == target:
return -1
if arr[half] < target:
return binarySearch(arr[half:], target, origin + half)
if arr[half] > target:
return binarySearch(arr[:half + 1], target, origin)
def findLargestNumbers(array, limit = 3, result = []):
"""
Recursive linear search of the largest values in an array
Args:
array (list): Array of numbers to search in
limit (int): Length of array returned. Default: 3
Returns:
list: Array of max values with length as limit
"""
if len(result) == 0:
result = [float('-inf')] * limit
if len(array) < 1:
return result
val = array[-1]
foundIndex = binarySearch(result, val)
if foundIndex != -1:
result.insert(foundIndex, val)
return findLargestNumbers(array[:-1],limit, result[1:])
return findLargestNumbers(array[:-1], limit,result)
It is quite flexible and might be inspiration for a more elaborated answer.
The recursive solution
The recursive function goes through the list 3 times to fins the largest number and removes the largest number from the list 3 times.
for i in array:
if i > max:
...
and
array.remove(max)
So, you have 3×N comparisons, plus 3x removal. I guess the removal is optimized in C, but there is again about 3×(N/2) comparisons to find the item to be removed.
So, a total of approximately 4.5 × N comparisons.
The other solution
The other solution goes through the list only once, but each time it compares to the three elements in sortedLargest:
for i in reversed(range(len(sortedLargest))):
...
and almost each time it sorts the sortedLargest with these three assignments:
array[0] = array[1]
array[idx-1] = array[idx]
array[idx] = element
So, you are N times:
calling check
creating and reversing a range(3)
accessing sortedLargest[i]
comparing num > sortedLargest[i]
calling shift
comparing idx == 0
and about 2×N/3 times doing:
array[0] = array[1]
array[idx-1] = array[idx]
array[idx] = element
and N/3 times array[0] = element
It is difficult to count, but that is much more than 4.5×N comparisons.
Leetcode - Three sums
https://leetcode.com/problems/3sum/
def threeNumberSum(array, targetSum):
array = sorted(array)
results = []
for idx, elem in enumerate(array):
i = idx + 1
j = len(array) - 1
target = targetSum - elem
while i < j:
currentSum = array[i] + array[j]
if currentSum == target:
result = [array[i], array[j], array[idx]]
results.append(sorted(result))
i += 1
j -= 1
elif currentSum < target:
i += 1
else:
j -= 1
return results
So time is O(n^2), I am fine with that, but space is O(n), according to Algoexpert.io, and I am not sure of why. His explanation was:
"We might end up storing every single number in our array, if every single number is used in some triplet, we will store a lot of numbers and it is going to be bound by O(n) space. Even if some numbers are used multiple times, it will be bounded by O(n)"
But I can't make sense of his explanation yet. If the provided array has (nearly) all unique triplet permutations summing to that target number, isn't space complexity going to be n choose 3 instead? If its n choose k=3 simplifying it would yield O(n^3).
Note, however, that the Algoexpert problem has one additional assumption with the input array that every element will be distinct, whereas the Leetcode version doesn't have that assumption. How would I formally address that information in space complexity analysis?
If your code is correct (and I have no reason to assume it isn't), then the space complexity for the list of matching triplets is in fact O(n2).
It's not O(n3) because the third member of any triplet is uniquely determined by the first two, so there is no freedom of choice for this value.
If all the numbers are unique, then the space requirement is definitely O(n2):
>>> [len(threeNumberSum(range(-i,i+1),0)) for i in range(1,10)]
[1, 2, 5, 8, 13, 18, 25, 32, 41]
You should be able to satisfy yourself that the terms in this series correspond to ceil(n2/2). (See https://oeis.org/A000982).
If there are repeated numbers in the list, then the overall space requirement should decrease (relative to n) due to the requirement for unique triplets in the returned array.
I have this script below that finds the smallest and largest values in an integer array. The goal is to complete this task in less than 1.5 x N comparisons, where N is the length of the input array, n_list. I want to ask a couple of questions.
1: (Inside the for loop) Is comparing the variables smallest or largest to n considered an array comparison? In the script below, I am counting it as one. If it is, why is this the case? The definition I was able to find said that an array comparison is between two arrays, and that's not really what I'm doing, IMO.
2: If I am not counting correctly, what am I doing wrong?
3: What would be a better approach to this problem?
Thanks so much, hope you're having a good day/night :)
def findExtremes(n_list):
smallest = n_list[0]
largest = n_list[0]
counter = 2 # See above
for n in n_list:
if n > largest:
largest = n
counter += 1
continue
elif n < smallest:
smallest = n
counter += 1
continue
else:
counter += 1
continue
return(counter)
You're only counting comparisons that succeed.
When the first if succeeds, you've done one comparison, and you correctly do counter += 1.
But if you get into the elif, you've done two comparisons: n > largest and n < largest, so you need to do counter += 2.
And if that comparison fails, you've still already done two comparisons, so you need to do counter += 2 in the else block as well.
You don't need to initialize counter = 2 at the beginning, you should set it to 0. You'll count the two comparisons with the first element of the list in the loop.
Actually, you might want to just skip those elements, since the result is known. You can do:
for n in n_list[1:]:
to skip over them. If you're supposed to count these unnecessary comparisons, then it makes sense to initialize counter = 2.
Your question about "array comparisons" doesn't seem to be relevant at all. There's nothing about comparing arrays in this problem, you're just comparing array elements to other elements of the same array.
Your algorithm performs anywhere from N+1 to 2*N comparisons. The best case is when the array is sorted from smallest to largest -- the test that updates largest succeeds for each element, so it never has to update smallest. The worst case is when it's sorted in the reverse order or all the numbers are the same: all the largest tests fail, so it has to test each element to see if it's the new smallest. On average with random data it tends to be close to the worst case, about 1.95*N.
I'm reordering the entries of an array so that the even ones (divisible by 2) appear first. The code snippet is as follows:
def even_odd(A):
next_even , next_odd = 0, len(A) - 1
while next_even < next_odd:
if A[next_even] % 2 == 0:
next_even += 1
else:
A[next_even], A[next_odd] = A[next_odd], A[next_even]
next_odd -= 1
The time complexity is given as O(N) which I'm guessing is because I'm going through the whole array? But how's the space complexity O(1)?
You use a fixed amount of space to reorder the list. That, by definition, is O(1). The fact that you're dealing with the length of A does not count against your function's space usage: A was already allocated by the problem definition. All you have added to that is two integers, next_even and next_odd: they are your O(1).
UPDATE per OP comment
The size of A does not "count against" your space complexity, as your algorithm uses the space already provided by the calling program. You haven't added any.
Sorry; I didn't realize you had an open question about the time complexity. Yes, your guess is correct: you go through the while loop N-1 times (N = len(A) ); each iteration takes no more than a constant amount of time. Therefore, the time complexity is bounded by N.
I guess the meaning here that the "additional" memory required for the reordering is o(1), excluded the original array.