Could anyone explain exactly what's happening under the hood to make the recursive approach in the following problem much faster and efficient in terms of time complexity?
The problem: Write a program that would take an array of integers as input and return the largest three numbers sorted in an array, without sorting the original (input) array.
For example:
Input: [22, 5, 3, 1, 8, 2]
Output: [5, 8, 22]
Even though we can simply sort the original array and return the last three elements, that would take at least O(nlog(n)) time as the fastest sorting algorithm would do just that. So the challenge is to perform better and complete the task in O(n) time.
So I was able to come up with a recursive solution:
def findThreeLargestNumbers(array, largest=[]):
if len(largest) == 3:
return largest
max = array[0]
for i in array:
if i > max:
max = i
array.remove(max)
largest.insert(0, max)
return findThreeLargestNumbers(array, largest)
In which I kept finding the largest number, removing it from the original array, appending it to my empty array, and recursively calling the function again until there are three elements in my array.
However, when I looked at the suggested iterative method, I composed this code:
def findThreeLargestNumbers(array):
sortedLargest = [None, None, None]
for num in array:
check(num, sortedLargest)
return sortedLargest
def check(num, sortedLargest):
for i in reversed(range(len(sortedLargest))):
if sortedLargest[i] is None:
sortedLargest[i] = num
return
if num > sortedLargest[i]:
shift(sortedLargest, i, num)
return
def shift(array, idx, element):
if idx == 0:
array[0] = element
return array
array[0] = array[1]
array[idx-1] = array[idx]
array[idx] = element
return array
Both codes passed successfully all the tests and I was convinced that the iterative approach is faster (even though not as clean..). However, I imported the time module and put the codes to the test by providing an array of one million random integers and calculating how long each solution would take to return back the sorted array of the largest three numbers.
The recursive approach was way much faster (about 9 times faster) than the iterative approach!
Why is that? Even though the recursive approach is traversing the huge array three times and, on top of that, every time it removes an element (which takes O(n) time as all other 999 elements would need to be shifted in the memory), whereas the iterative approach is traversing the input array only once and yes making some operations at every iteration but with a very negligible array of size 3 that wouldn't even take time at all!
I really want to be able to judge and pick the most efficient algorithm for any given problem so any explanation would tremendously help.
Advice for optimization.
Avoid function calls. Avoid creating temporary garbage. Avoid extra comparisons. Have logic that looks at elements as little as possible. Walk through how your code works by hand and look at how many steps it takes.
Your recursive code makes only 3 function calls, and as pointed out elsewhere does an average of 1.5 comparisons per call. (1 while looking for the min, 0.5 while figuring out where to remove the element.)
Your iterative code makes lots of comparisons per element, calls excess functions, and makes calls to things like sorted that create/destroy junk.
Now compare with this iterative solution:
def find_largest(array, limit=3):
if len(array) <= limit:
# Special logic not needed.
return sorted(array)
else:
# Initialize the answer to values that will be replaced.
min_val = min(array[0:limit])
answer = [min_val for _ in range(limit)]
# Now scan for smallest.
for i in array:
if answer[0] < i:
# Sift elements down until we find the right spot.
j = 1
while j < limit and answer[j] < i:
answer[j-1] = answer[j]
j = j+1
# Now insert.
answer[j-1] = i
return answer
There are no function calls. It is possible that you can make up to 6 comparisons per element (verify that answer[0] < i, verify that (j=1) < 3, verify that answer[1] < i, verify that (j=2) < 3, verify that answer[2] < i, then find that (j=3) < 3 is not true). You will hit that worst case if array is sorted. But most of the time you only do the first comparison then move to the next element. No muss, no fuss.
How does it benchmark?
Note that if you wanted the smallest 100 elements, then you'd find it worthwhile to use a smarter data structure such as a heap to avoid the bubble sort.
I am not really confortable with python, but I have a different approach to the problem for what it's worth.
As far as I saw, all solutions posted are O(NM) where N is the length of the array and M the length of the largest elements array.
Because of your specific situation whereN >> M you could say it's O(N), but the longest the inputs the more it will be O(NM)
I agree with #zvone that it seems you have more steps in the iterative solution, which sounds like an valid explanation to your different computing speeds.
Back to my proposal, implements binary search O(N*logM) with recursion:
import math
def binarySearch(arr, target, origin = 0):
"""
Recursive binary search
Args:
arr (list): List of numbers to search in
target (int): Number to search with
Returns:
int: index + 1 from inmmediate lower element to target in arr or -1 if already present or lower than the lowest in arr
"""
half = math.floor((len(arr) - 1) / 2);
if target > arr[-1]:
return origin + len(arr)
if len(arr) == 1 or target < arr[0]:
return -1
if arr[half] < target and arr[half+1] > target:
return origin + half + 1
if arr[half] == target or arr[half+1] == target:
return -1
if arr[half] < target:
return binarySearch(arr[half:], target, origin + half)
if arr[half] > target:
return binarySearch(arr[:half + 1], target, origin)
def findLargestNumbers(array, limit = 3, result = []):
"""
Recursive linear search of the largest values in an array
Args:
array (list): Array of numbers to search in
limit (int): Length of array returned. Default: 3
Returns:
list: Array of max values with length as limit
"""
if len(result) == 0:
result = [float('-inf')] * limit
if len(array) < 1:
return result
val = array[-1]
foundIndex = binarySearch(result, val)
if foundIndex != -1:
result.insert(foundIndex, val)
return findLargestNumbers(array[:-1],limit, result[1:])
return findLargestNumbers(array[:-1], limit,result)
It is quite flexible and might be inspiration for a more elaborated answer.
The recursive solution
The recursive function goes through the list 3 times to fins the largest number and removes the largest number from the list 3 times.
for i in array:
if i > max:
...
and
array.remove(max)
So, you have 3×N comparisons, plus 3x removal. I guess the removal is optimized in C, but there is again about 3×(N/2) comparisons to find the item to be removed.
So, a total of approximately 4.5 × N comparisons.
The other solution
The other solution goes through the list only once, but each time it compares to the three elements in sortedLargest:
for i in reversed(range(len(sortedLargest))):
...
and almost each time it sorts the sortedLargest with these three assignments:
array[0] = array[1]
array[idx-1] = array[idx]
array[idx] = element
So, you are N times:
calling check
creating and reversing a range(3)
accessing sortedLargest[i]
comparing num > sortedLargest[i]
calling shift
comparing idx == 0
and about 2×N/3 times doing:
array[0] = array[1]
array[idx-1] = array[idx]
array[idx] = element
and N/3 times array[0] = element
It is difficult to count, but that is much more than 4.5×N comparisons.
Related
I am working on my python, doing codewars. The description is as follows:
The maximum sum subarray problem consists in finding the maximum sum of a contiguous subsequence in an array or list of integers:
max_sequence([-2, 1, -3, 4, -1, 2, 1, -5, 4])
should be 6: [4, -1, 2, 1]
Easy case is when the list is made up of only positive numbers and the maximum sum is the sum of the whole array. If the list is made up of only negative numbers, return 0 instead.
Empty list is considered to have zero greatest sum. Note that the empty list or array is also a valid sublist/subarray.
Great! Done! here's my code, which passes the tests:
def max_sequence(arr):
sums = []
lists = [[]]
for i in range(len(arr) + 1):
for j in range(i):
lists.append(arr[j: i])
for i in lists:
sums.append(sum(i))
return max(sums)
However, for submission, codewars requires you to pass a larger battery of tests, and the tests timeout for larger sets.
In the discussion, many people have the same problem as me. One answer in particular gets to the root of the question, which is what i'm asking here (see the comment below):
Your code is not optmised to work with longer arrays, whilst your code likely works, it takes too long to solve the harder problems so times out. This questions is as much an optimisation problem as any. So you need to find a way to optimise your solution
That is very true for me! What am i doing "wrong" in this data structure, and how can i improve it? My current guesses for the most expensive computations are:
loop within loop (for i in range.... for j in range i)
lists.append(arr[j:i])
Any advice? How to improve performance here? I am thinking as much about general data structures and learning as i am about solving this specific question. Thank you!
Similar idea with earlier post, but it tries to bail out earlier when hitting edge cases: (it's still achieved O(n) )
def maxSequence(arr):
if not arr: return 0 # check if it's empty list
if max(arr) < 0: return 0 # check if all negatives
maxx,curr= 0, 0
for x in arr:
curr += x
if curr < 0:
curr = 0
if curr> maxx:
maxx = curr
return maxx
Reference: https://en.wikipedia.org/wiki/Maximum_subarray_problem#Kadane's_algorithm
You can use Kadane's Algorithm. The idea is that keep adding elements to curr and get the maximum of curr and num. When the sum of the subarray is positive, it keeps going. When the sum of the subarray is negative, it gives up the negative subarray.
You can consider this example with the following code: [-1,1000,-2]. Initially, curr = -1. Since it is negative, curr gives up -1 and gets the value of 1000. Finally, since 1000 is greater than 998, it returns 1000 as the answer.
This only has a time complexity of O(n) instead of the brute force approach that has an O(n^3).
def max_sequence(arr):
if not arr or max(arr) < 0:
return 0
curr = max_sub = arr[0]
for num in arr[1:]:
curr = max(num, curr + num)
max_sub = max(max_sub, curr)
return max_sub
Input is an array that has at most one element that appears at least 60% a time. The goal is to find if this array has such an element and if yes, find that element. I came up with a divide and conquer function that finds such an element.
from collections import Counter
def CommonElement(a):
c = Counter(a)
return c.most_common(1) #Returns the element and it's frequency
def func(array):
if len(array) == 1:
return array[0]
mid = len(array)//2
left_element = func(array[:mid])
right_element = func(array[mid:])
if left_element == right_element:
return right_element
most_common_element = CommonElement(array)
element_count = most_common_element[0][1] #Getting the frequency of the element
percent = element_count/len(array)
if percent >= .6:
return most_common_element[0][0] #Returning the value of the element
else:
return None
array = [10,9,10,10,5,10,10,10,12,42,10,10,44,10,23,10] #Correctly Returns 10
array = [10,9,10,8,5,10,10,10,12,42,10,12,44,10,23,5] #Correctly Returns None
result = func(array)
print(result)
This function works but it's in O(n log(n)). I want to implement an algorithm that's O(n)
The recursion function for my algorithm is T(n) = 2T(n/2) + O(n). I think the goal is to eliminate the need to find frequency, which takes O(n). Any thoughts?
If you are guaranteed to have a list 60% of which is a given number, that number is guaranteed to be the median. To see this intuitively, sort the list. The number in question represents a contiguous window that is 60% of the length of the list. There is no way to place that window so that it doesn't cover the middle.
There are plenty of divide-and-conquer algorithms for finding the median. A common one is called introselect. You can find an implementation in numpy's partition and argpartition functions (it's written in C). The basic idea is to do quicksort, but only recurse into the portion that contains the index you care about. This reduces the algorithm to O(n).
By the way, you could search for any index between 40% and 60% of the length of the list. 50% seems like a reasonable middle ground.
To verify that the median appears > 60% of the time, run a single loop over the array, counting the number of times the median appears.
You can create a frequency counter for all elements in the list one time in O(n). Then, iterate the lookup table and see if any are at least 60% of the elements (in other words, count / len(lst) >= 0.6).
>>> from collections import Counter
>>> L = [4, 2, 3, 2, 4, 4, 4]
>>> Counter(L)
Counter({4: 4, 2: 1, 3: 1})
>>> Counter(L).most_common(1)
[(4, 4)]
>>> item, count = Counter(L).most_common(1)[0]
>>> count / len(L)
0.6666666666666666
>>> count / len(L) >= 0.6
True
Divide & conquer is a creative, but inappropriate, approach for this problem.
...or so I thought, but see this answer.
There's a pretty simple algorithm for finding the majority element of a collection, if the collection has one:
def majority(l):
count, candidate = 0, None
for element in l:
if count == 0:
count, candidate = 1, element
elif element == candidate:
count += 1
else:
count -= 1
return candidate
This algorithm essentially pairs each element of the input against another element with a different value until all unpaired elements have the same value, then returns that value. If the input has a majority element, the algorithm must return that.
You can compute a candidate with this algorithm, then make another pass through the input and see if that candidate is a 60% supermajority. This works in O(1) space and O(n) time without mutating the input, while hash-based or introselect-based algorithms would need more space or mutate the input. It's also immune to hash collision attacks (unlike Counter and other hash-based approaches) and doesn't require elements to have an order relation (unlike introselect).
I've been studying python algorithm and would like to solve a problem that is:
A positive integer array A and an integer K are given.
Find the largest even sum of the array A with K elements.
If not possible, return -1.
For example, if there is an array A= [1,2,3,4,4,5] and K= 3,
the answer is 12 (5+4+3),
which is the largest even sum with K (3) elements.
However, if A= [3, 3, 3] and K= 1,
the answer is -1 because it cannot make an even sum with one element.
I tried to exclude every minimum odd from the array, but it failed when K=n in the while loop.
Is there any simple way to solve this problem? I would sincerely appreciate if you could give some advice.
Sort the array and "take" the biggest K elements.
If it's already even sum - you are done.
Otherwise, you need to replace exactly one element, replace an even element you have chosen with an odd one you have not, or the other way around. You need the difference between the two elements to be minimized.
A naive solution will check all possible ways to do that, but that's O(n^2). You can do better, by checking the actual two viable candidates:
The maximal odd element you did not choose, and the minimal
even element you have chosen
The maximal even element you did not choose and the minimal odd element you have chosen.
Choose the one that the difference between the two elements is minimal. If no such two elements exist (i.e. your k=3, [3,3,3] example) - there is no viable solution.
Time complexity is O(nlogn) for sorting.
In my (very rusty) python, it should be something like:
def FindMaximalEvenArray(a, k):
a = sorted(a)
chosen = a[len(a)-k:]
not_chosen = a[0:len(a)-k]
if sum(chosen) % 2 == 0:
return sum(chosen)
smallest_chosen_even = next((x for x in chosen if x % 2 == 0), None)
biggest_not_chosen_odd = next((x for x in not_chosen[::-1] if x % 2 != 0), None)
candidiate1 = smallest_chosen_even - biggest_not_chosen_odd if smallest_chosen_even and biggest_not_chosen_odd else float("inf")
smallest_chosen_odd = next((x for x in chosen if x % 2 != 0), None)
biggest_not_chosen_even = next((x for x in not_chosen[::-1] if x % 2 == 0), None)
candidiate2 = smallest_chosen_odd - biggest_not_chosen_even if smallest_chosen_odd and biggest_not_chosen_even else float("inf")
if candidiate1 == float("inf") and candidiate2 == float("inf"):
return -1
return sum(chosen) - min(candidiate1, candidiate2)
Note: This can be done even better (in terms of time complexity), because you don't actually care for the order of all elements, only finding the "candidates" and the top K elements. So you could use Selection Algorithm instead of sorting, which will make this run in O(n) time.
I know merge sort is the best way to sort a list of arbitrary length, but I am wondering how to optimize my current method.
def sortList(l):
'''
Recursively sorts an arbitrary list, l, to increasing order.
'''
#base case.
if len(l) == 0 or len(l) == 1:
return l
oldNum = l[0]
newL = sortList(l[1:]) #recursive call.
#if oldNum is the smallest number, add it to the beginning.
if oldNum <= newL[0]:
return [oldNum] + newL
#find where oldNum goes.
for n in xrange(len(newL)):
if oldNum >= newL[n]:
try:
if oldNum <= newL[n+1]:
return newL[:n+1] + [oldNum] + newL[n+1:]
#if index n+1 is non-existant, oldNum must be the largest number.
except IndexError:
return newL + [oldNum]
What is the complexity of this function? I was thinking O(n^2) but I wasn't sure. Also, is there anyway to further optimize this procedure? (besides ditching it and going for merge sort!).
There's a few places I'd optimize your code.
You do a lot of list copies: each time you slice, you create a new copy of the list. That can be avoided by adding an index to the function declaration that indicates where in the array to start sorting from.
You should follow PEP 8 for naming: sort_list rather than sortList.
The code that does the insertion is a bit weird; intentionally raising an out-of-bounds index exception isn't normal programming practice. Instead, just percolate the value up the array until it's in the right place.
Applying these changes gives this code:
def sort_list(l, i=0):
if i == len(l): return
sort_list(l, i+1)
for j in xrange(i+1, len(l)):
if l[j-1] <= l[j]: return
l[j-1], l[j] = l[j], l[j-1]
This now sorts the array in-place, so there's no return value.
Here's some simple tests:
cases = [
[1, 2, 0, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[5, 4, 3, 2, 1, 1]
]
for c in cases:
got = c[:]
sort_list(got)
if sorted(c) != got:
print "sort_list(%s) = %s, want %s" % (c, got, sorted(c))
The time complexity is, as you suggest, O(n^2) where n is the length of the list. My version uses O(n) additional memory, whereas yours, because of the way the list gets copied at each stage, uses O(n^2).
One more step, which further improves the memory usage is to eliminate the recursion. Here's a version that does that:
def sort_list(l):
for i in xrange(len(l)-2, -1, -1):
for j in xrange(i+1, len(l)):
if l[j-1] <= l[j]: break
l[j-1], l[j] = l[j], l[j-1]
This works just the same as the recursive version, but does it iteratively; first sorting the last two elements in the array, then the last three, then the last four, and so on until the whole array is sorted.
This still has runtime complexity O(n^2), but now uses O(1) additional memory. Also, avoiding recursion means you can sort longer lists without hitting the notoriously low recursion limit in Python. And another benefit is that this code is now O(n) in the best case (when the array is already sorted).
A young Euler came up with a formula that seems appropriate here. The story goes that in grade school his teacher was very tired and to keep the class busy for a while they were told to add up all the numbers zero to one hundred. Young Euler came back with this:
This is applicable here because your run-time is going to be proportional to the sum of all the numbers up to the length of your list because in the worst case your function will be sorting an already sorted list and will go through the entire length newL each time to find the position of the next element at the end of the list.
I was attempting to solve a programing challenge and the program i wrote solved the small test data correctly for this question. But When they run it against the larger datasets, my program timed out on some of the occasions . I am mostly a self taught programmer, if there is a better algorithm/implementation than my logic can you guys tell me.thanks.
Question
Given an array of integers, a, return the maximum difference of any
pair of numbers such that the larger integer in the pair occurs at a
higher index (in the array) than the smaller integer. Return -1 if you
cannot find a pair that satisfies this condition.
My Python Function
def maxDifference( a):
diff=0
find=0
leng = len(a)
for x in range(0,leng-1):
for y in range(x+1,leng):
if(a[y]-a[x]>=diff):
diff=a[y]-a[x]
find=1
if find==1:
return diff
else:
return -1
Constraints:
1 <= N <= 1,000,000
-1,000,000 <= a[i] <= 1,000,000 i belongs to [1,N]
Sample Input:
Array { 2,3,10,2,4,8,1}
Sample Output:
8
Well... since you don't care for anything else than finding the highest number following the lowest number, provided that difference is the highest so far, there's no reason to do several passes or using max() over a slice of the array:
def f1(a):
smallest = a[0]
result = 0
for b in a:
if b < smallest:
smallest = b
if b - smallest > result:
result = b - smallest
return result if result > 0 else -1
Thanks #Matthew for the testing code :)
This is very fast even on large sets:
The maximum difference is 99613 99613 99613
Time taken by Sojan's method: 0.0480000972748
Time taken by #Matthews's method: 0.0130000114441
Time taken by #GCord's method: 0.000999927520752
The reason your program takes too long is that your nested loop inherently means quadratic time.
The outer loop goes through N-1 indices. The inner loop goes through a different number of indices each time, but the average is obviously (N-1)/2 rounded up. So, the total number of times through the inner loop is (N-1) * (N-1)/2, which is O(N^2). For the maximum N=1000000, that means 499999000001 iterations. That's going to take a long time.
The trick is to find a way to do this in linear time.
Here's one solution (as a vague description, rather than actual code, so someone can't just copy and paste it when they face the same test as you):
Make a list of the smallest value before each index. Each one is just min(smallest_values[-1], arr[i]), and obviously you can do this in N steps.
Make a list of the largest value after each index. The simplest way to do this is to reverse the list, do the exact same loop as above (but with max instead of min), then reverse again. (Reversing a list takes N steps, of course.)
Now, for each element in the list, instead of comparing to every other element, you just have to compare to smallest_values[i] and largest_values[i]. Since you're only doing 2 comparisons for each of the N values, this takes 2N time.
So, even being lazy and naive, that's a total of N + 3N + 2N steps, which is O(N). If N=1000000, that means 6000000 steps, which is a whole lot faster than 499999000001.
You can obviously see how to remove the two reverses, and how to skip the first and last comparisons. If you're smart, you can see how to take the whole largest_values out of the equation entirely. Ultimately, I think you can get it down to 2N - 3 steps, or 1999997. But that's all just a small constant improvement; nowhere near as important as fixing the basic algorithmic problem. You'd probably get a bigger improvement than 3x (maybe 20x), for less work, by just running the naive code in PyPy instead of CPython, or by converting to NumPy—but you're not going to get the 83333x improvement in any way other than changing the algorithm.
Here's a linear time solution. It keeps a track of the minimum value before each index of the list. These minimum values are stored in a list min_lst. Finally, the difference between corresponding elements of the original and the min list is calculated into another list of differences by zipping the two. The maximum value in this differences list should be the required answer.
def get_max_diff(lst):
min_lst = []
running_min = lst[0]
for item in lst:
if item < running_min:
running_min = item
min_lst.append(running_min)
val = max(x-y for (x, y) in zip(lst, min_lst))
if not val:
return -1
return val
>>> get_max_diff([5, 6, 2, 12, 8, 15])
13
>>> get_max_diff([2, 3, 10, 2, 4, 8, 1])
8
>>> get_max_diff([5, 4, 3, 2, 1])
-1
Well, I figure since someone in the same problem can copy your code and run with that, I won't lose any sleep over them copying some more optimized code:
import time
import random
def max_difference1(a):
# your function
def max_difference2(a):
diff = 0
for i in range(0, len(a)-1):
curr_diff = max(a[i+1:]) - a[i]
diff = max(curr_diff, diff)
return diff if diff != 0 else -1
my_randoms = random.sample(range(100000), 1000)
t01 = time.time()
max_dif1 = max_difference1(my_randoms)
dt1 = time.time() - t01
t02 = time.time()
max_dif2 = max_difference2(my_randoms)
dt2 = time.time() - t02
print("The maximum difference is", max_dif1)
print("Time taken by your method:", dt1)
print("Time taken by my method:", dt2)
print("My method is", dt1/dt2, "times faster.")
The maximum difference is 99895
Time taken by your method: 0.5533690452575684
Time taken by my method: 0.08005285263061523
My method is 6.912546237558299 times faster.
Similar to what #abarnert said (who always snipes me on these things I swear), you don't want to loop over the list twice. You can exploit the fact that you know that your larger value has to be in front of the smaller one. You also can exploit the fact that you don't care for anything except the largest number, that is, in the list [1,3,8,5,9], the maximum difference is 8 (9-1) and you don't care that 3, 8, and 5 are in there. Thus: max(a[i+1:]) - a[i] is the maximum difference for a given index.
Then you compare it with diff, and take the larger of the 2 with max, as calling default built-in python functions is somewhat faster than if curr_diff > diff: diff = curr_diff (or equivalent).
The return line is simply your (fixed) line in 1 line instead of 4
As you can see, in a sample of 1000, this method is ~6x faster (note: used python 3.4, but nothing here would break on python 2.x)
I think the expected answer for
1, 2, 4, 2, 3, 8, 5, 6, 10
will be 8 - 2 = 6 but instead Saksham Varma code will return 10 - 1 = 9.
Its max(arr) - min(arr).
Don't we have to reset the min value when there is a dip
. ie; 4 -> 2 will reset current_smallest = 2 and continue diff the calculation with value '2'.
def f2(a):
current_smallest = a[0]
large_diff = 0
for i in range(1, len(a)):
# Identify the dip
if a[i] < a[i-1]:
current_smallest = a[i]
if a[i] - current_smallest > large_diff:
large_diff = a[i] - current_smallest