Develop Reference

Codility FibFrog Algorithm - Improving Time Complexity from O(N * log(N) ** N) to O(N*log(N)) [duplicate]

This question already has answers here:
FibFrog Codility Problem - Optimising for Performance
(2 answers)
Closed last year.
I'm trying to solve the Codility FibFrog problem and I came up with the following solution:
def jumps_from(position, fb, A):
paths = set([])
for i in fb:
newPos = position + i
if newPos == len(A):
return set([-1])
elif newPos < len(A):
if A[newPos] == 1:
paths.add(newPos)
else: break
return paths
def solution(A):
if len(A) < 3: return 1
fibonaccis = fibonacci(len(A))
if len(A) + 1 in fibonaccis: return 1
paths = set([-1])
steps = 0
while True:
paths = set([idx for pos in paths for idx in jumps_from(pos, fibonaccis, A)])
if len(paths) == 0: return -1
if -1 in paths:
return steps + 1
steps += 1
return steps
def fibonacci(N):
arr = [0] * (N + 2)
arr[1] = 1
for i in range(2, N + 2):
arr[i] = arr[i-1] + arr[i-2]
return dict.fromkeys(arr[2:], 1)
Codility detects the runtime of this as O(N * log(N) ** N).
Codility Report: https://app.codility.com/demo/results/trainingJV7YAC-G3B/
I'm comparing this with the following solution, which scores 100% on Codility, and has runtime O(N * log(N)):
def gen_fib(n):
fn = [0,1]
i = 2
s = 2
while s < n:
s = fn[i-2] + fn[i-1]
fn.append(s)
i+=1
return fn
def new_paths(A, n, last_pos, fn):
"""
Given an array A of len n.
From index last_pos which numbers in fn jump to a leaf?
returns list: set of indexes with leaves.
"""
paths = []
for f in fn:
new_pos = last_pos + f
if new_pos == n or (new_pos < n and A[new_pos]):
paths.append(new_pos)
return paths
def solution(A):
n = len(A)
if n < 3:
return 1
# A.append(1) # mark final jump
fn = sorted(gen_fib(100000)[2:]) # Fib numbers with 0, 1, 1, 2.. clipped to just 1, 2..
# print(fn)
paths = set([-1]) # locate all the leaves that are one fib jump from the start position.
jump = 1
while True:
# Considering each of the previous jump positions - How many leaves from there are one fib jump away
paths = set([idx for pos in paths for idx in new_paths(A, n, pos, fn)])
# no new jumps means game over!
if not paths:
break
# If there was a result in the new jumps record that
if n in paths:
return jump
jump += 1
return -1
I'm not sure why my solution differs in runtime, since the approach is exactly the same - compute all the indices you can jump to from -1, and then compute all the indices you can jump to from the new positions, until you get to the other side of the river, or no new positions can be found.

Please refer to the first point in my previous answer.
If len(A) = 100000, you are calculating 100003 fibonacci numbers, while we only need fibonacci numbers which are less than 100k, which would be <30 of them.
The current fibonacci function is still returning N fibonacci numbers instead of just returning fibonacci numbers which are less than N. For N=100k, it should be just 25 numbers instead of over 100k.
Please update your fibonacci function to this -
def fibonacci(N):
arr = [1, 1]
while arr[-1] < N:
arr.append(arr[-1] + arr[-2])
return dict.fromkeys(arr[1:], 1)
I just ran a test locally, and looks like your fibonacci function takes ~1 sec to generate the first 100k fibonacci numbers, and that's the reason it might be failing the performance test, even though the rest of your code is optimal. I think you should be able to clear it with the required performance limits after correcting the fibonacci function.

How to tell if number can be writen as sum of n different squares?

I know how to check if the number can be represented as the sum of two squares with a brute-force approach.
def sumSquare( n) :
i = 1
while i * i <= n :
j = 1
while(j * j <= n) :
if (i * i + j * j == n) :
print(i, "^2 + ", j , "^2" )
return True
j = j + 1
i = i + 1
return False
But how to do it for n distinct positive integers. So the question would be:
Function which checks if the number can be written as sum of 'n' different squares
I have some examples.
For e.g.
is_sum_of_squares(18, 2) would be false because 18 can be written as the sum of two squares (3^2 + 3^2) but they are not distinct.
(38,3) would be true because 5^2+3^2+2^2 = 38 and 5!=3!=2.
I can't extend the if condition for more values. I think it could be done with recursion, but I have problems with it.
I found this function very useful since it finds the number of squares the number can be split into.
def findMinSquares(n):
T = [0] * (n + 1)
for i in range(n + 1):
T[i] = i
j = 1
while j * j <= i:
T[i] = min(T[i], 1 + T[i - j * j])
j += 1
return T[n]
But again I can't do it with recursion. Sadly I can't wrap my head around it. We started learning it a few weeks ago (I am in high school) and it is so different from the iterative approach.

Recursive approach:
def is_sum_of_squares(x, n, used=None):
x_sqrt = int(x**0.5)
if n == 1:
if x_sqrt**2 == x:
return used.union([x_sqrt])
return None
used = used or set()
for i in set(range(max(used, default=0)+1, int((x/n)**0.5))):
squares = is_sum_of_squares(x-i**2, n-1, used.union([i]))
if squares:
return squares
return None

Quite a compelling exercise. I have attempted solving it using recursion in a form of backtracking. Start with an empty list, run a for loop to add numbers to it from 1 to max feasible (square root of target number) and for each added number continue with recursion. Once the list reaches the required size n, validate the result. If the result is incorrect, backtrack by removing the last number.
Not sure if it is 100% correct though. In terms of speed, I tried it on the (1000,13) input and the process finished reasonably fast (3-4s).
def is_sum_of_squares(num, count):
max_num = int(num ** 0.5)
return backtrack([], num, max_num, count)
def backtrack(candidates, target, max_num, count):
"""
candidates = list of ints of max length <count>
target = sum of squares of <count> nonidentical numbers
max_num = square root of target, rounded
count = desired size of candidates list
"""
result_num = sum([x * x for x in candidates]) # calculate sum of squares
if result_num > target: # if sum exceeded target number stop recursion
return False
if len(candidates) == count: # if candidates reach desired length, check if result is valid and return result
result = result_num == target
if result: # print for result sense check, can be removed
print("Found: ", candidates)
return result
for i in range(1, max_num + 1): # cycle from 1 to max feasible number
if candidates and i <= candidates[-1]:
# for non empty list, skip numbers smaller than the last number.
# allow only ascending order to eliminate duplicates
continue
candidates.append(i) # add number to list
if backtrack(candidates, target, max_num, count): # next recursion
return True
candidates.pop() # if combination was not valid then backtrack and remove the last number
return False
assert(is_sum_of_squares(38, 3))
assert(is_sum_of_squares(30, 3))
assert(is_sum_of_squares(30, 4))
assert(is_sum_of_squares(36, 1))
assert not(is_sum_of_squares(35, 1))
assert not(is_sum_of_squares(18, 2))
assert not(is_sum_of_squares(1000, 13))

how to use for-loop to sum first n integer fractions

How to create a program that uses a for-loop to sum the first n
integer fractions 1/2 + 2/3 + 3/4 + .... (n-1) / n + n / (n + 1). (n> 0 is a positive
integer and the last number in the sum is n / (n + 1)
I have tried many times but nothing seems to work.

You can try it.
def sum_fraction(n):
# Initialize the sum variable
sum = 0
# For loop : start from 1 ( n greater than zero ) : end with n
for i in range(1, n):
# Sum current term with all previous term (term by term)
sum += i / (i + 1)
return sum
n = 5
res = sum_fraction(n)

n = int(input("Enter the nth term"))
fsum = 0
#Recommended to not use 'sum' itself
#as it can cause problems with python's built in function which also has the same name
for i in range(1, n+1):
fsum += i/(i+1)
print(fsum)
I guess this should solve your problem
If u want to know more about range function,see this
https://www.w3schools.com/python/ref_func_range.asp
Note that the range function only returns till the stop value and not including the stop value.The stop value in your case is the variable n.That is why we have to use n+1 in the range function

How to write 2**n - 1 as a recursive function?

I need a function that takes n and returns 2n - 1 . It sounds simple enough, but the function has to be recursive. So far I have just 2n:
def required_steps(n):
if n == 0:
return 1
return 2 * req_steps(n-1)
The exercise states: "You can assume that the parameter n is always a positive integer and greater than 0"

2**n -1 is also 1+2+4+...+2n-1 which can made into a single recursive function (without the second one to subtract 1 from the power of 2).
Hint: 1+2*(1+2*(...))
Solution below, don't look if you want to try the hint first.
This works if n is guaranteed to be greater than zero (as was actually promised in the problem statement):
def required_steps(n):
if n == 1: # changed because we need one less going down
return 1
return 1 + 2 * required_steps(n-1)
A more robust version would handle zero and negative values too:
def required_steps(n):
if n < 0:
raise ValueError("n must be non-negative")
if n == 0:
return 0
return 1 + 2 * required_steps(n-1)
(Adding a check for non-integers is left as an exercise.)

To solve a problem with a recursive approach you would have to find out how you can define the function with a given input in terms of the same function with a different input. In this case, since f(n) = 2 * f(n - 1) + 1, you can do:
def required_steps(n):
return n and 2 * required_steps(n - 1) + 1
so that:
for i in range(5):
print(required_steps(i))
outputs:
0
1
3
7
15

You can extract the really recursive part to another function
def f(n):
return required_steps(n) - 1
Or you can set a flag and define just when to subtract
def required_steps(n, sub=True):
if n == 0: return 1
return 2 * required_steps(n-1, False) - sub
>>> print(required_steps(10))
1023

Using an additional parameter for the result, r -
def required_steps (n = 0, r = 1):
if n == 0:
return r - 1
else:
return required_steps(n - 1, r * 2)
for x in range(6):
print(f"f({x}) = {required_steps(x)}")
# f(0) = 0
# f(1) = 1
# f(2) = 3
# f(3) = 7
# f(4) = 15
# f(5) = 31
You can also write it using bitwise left shift, << -
def required_steps (n = 0, r = 1):
if n == 0:
return r - 1
else:
return required_steps(n - 1, r << 1)
The output is the same

Have a placeholder to remember original value of n and then for the very first step i.e. n == N, return 2^n-1
n = 10
# constant to hold initial value of n
N = n
def required_steps(n, N):
if n == 0:
return 1
elif n == N:
return 2 * required_steps(n-1, N) - 1
return 2 * required_steps(n-1, N)
required_steps(n, N)

One way to get the offset of "-1" is to apply it in the return from the first function call using an argument with a default value, then explicitly set the offset argument to zero during the recursive calls.
def required_steps(n, offset = -1):
if n == 0:
return 1
return offset + 2 * required_steps(n-1,0)

On top of all the awesome answers given earlier, below will show its implementation with inner functions.
def outer(n):
k=n
def p(n):
if n==1:
return 2
if n==k:
return 2*p(n-1)-1
return 2*p(n-1)
return p(n)
n=5
print(outer(n))
Basically, it is assigning a global value of n to k and recursing through it with appropriate comparisons.

Find subset with K elements that are closest to eachother

Given an array of integers size N, how can you efficiently find a subset of size K with elements that are closest to each other?
Let the closeness for a subset (x1,x2,x3,..xk) be defined as:
2 <= N <= 10^5
2 <= K <= N
constraints: Array may contain duplicates and is not guaranteed to be sorted.
My brute force solution is very slow for large N, and it doesn't check if there's more than 1 solution:
N = input()
K = input()
assert 2 <= N <= 10**5
assert 2 <= K <= N
a = []
for i in xrange(0, N):
a.append(input())
a.sort()
minimum = sys.maxint
startindex = 0
for i in xrange(0,N-K+1):
last = i + K
tmp = 0
for j in xrange(i, last):
for l in xrange(j+1, last):
tmp += abs(a[j]-a[l])
if(tmp > minimum):
break
if(tmp < minimum):
minimum = tmp
startindex = i #end index = startindex + K?
Examples:
N = 7
K = 3
array = [10,100,300,200,1000,20,30]
result = [10,20,30]
N = 10
K = 4
array = [1,2,3,4,10,20,30,40,100,200]
result = [1,2,3,4]

Your current solution is O(NK^2) (assuming K > log N). With some analysis, I believe you can reduce this to O(NK).
The closest set of size K will consist of elements that are adjacent in the sorted list. You essentially have to first sort the array, so the subsequent analysis will assume that each sequence of K numbers is sorted, which allows the double sum to be simplified.
Assuming that the array is sorted such that x[j] >= x[i] when j > i, we can rewrite your closeness metric to eliminate the absolute value:
Next we rewrite your notation into a double summation with simple bounds:
Notice that we can rewrite the inner distance between x[i] and x[j] as a third summation:
where I've used d[l] to simplify the notation going forward:
Notice that d[l] is the distance between each adjacent element in the list. Look at the structure of the inner two summations for a fixed i:
j=i+1 d[i]
j=i+2 d[i] + d[i+1]
j=i+3 d[i] + d[i+1] + d[i+2]
...
j=K=i+(K-i) d[i] + d[i+1] + d[i+2] + ... + d[K-1]
Notice the triangular structure of the inner two summations. This allows us to rewrite the inner two summations as a single summation in terms of the distances of adjacent terms:
total: (K-i)*d[i] + (K-i-1)*d[i+1] + ... + 2*d[K-2] + 1*d[K-1]
which reduces the total sum to:
Now we can look at the structure of this double summation:
i=1 (K-1)*d[1] + (K-2)*d[2] + (K-3)*d[3] + ... + 2*d[K-2] + d[K-1]
i=2 (K-2)*d[2] + (K-3)*d[3] + ... + 2*d[K-2] + d[K-1]
i=3 (K-3)*d[3] + ... + 2*d[K-2] + d[K-1]
...
i=K-2 2*d[K-2] + d[K-1]
i=K-1 d[K-1]
Again, notice the triangular pattern. The total sum then becomes:
1*(K-1)*d[1] + 2*(K-2)*d[2] + 3*(K-3)*d[3] + ... + (K-2)*2*d[K-2]
+ (K-1)*1*d[K-1]
Or, written as a single summation:
This compact single summation of adjacent differences is the basis for a more efficient algorithm:
Sort the array, order O(N log N)
Compute the differences of each adjacent element, order O(N)
Iterate over each N-K sequence of differences and calculate the above sum, order O(NK)
Note that the second and third step could be combined, although with Python your mileage may vary.
The code:
def closeness(diff,K):
acc = 0.0
for (i,v) in enumerate(diff):
acc += (i+1)*(K-(i+1))*v
return acc
def closest(a,K):
a.sort()
N = len(a)
diff = [ a[i+1] - a[i] for i in xrange(N-1) ]
min_ind = 0
min_val = closeness(diff[0:K-1],K)
for ind in xrange(1,N-K+1):
cl = closeness(diff[ind:ind+K-1],K)
if cl < min_val:
min_ind = ind
min_val = cl
return a[min_ind:min_ind+K]

itertools to the rescue?
from itertools import combinations
def closest_elements(iterable, K):
N = set(iterable)
assert(2 <= K <= len(N) <= 10**5)
combs = lambda it, k: combinations(it, k)
_abs = lambda it: abs(it[0] - it[1])
d = {}
v = 0
for x in combs(N, K):
for y in combs(x, 2):
v += _abs(y)
d[x] = v
v = 0
return min(d, key=d.get)
>>> a = [10,100,300,200,1000,20,30]
>>> b = [1,2,3,4,10,20,30,40,100,200]
>>> print closest_elements(a, 3); closest_elements(b, 4)
(10, 20, 30) (1, 2, 3, 4)

This procedure can be done with O(N*K) if A is sorted. If A is not sorted, then the time will be bounded by the sorting procedure.
This is based on 2 facts (relevant only when A is ordered):
The closest subsets will always be subsequent
When calculating the closeness of K subsequent elements, the sum of distances can be calculated as the sum of each two subsequent elements time (K-i)*i where i is 1,...,K-1.
When iterating through the sorted array, it is redundant to recompute the entire sum, we can instead remove K times the distance between the previously two smallest elements, and add K times the distance of the two new largest elements. this fact is being used to calculate the closeness of a subset in O(1) by using the closeness of the previous subset.
Here's the pseudo-code
List<pair> FindClosestSubsets(int[] A, int K)
{
List<pair> minList = new List<pair>;
int minVal = infinity;
int tempSum;
int N = A.length;
for (int i = K - 1; i < N; i++)
{
tempSum = 0;
for (int j = i - K + 1; j <= i; j++)
tempSum += (K-i)*i * (A[i] - A[i-1]);
if (tempSum < minVal)
{
minVal = tempSum;
minList.clear();
minList.add(new pair(i-K, i);
}
else if (tempSum == minVal)
minList.add(new pair(i-K, i);
}
return minList;
}
This function will return a list of pairs of indexes representing the optimal solutions (the starting and ending index of each solution), it was implied in the question that you want to return all solutions of the minimal value.

try the following:
N = input()
K = input()
assert 2 <= N <= 10**5
assert 2 <= K <= N
a = some_unsorted_list
a.sort()
cur_diff = sum([abs(a[i] - a[i + 1]) for i in range(K - 1)])
min_diff = cur_diff
min_last_idx = K - 1
for last_idx in range(K,N):
cur_diff = cur_diff - \
abs(a[last_idx - K - 1] - a[last_idx - K] + \
abs(a[last_idx] - a[last_idx - 1])
if min_diff > cur_diff:
min_diff = cur_diff
min_last_idx = last_idx
From the min_last_idx, you can calculate the min_first_idx. I use range to preserve the order of idx. If this is python 2.7, it will take linearly more RAM. This is the same algorithm that you use, but slightly more efficient (smaller constant in complexity), as it does less then summing all.

After sorting, we can be sure that, if x1, x2, ... xk are the solution, then x1, x2, ... xk are contiguous elements, right?
So,
take the intervals between numbers
sum these intervals to get the intervals between k numbers
Choose the smallest of them

My initial solution was to look through all the K element window and multiply each element by m and take the sum in that range, where m is initialized by -(K-1) and incremented by 2 in each step and take the minimum sum from the entire list. So for a window of size 3, m is -2 and the values for the range will be -2 0 2. This is because I observed a property that each element in the K window add a certain weight to the sum. For an example if the elements are [10 20 30] the sum is (30-10) + (30-20) + (20-10). So if we break down the expression we have 2*30 + 0*20 + (-2)*10. This can be achieved in O(n) time and the entire operation would be in O(NK) time. However it turns out that this solution is not optimal, and there are certain edge cases where this algorithm fails. I am yet to figure out those cases, but shared the solution anyway if anyone can figure out something useful from it.
for(i = 0 ;i <= n - k;++i)
{
diff = 0;
l = -(k-1);
for(j = i;j < i + k;++j)
{
diff += a[j]*l;
if(min < diff)
break;
l += 2;
}
if(j == i + k && diff > 0)
min = diff;
}

You can do this is O(n log n) time with a sliding window approach (O(n) if the array is already sorted).
First, suppose we've precomputed, at every index i in our array, the sum of distances from A[i] to the previous k-1 elements. The formula for that would be
(A[i] - A[i-1]) + (A[i] - A[i-2]) + ... + (A[i] - A[i-k+1]).
If i is less than k-1, we just compute the sum to the array boundary.
Suppose we also precompute, at every index i in our array, the sum of distances from A[i] to the next k-1 elements. Then we could solve the whole problem with a single pass of a sliding window.
If our sliding window is on [L, L+k-1] with closeness sum S, then the closeness sum for the interval [L+1, L+k] is just S - dist_sum_to_next[L] + dist_sum_to_prev[L+k]. The only changes in the sum of pairwise distances are removing all terms involving A[L] when it leaves our window, and adding all terms involving A[L+k] as it enters our window.
The only remaining part is how to compute, at a position i, the sum of distances between A[i] and the previous k-1 elements (the other computation is totally symmetric). If we know the distance sum at i-1, this is easy: subtract the distance from A[i-1] to A[i-k], and add in the extra distance from A[i-1] to A[i] k-1 times
dist_sum_to_prev[i] = (dist_sum_to_prev[i - 1] - (A[i - 1] - A[i - k])
+ (A[i] - A[i - 1]) * (k - 1)
Python code:
def closest_subset(nums: List[int], k: int) -> List[int]:
"""Given a list of n (poss. unsorted and non-unique) integers nums,
returns a (sorted) list of size k that minimizes the sum of pairwise
distances between all elements in the list.
Runs in O(n lg n) time, uses O(n) auxiliary space.
"""
n = len(nums)
assert len(nums) == n
assert 2 <= k <= n
nums.sort()
# Sum of pairwise distances to the next (at most) k-1 elements
dist_sum_to_next = [0] * n
# Sum of pairwise distances to the last (at most) k-1 elements
dist_sum_to_prev = [0] * n
for i in range(1, n):
if i >= k:
dist_sum_to_prev[i] = ((dist_sum_to_prev[i - 1] -
(nums[i - 1] - nums[i - k]))
+ (nums[i] - nums[i - 1]) * (k - 1))
else:
dist_sum_to_prev[i] = (dist_sum_to_prev[i - 1]
+ (nums[i] - nums[i - 1]) * i)
for i in reversed(range(n - 1)):
if i < n - k:
dist_sum_to_next[i] = ((dist_sum_to_next[i + 1]
- (nums[i + k] - nums[i + 1]))
+ (nums[i + 1] - nums[i]) * (k - 1))
else:
dist_sum_to_next[i] = (dist_sum_to_next[i + 1]
+ (nums[i + 1] - nums[i]) * (n-i-1))
best_sum = math.inf
curr_sum = 0
answer_right_bound = 0
for i in range(n):
curr_sum += dist_sum_to_prev[i]
if i >= k:
curr_sum -= dist_sum_to_next[i - k]
if curr_sum < best_sum and i >= k - 1:
best_sum = curr_sum
answer_right_bound = i
return nums[answer_right_bound - k + 1:answer_right_bound + 1]