Develop Reference

Can an algorithm with higher Complexity be Faster?

I have written a code for a problem and used 2 double-nested loops within the implementation, but this code runs too long with big O as O(n^2).
So I googled for a faster solution for the same problem and found the second code below, which uses a tripled-nested loop with big O as O(n^3).
Is it because the number of computations is higher for the first code, although it has lower big O?
If so can I conclude that big O is not reliable for small "n" values and I have to do experimentation to be able to judge?
Code 1:
def sherlockAndAnagrams(s):
# 1 . Traverse all possible substrings within string
count = 0
lst_char_poss_str = []
len_s = len(s)
for i in range(len_s):#for each char in string
temp_str = ""#a temp string to include characters next to evaluating char
for j in range(i , len_s):#for all possible length of string from that char
temp_str += s[j] #possible substrings from that char
lst_char_poss_str.append(temp_str)#All possible substrings within string
# 2 . Check if any two substrings of equal length are anagrams
new_lst_char_poss_str = []
for i in lst_char_poss_str:
i = list(i)#sorted list, so, "abb" and "bba" will be both "abb"
i.sort()
new_lst_char_poss_str.append(i)#a 2-d list of lists of characters for All possible substrings within string
len_new_s = len(new_lst_char_poss_str)
for i in range (len_new_s - 1):
for j in range (i + 1, len_new_s):
if new_lst_char_poss_str[i] == new_lst_char_poss_str[j]:
count += 1
return(count)
Code 2:
def sherlockAndAnagrams(s):
count = 0
slen = len(s)
for i in range(slen):
for j in range(i+1, slen):
substr = ''.join(sorted(s[i:j]))#Sortingall characters after a char in string
sublen = len(substr)
for x in range(i+1, slen):
if x + sublen > slen: #if index out of range
break
substr2 = ''.join(sorted(s[x:x+sublen]))
if substr == substr2:
anagrams += 1
return count

You might have an algorithm whose running time is 1,000,000 n, because you may be doing some other operations. But you might have an algorithm of this running time. 1,000,000n is O (n), because this is <= some constant time n and you might have some other algorithm with the running time of 2 n^2.
You would say that 1,000,000 n algorithm is better than 2 n^2. The one with the linear running time which is O (n) running time is better than O ( n^2). It is true but in the limit and the limit is achieved very late when n is really large. For small instances this 2 n^2 might actually take less amount of time than your 1,000,000 n. We must be careful about the constants also.

There are lot of point to be considered:
the second algorithm always return 0, nobody increment count
in the first : temp_str += s[j] is not efficient, in the second this string concatenation is not used.
the second is faster because use slicing to retrieve pieces of the string. but to be sure maybe you must do a precise profile of the code.
other than this, as told by #pjs big O notation is an asymptotical explanation.

Number of ways to form a string of length n using k characters such that at most two adjacent characters can be same

Hi please do let me know what is wrong with my recurrence relation.
My Logic:
The number of ways to allot the first element=k and number of ways such that the next (n-1) elements are different (k-1)*f(n-2)
and number of ways to allot two same elements: k + (k-1)*f(n-1)
Hence the relation:
no. of ways = ∑(k + (k-1)*f(n-1) + k + (k-1)*f(n-1))
I can see some mistakes like the inclusion of duplicates, however, I am unable to figure out the relation.
You can also find the code below:
def count_num_ways(n, k):
dp = [0 for i in range(n+1)]
dp[0] = 0
dp[1] = k
for i in range(2, n+1):
dp[i] = sum(
[
k+( k-1 )*dp[n-2],
k+(k-1)*dp[n-1]-k
]
)
print(dp)
Thanks.

Note: character position in my explanation begins at 1, so the first character is at position 1, the second character at position 2 and so on.
Let's define:
valid string: string formed using k characters such that at most two adjacent characters can be the same.
f(n): number of valid string of length n and the last two characters are different.
g(n): number of valid string of length n and the last two characters are the same.
h(n): number of valid string of length n.
Assume you have already calculated the value of f(m) and g(m) for all m <= n: let's find f(n+1) and g(n+1).
Let's calculate f(n+1): number of valid string of length n+1 and the last two characters are different. Based on every valid string of length n you have found:
if last two characters are different, you have k-1 ways to place a character at position n+1 and form a valid string of length n+1 where last two characters are different.
if last two characters are the same, you have k-1 ways to place a character at position n+1 and form a valid string of length n+1 where last two characters are different.
These two options are clearly mutually excluse because last two characters of every valid string of length n can't be both equals and different. Also, these two options are enough to help us to correctly calculate all valid string of length n+1 where last two characters are different, because every valid string of length n+1 where last two characters are different has the characters at position n-1 and n either different or the same (both options above).
So we can finally say: f(n+1) = (k-1) * f(n) + (k-1) * g(n)
                                                    | first option |    |second option|
Let's calculate g(n+1): number of valid string of length n+1 and the last two characters are the same. Based on every valid string of length n you have found:
if last two characters are different, you have just 1 way to place a character at position n+1 and form a valid string of length n+1 where last two characters are the same.
if last two characters are the same, you have 0 ways to place a character at position n+1 and form a valid string of length n+1 where last two characters are the same, because a valid string has at most two equals adjacent characters. So if characters at position n+1 and n are the same then last 3 characters are the same.
Once again, these two options are clearly mutually excluse because last two characters of every valid string of length n can't be both equals and different. Also, these two options are enough to help us to correctly calculate all valid string of length n+1 where last two characters are the same because every valid string of length n+1 where last two characters are the same has the characters at position n-1 and n either different or the same (both options above).
So we can finally say: g(n+1) = f(n)
                                                 | first option |
Now it's straightforward to see that: h(n) = f(n) + g(n) because last two characters of a valid string of length n are either equals or different.
Given that equation and replacing g(n) with it's value, we have: h(n) = f(n) + f(n-1).
Also, given both f(n) and g(n) just need 1 previous value to be correctly calculated, we have to calculate f(1) and `g(1)`` beforehand.
It's easy to see that:
f(1) = k
g(1) = 0
End of mathematical part. Begin of programming part :)
Note: all solutions are written in python code.
You have 3 ways to do this, both of them calculate h(n) with O(n) time complexity and the third calculates h(n) with O(log n) time complexity. Let's check them!
Solution #1: Recursive
Time Complexity: O(n)
For solution #1 let's ask for values and assume previous values are already calculated (for this we need recursion).
def f(n, k):
return k if n == 1 else (k - 1) * (f(n - 1, k) + g(n - 1, k))
def g(n, k):
return 0 if n == 1 else f(n - 1, k)
def h(n, k):
return f(n, k) + g(n, k)
Solution #2: Iterative
Time Complexity: O(n)
For solution #2 let's calculate the next value of those formulas based on previous already calculated values.
def h(n, k):
fn, gn = k, 0
for _ in range(n):
hn = fn + gn
fn, gn = (k - 1) * (fn + gn), fn
return hn
Solution #3: Matrix Exponentiation and Binary Exponentiation
Time Complexity: O(log n)
Given h(n) = f(n) + g(n) and g(n) = f(n-1) then h(n) = f(n) + f(n-1).
Also, given f(n) = (k-1) * f(n-1) + (k-1) * g(n-1) and g(n) = f(n-1)
then f(n) = (k-1) * f(n-1) + (k-1) * f(n-2).
So with matrix and vectors below and assuming f(0) = 0, we can correctly calculate h(n) as follows:
| k - 1 k - 1 |^(n-1) * | f(n-1) | = | f(n) |
| 1 0 | | f(n-2) | | f(n-1) |
Code:
def h(n, k):
def _matrix_multiplication(matrix_a, matrix_b):
result = [ [ 0 ] * len(matrix_b[ 0 ]) for _ in range(len(matrix_a)) ]
for i in range(len(result)):
for j in range(len(result[ 0 ])):
for k in range(len(matrix_a[ 0 ])):
result[ i ][ j ] += matrix_a[ i ][ k ] * matrix_b[ k ][ j ]
return result
def _binary_exponentiation(matrix, n):
result = [
[ 1, 0 ],
[ 0, 1 ]
]
while n != 0:
if n % 2 == 1:
result = _matrix_multiplication(result, matrix)
n //= 2
matrix = _matrix_multiplication(matrix, matrix)
return result
matrix = [
[ k - 1, k - 1 ],
[ 1, 0 ]
]
matrix = _binary_exponentiation(matrix, n - 1)
vector1 = [ [ k ], [ 0 ] ]
vector2 = _matrix_multiplication(matrix, vector1)
return sum(row[ 0 ] for row in vector2)
I hope any of these 3 solutions help you!

Let f(n) equals the number of ways to form n elements while the last two characters are the same,
---+-+-+-+-+---+-+
...|a|a| | |...| |
---+-+-+-+-+---+-+
\---v---/
n
f(n) = number of ways to fill these n elements.
and let g(n) equals the number of ways to form n elements while the last two characters are different.
---+-+-+-+-+---+-+
...|a|b| | |...| |
---+-+-+-+-+---+-+
\---v---/
n
g(n) = number of ways to fill these n elements.
As f(n), since the last two characters are the same, we need to choose a different character which is (k-1) and the last two characters will be different which is g(n-1), the result is:
f(n) = (k-1)*g(n-1)
As g(n), since the last two characters are different, we can either choose the same character or a different one. For the first case, the same character is the only choice: 1, and the last two characters are the same: f(n-1). For the second case, different character options: (k-1) and the different function: g(n-1), the result is:
g(n) = 1*f(n-1) + (k-1)*g(n-1)
For the first element, we choose a character k, and the answer is
k*g(n-1)
Ps:
You can actually substitute f(n-1) in the second equation, but this is more intuitive I think.
Code
Here is an example Python code
def f(n):
if n == 0:
return 1
return (k-1)*g(n-1)
def g(n):
if n == 0:
return 1
return f(n-1) + (k-1)*g(n-1)
n, k = 4, 2
print(k*g(n-1))

Based on your question, I draw 2 conclusions:
The same character can be used any number of times when forming a string.
k > 1 because otherwise the solution will not exist for n > 2.
In such case, it's a typical combinatorics problem and you can count the number of combinations with a simple math formula. You need to consider 2 cases - all adjacent characters are different, and exactly 2 adjacent characters are the same.
Explanation
Case A - all adjacent characters are different
There are k ways to pick the first character in the string. For the next characters, there are always k - 1 ways (they must be different!). Hence, the total number of combinations is k(k-1)n-1.
Case B - 2 adjacent characters are same
Let's say that 2 identical characters are first and second characters of the string. There are k ways to create such pair because both elements are equal! Then, for every other position in the string, there are k - 1 ways to pick a character.
What, if a pair of identical elements is in the middle or at the end? Well, the number of combinations will remain the same - it's always k ways to pick the first element in the string and k - 1 ways to pick the other elements. If 2 positions have to be occupied by equal characters, then there are, as before, k - 1 ways to set them. Thereof, there are k(k-1)n-2 for a fixed pair position.
Since the number of such positions is n-1, the number of combinations is (n-1)k(k-1)n-2
Result
So, the overall number of combinations is k(k-1)n-1 + (n-1)k(k-1)n-2

Optimal brute force solution for finding longest palindromic substring

This is my current approach:
def isPalindrome(s):
if (s[::-1] == s):
return True
return False
def solve(s):
l = len(s)
ans = ""
for i in range(l):
subStr = s[i]
for j in range(i + 1, l):
subStr += s[j]
if (j - i + 1 <= len(ans)):
continue
if (isPalindrome(subStr)):
ans = max(ans, subStr, key=len)
return ans if len(ans) > 1 else s[0]
print(solve(input()))
My code exceeds the time limit according to the auto scoring system. I've already spend some time to look up on Google, all of the solutions i found have the same idea with no optimization or using dynamic programming, but sadly i must and only use brute force to solve this problem. I was trying to break the loop earlier by skipping all the substrings that are shorter than the last found longest palindromic string, but still end up failing to meet the time requirement. Is there any other way to break these loops earlier or more time-efficient approach than the above?

With subStr += s[j], a new string is created over the length of the previous subStr. And with s[::-1], the substring from the previous offset j is copied over and over again. Both are inefficient because strings are immutable in Python and have to be copied as a new string for any string operation. On top of that, the string comparison in s[::-1] == s is also inefficient because you've already compared all of the inner substrings in the previous iterations and need to compare only the outermost two characters at the current offset.
You can instead keep track of just the index and the offset of the longest palindrome so far, and only slice the string upon return. To account for palindromes of both odd and even lengths, you can either increase the index by 0.5 at a time, or double the length to avoid having to deal with float-to-int conversions:
def solve(s):
length = len(s) * 2
index_longest = offset_longest = 0
for index in range(length):
offset = 0
for offset in range(1 + index % 2, min(index, length - index), 2):
if s[(index - offset) // 2] != s[(index + offset) // 2]:
offset -= 2
break
if offset > offset_longest:
index_longest = index
offset_longest = offset
return s[(index_longest - offset_longest) // 2: (index_longest + offset_longest) // 2 + 1]

Solved by using the approach "Expand Around Center", thanks #Maruthi Adithya

This modification of your code should improve performance. You can stop your code when the max possible substring is smaller than your already computed answer. Also, you should start your second loop with j+ans+1 instead of j+1 to avoid useless iterations :
def solve(s):
l = len(s)
ans = ""
for i in range(l):
if (l-i+1 <= len(ans)):
break
subStr = s[i:len(ans)]
for j in range(i + len(ans) + 1, l+1):
if (isPalindrome(subStr)):
ans = subStr
subStr += s[j]
return ans if len(ans) > 1 else s[0]

This is a solution that has a time complexity greater than the solutions provided.
Note: This post is to think about the problem better and does not specifically answer the question. I have taken a mathematical approach to find a time complexity greater than 2^L (where L is size of input string)
Note: This is a post to discuss potential algorithms. You will not find the answer here. And the logic shown here has not been proven extensively.
Do let me know if there is something that I haven't considered.
Approach: Create set of possible substrings. Compare and find the maximum pair* from this set that has the highest possible pallindrome.
Example case with input string: "abc".
In this example, substring set has: "a","b","c","ab","ac","bc","abc".
7 elements.
Comparing each element with all other elements will involve: 7^2 = 49 calculations.
Hence, input size is 3 & no of calculations is 49.
Time Complexity:
First compute time complexity for generating the substring set:
<img src="https://latex.codecogs.com/gif.latex?\sum_{a=1}^{L}\left&space;(&space;C_{a}^{L}&space;\right&space;)" title="\sum_{a=1}^{L}\left ( C_{a}^{L} \right )" />
(The math equation is shown in the code snippet)
Here, we are adding all the different substring size combination from the input size L.
To make it clear: In the above example input size is 3. So we find all the pairs with size =1 (i.e: "a","b","c"). Then size =2 (i.e: "ab","ac","bc") and finally size = 3 (i.e: "abc").
So choosing 1 character from input string = combination of taking L things 1 at a time without repetition.
In our case number of combinations = 3.
This can be mathematically shown as (where a = 1):
<img src="https://latex.codecogs.com/gif.latex?C_{a}^{L}" title="C_{a}^{L}" />
Similarly choosing 2 char from input string = 3
Choosing 3 char from input string = 1
Finding time complexity of palindrome pair from generated set with maximum length:
Size of generated set: N
For this we have to compare each string in set with all other strings in set.
So N*N, or 2 for loops. Hence the final time complexity is:
<img src="https://latex.codecogs.com/gif.latex?\sum_{a=1}^{L}\left&space;(&space;C_{a}^{L}&space;\right&space;)^{2}" title="\sum_{a=1}^{L}\left ( C_{a}^{L} \right )^{2}" />
This is diverging function greater than 2^L for L > 1.
However, there can be multiple optimizations applied to this. For example: there is no need to compare "a" with "abc" as "a" will also be compared with "a". Even if this optimization is applied, it will still have a time complexity > 2^L (For the most cases).
Hope this gave you a new perspective to the problem.
PS: This is my first post.

You should not find the string start from the beginning of that string, but you should start from the middle of it & expand the current string
For example, for the string xyzabccbalmn, your solution will cost ~ 6 * 11 comparison but searching from the middle will cost ~ 11 * 2 + 2 operations
But anyhow, brute-forcing will never ensure that your solution will run fast enough for any arbitrary string.

Try this:
def solve(s):
if len(s)==1:
print(0)
return '1'
if len(s)<=2 and not(isPalindrome(s)):
print (0)
return '1'
elif isPalindrome(s):
print( len(s))
return '1'
elif isPalindrome(s[0:len(s)-1]) or isPalindrome(s[1:len(s)]):
print (len(s)-1)
return '1'
elif len(s)>=2:
solve(s[0:len(s)-1])
return '1'
return 0

How to understand leetcode 494 Target Sum ( knapsack problem ) fastest python code using bit operation

The problem is described as follows
https://leetcode.com/problems/target-sum/
You are given a list of non-negative integers, a1, a2, ..., an, and a target, S. Now you have 2 symbols + and -. For each integer, you should choose one from + and - as its new symbol.
Find out how many ways to assign symbols to make sum of integers equal to target S.
Constraints:
The length of the given array is positive and will not exceed 20.
The sum of elements in the given array will not exceed 1000.
Your output answer is guaranteed to be fitted in a 32-bit integer.
I find this submission in leetcode submission detail Accepted Solutions Runtime Distribution
class Solution:
def findTargetSumWays(self, nums, S):
a = sum(nums) - S
if a < 0 or a%2==1: return 0
S = [((1<<(i*21))+1) for i in nums]
return reduce(lambda p,i:(p*i)&(1<<((a//2+1)*21))-1,S,1)>>(21*a//2)
Simplify reduce, it becomes
class Solution:
def findTargetSumWays(self, nums, S):
a = sum(nums) - S
if a < 0 or a%2==1: return 0
auxarr = [((1<<(i*21))+1) for i in nums]
ret=1
for i in auxarr:
ret= (ret*i)&(1<<((a//2+1)*21))-1
return ret>>(21*a//2)
It transforms the original problem into another problem that finds the number of selections that select some nums[i] that their sum is (sum(nums)-S)/2.
I know how to solve such knapsack problems with dp, but I can't understand the above code, I am very curious how such code works, please help me.
# my dp code
class Solution:
def findTargetSumWays(self, nums: List[int], S: int) -> int:
S=sum(nums)-S
if S%2!=0 or S<0: return 0
S//=2
dp=[0]*(S+1)
dp[0]=1
for c in nums:
for j in range(S,c-1,-1):
dp[j]+=dp[j-c]
return dp[S]

It seems to use characteristics of a polynomial where you multiply terms formed of (B^n+1) where B is a power of 2 large enough to avoid overlapping.
So, let's say you have 3 numbers (x,y,z) to add, it will compute:
(B^x + 1)(B^y + 1)(B^z + 1)
The exponents of these polynomials will add up in the result
B^(x+y+z) + B^(x+z) + B^(y+z) + B^z + B^(x+y) + B^x + B^y + 1
So, if any combination of exponent (i.e. numbers) adds up to the same total, the number of times B^total occurs will be the number of ways to obtain that total. Leveraging this characteristic of polynomials, we will find ways*B^total in the result. If the number of ways does not overlap with the value of B^(total+1), it can be extracted using masks and integer divisions.
For example, given 4 numbers h,i,j,k, the products will produce the sum of B raised to powers corresponding to every combination of 1 up to 4 of the numbers added together. So, if we are looking for a total of T, and h+i and j+k equal T. Then the product will contain 2*B^T formed by B^(h+i) + B^(j+k). This corresponds to two ways to form the sum T.
Given that there are 2 possibility for each number (+ or -), there is a maximum of 2^20 possible ways to combine them. To make sure that any sum of B^x does not overlap with B^(x+1), the value of 2^21 is chosen for B.
This is why the offset array (variable name S is a really poor choice here) is formed of (B^n+1) for each n in nums, where B is 2^21, so (2^21^n+1) ... (2^(21n)+1) ... (1<<(21*n))+1
To be able to use the polynomial approach, the problem needs to be converted to an Absence/Presence problem. This is done by reasoning that there has to be a combination of numbers that produces a zero sum by cancelling each other out, leaving the rest to be positive and add up to S. So, if we remove S from the total of numbers, there will be a combination that adds up to half of what remains (a//2). This will be the total we will be looking for.
The reduce function implements the polynomial product and applies a mask ((1<<((a//2+1)*21))-1) to cut off any power of B that is beyond B^(a/2). The final result cuts off the part below B^(a/2) by shifting bits.
This results in the multiple of B^(a/2) which corresponds to the number of ways to produce the sum of exponents (i.e the sum of numbers).

How can I profile the algorithm so it will consume less memory space?

There is a problem which is "Last Digit of the Sum of Fibonacci Numbers". I have already optimised the naive approach but this code is not working for higher values. Example: 613455
Everytime I run this program, the os crashes (memory limit exceeds).
My code is:
def SumFib(n):
result = []
for i in range(0, n+1):
if i <= 1:
result.append(i)
else:
result.append(result[i-1] + result[i-2])
return (sum(result) % 10)
print(SumFib(int(input())))
Need some help to get over this problem.

In order to consume less memory, store less values. For example:
def sum_fib(n):
if n == 0:
return 0
elif n == 1:
return 1
elif n == 2:
return 2
res_sum = 2
a, b = 1, 1
for _ in range(n-2):
a, b = b, a+b
res_sum += b
return res_sum % 10
The memory requirements (at a glance) are constant. Your original snippet has lineal memory requirements (when n grows, the memory requirements grow).
(More intelligent things can be done regarding the modulo operation if n is really huge, but that is off topic to the original question)
Edit:
I was curious and ended up doing some homework.
Lemma 1: The last digit of the fibonacci follows a 60-length cycle.
Demonstration: One can check that, starting by (0,1), element 60 has last digit 0 and element 61 has last digit 1 (this can be checked empirically). From this follows that fib(n) equals fib(n % 60).
Lemma 2: The last digit of the sum of the first 60 digits is 0.
Demonstration: Can be checked empirically too.
Conclusion: sum_fib(n) == sum_fib(n % 60). So the algorithm to solve "last digit of the sum of the n-th first fibonacci numbers can be solved in O(1) by taking n % 60 and evaluating sum_fib of that number, i.e., sum_fib(n % 60). Or one can precreate the digit list of 60 elements and do a simple-lookup.

Instead of using range you can try a custom function that works as generator.
Generators calculate values on the fly instead of loading all at once like iterators.
Replace the below function with range
def generate_value(start, end, step):
count =start
while True:
if count <=end:
yield count
count+=step
break