Develop Reference

Find single number in pairs of unique numbers of a Python list in O(lg n)

I have a question for Divide and Conquering in programming algorithms. Suppose you are given a random integer list in Python which consists of:
Unique contiguous pairs of integers
A single integer somewhere in the list
And the conditions are exclusive, meaning while [2,2,1,1,3,3,4,5,5,6,6] is valid, these are not:
[2,2,2,2,3,3,4] (violates condition 1: because there are two pairs of 2s while there can only be a maximum of 1 pair of any number)
[1,4,4,5,5,6,6,1] (violates condition 1: because there is a pair of 1s but they are not contiguous).
[1,4,4,5,5,6,6,3] (violates condition 2: there are 2 single numbers, 1 and 3)
Now the question is can you find the 'single' number index in an O(lgn) algorithm?
My original jab is this:
def single_num(array, arr_max_len):
i = 0
while (i < arr_max_len):
if (arr_max_len - i == 1):
return i
elif (array[i] == array[i + 1]):
i = i + 2
else:
return i # don't have to worry about odd index because it will never happen
return None
However, the algorithm seems to run at O(n/2) time, which seems like the best it could do.
Even if I use divide and conquer, I don't think it's going to get better than O(n/2) time, unless there's some method that's beyond my scope of comprehension at the moment.
Anyone has any better idea, or can I arguably say, this is already in O(log n) time?
EDIT: It seems like Manuel has the best solution, if allowed Ill have some time to implement a solution myself for understanding, and then accept Manuel’s answer.

Solution
Just binary search the even indexes to find the first whose value differs from the next value.
from bisect import bisect
def single_num(a):
class E:
def __getitem__(_, i):
return a[2*i] != a[2*i+1]
return 2 * bisect(E(), False, 0, len(a)//2)
Explanation
Visualization of the virtual "list" E() that I'm searching on:
0 1 2 3 4 5 6 7 8 9 10 (indices)
a = [2, 2, 1, 1, 3, 3, 4, 5, 5, 6, 6]
E() = [False, False, False, True, True]
0 1 2 3 4 (indices)
In the beginning, the pairs match (so != results in False-values). Starting with the single number, the pairs don't match (so != returns True). Since False < True, that's a sorted list which bisect happily searches in.
Alternative implementation
Without bisect, if you're not yet tired of writing binary searches:
def single_num(a):
i, j = 0, len(a) // 2
while i < j:
m = (i + j) // 2
if a[2*m] == a[2*m+1]:
i = m + 1
else:
j = m
return 2*i
Sigh...
I wish bisect would support giving it a callable so I could just do return 2 * bisect(lambda i: a[2*i] != a[2*i+1], False, 0, len(a)//2). Ruby does, and it's perhaps the most frequent reason I sometimes solve coding problems with Ruby instead of Python.
Testing
Btw I tested both with all possible cases for up to 1000 pairs:
from random import random
for pairs in range(1001):
a = [x for _ in range(pairs) for x in [random()] * 2]
single = random()
assert len(set(a)) == pairs and single not in a
for i in range(0, 2*pairs+1, 2):
a.insert(i, single)
assert single_num(a) == i
a.pop(i)

A lg n algorithm is one in which you split the input into smaller parts, and discard some of the smaller part such that you have a smaller input to work with. Since this is a searching problem, the likely solution for a lg n time complexity is binary search, in which you split the input in half each time.
My approach is to start off with a few simple cases, to spot any patterns that I can make use of.
In the following examples, the largest integer is the target number.
# input size: 3
[1,1,2]
[2,1,1]
# input size: 5
[1,1,2,2,3]
[1,1,3,2,2]
[3,1,1,2,2]
# input size: 7
[1,1,2,2,3,3,4]
[1,1,2,2,4,3,3]
[1,1,4,2,2,3,3]
[4,1,1,2,2,3,3]
# input size: 9
[1,1,2,2,3,3,4,4,5]
[1,1,2,2,3,3,5,4,4]
[1,1,2,2,5,3,3,4,4]
[1,1,5,2,2,3,3,4,4]
[5,1,1,2,2,3,3,4,4]
You probably notice that the input size is always an odd number i.e. 2*x + 1.
Since this is a binary search, you can check if the middle number is your target number. If the middle number is the single number (if middle_number != left_number and middle_number != right_number), then you have found it. Otherwise, you have to search the left side or the right side of the input.
Notice that in the sample test cases above, in which the middle number is not the target number, there is a pattern between the middle number and its pair.
For input size 3 (2*1 + 1), if middle_number == left_number, the target number is on the right, and vice versa.
For input size 5 (2*2 + 1), if middle_number == left_number, the target number is on the left, and vice versa.
For input size 7 (2*3 + 1), if middle_number == left_number, the target number is on the right, and vice versa.
For input size 9 (2*4 + 1), if middle_number == left_number, the target number is on the left, and vice versa.
That means the parity of x in 2*x + 1 (the array length) affects whether to search the left or right side of the input: search the right if x is odd and search the left if x is even, if middle_number == left_number (and vice versa).
Base on all these information, you can come up with a recursive solution. Note that you have to ensure that the input size is odd in each recursive call. (Edit: Ensuring that input size is odd makes the code even more messy. You probably want to come up with a solution in which parity of input size does not matter.)
def find_single_number(array: list, start_index: int, end_index: int):
# base case: array length == 1
if start_index == end_index:
return start_index
middle_index = (start_index + end_index) // 2
# base case: found target
if array[middle_index] != array[middle_index - 1] and array[middle_index] != array[middle_index + 1]:
return middle_index
# make use of parity of array length to search left or right side
# end_index == array length - 1
x = (end_index - start_index) // 2
# ensure array length is odd
include_middle = (middle_index % 2 == 0)
if array[middle_index] == array[middle_index - 1]: # middle == number on its left
if x % 2 == 0: # x is even
# search left side
return find_single_number(
array,
start_index,
middle_index if include_middle else middle_index - 1
)
else: # x is odd
# search right side side
return find_single_number(
array,
middle_index if include_middle else middle_index + 1,
end_index,
)
else: # middle == number on its right
if x % 2 == 0: # x is even
# search right side side
return find_single_number(
array,
middle_index if include_middle else middle_index + 1,
end_index,
)
else: # x is odd
# search left side
return find_single_number(
array,
start_index,
middle_index if include_middle else middle_index - 1
)
# test out the code
if __name__ == '__main__':
array = [2,2,1,1,3,3,4,5,5,6,6] # target: 4 (index: 6)
print(find_single_number(array, 0, len(array) - 1))
array = [1,1,2] # target: 2 (index: 2)
print(find_single_number(array, 0, len(array) - 1))
array = [1,1,3,2,2] # target: 3 (index: 2)
print(find_single_number(array, 0, len(array) - 1))
array = [1,1,4,2,2,3,3] # target: 4 (index: 2)
print(find_single_number(array, 0, len(array) - 1))
array = [5,1,1,2,2,3,3,4,4] # target: 5 (index:0)
print(find_single_number(array, 0, len(array) - 1))
My solution is probably not the most efficient or elegant, but I hope my explanation helps you understand the approach towards tackling these kind of algorithmic problems.
Proof that it has a time complexity of O(lg n):
Let's assume that the most important operation is the comparison of the middle number against the left and right number (if array[middle_index] != array[middle_index - 1] and array[middle_index] != array[middle_index + 1]), and that it has a time cost of 1 unit. Let us refer to this comparison as the main comparison.
Let T be time cost of the algorithm.
Let n be the length of the array.
Since this solution involves recursion, there is a base case and recursive case.
For the base case (n = 1), it is just the main comparison, so:
T(1) = 1.
For the recursive case, the input is split in half (either left half or right half) each time; at the same time, there is one main comparison. So:
T(n) = T(n/2) + 1
Now, I know that the input size must always be odd, but let us assume that n = 2k for simplicity; the time complexity would still be the same.
We can rewrite T(n) = T(n/2) + 1 as:
T(2k) = T(2k-1) + 1
Also, T(1) = 1 is:
T(20) = 1
When we expand T(2k) = T(2k-1) + 1, we get:
T(2k)
= T(2k-1) + 1
= [T(2k-2) + 1] + 1 = T(2k-2) + 2
= [T(2k-3) + 1] + 2 = T(2k-3) + 3
= [T(2k-4) + 1] + 3 = T(2k-4) + 4
= ...(repeat until k)
= T(2k-k) + k = T(20) + k = k + 1
Since n = 2k, that means k = log2 n.
Substituting n back in, we get:
T(n) = log2 n + 1
1 is a constant so it can be dropped; same goes for the base of the log operation.
Therefore, the upperbound of the time complexity of the algorithm is:
T(n) = lg n

How to find even sum

I need to find the sum of all even numbers below the inserted number. For example if I insert 8 then the sum would be 2+4+6+8=20. If I insert 9 then it also needs to be 20. And it needs to be based on recursion.
This is what I have so far:
def even(a):
if a == 0:
else:
even(a - 1)
even(8)
I cannot figure out what to change under the "if" part for it to give the right outcome

If the function is called with an odd number, n, then you can immediately call again with the number below (an even).
Then if the function is called with an even number return that even number plus the result of summing all the even numbers below this number by calling again with n - 2.
Finally, your base case occurs when n = 0. In this case just return 0.
So we have
def even_sum(n):
if n % 2 == 1: # n is odd
return even_sum(n - 1)
if n == 0:
return 0
return n + even_sum(n - 2)
which works as expected
>>> even_sum(8)
20
>>> even_sum(9)
20
>>> even_sum(0)
0

To design a recursive algorithm, the first thing to wonder is "In what cases can my algorithm return an answer trivially?". In your case, the answer is "If it is called with 0, the algorithm answers 0". Hence, you can write:
def even(n):
if n == 0:
return 0
Now the next question is "Given a particular input, how can I reduce the size of this input, so that it will eventually reach the trivial condition?"
If you have an even number, you want to have this even number + the sum of even numbers below it, which is the result of even(n-2). If you have an odd number, you want to return the sum of even numbers below it. Hence the final version of your function is:
def even(n):
if n == 0 or n == 1:
return 0
if n % 2 == 0:
return n + even(n - 2)
return even(n - 1)

Both with o(n) time complexity
With For loop
num = int(input("Enter a number: ")) # given number to find sum
my_sum = 0
for n in range(num + 1):
if n % 2 == 0:
my_sum += n
print(my_sum)
With recursion
def my_sum(num):
if num == 0:
return 0
if num % 2 == 1:
return my_sum(num - 1)
return num + my_sum(num - 2)
always avoid O(n^2) and greater time complexity

For a recursive solution:
def evenSum(N): return 0 if N < 2 else N - N%2 + evenSum(N-2)
If you were always given an even number as input, you could simply recurse using N + f(N-2).
For example: 8 + ( 6 + (4 + ( 2 + 0 ) ) )
But the odd numbers will require that you strip the odd bit in the calculation (e.g. subtracting 1 at each recursion)
For example: 9-1 + ( 7-1 + ( 5-1 + ( 3-1 + 0 ) ) )
You can achieve this stripping of odd bits by subtracting the modulo 2 of the input value. This subtracts zero for even numbers and one for odd numbers.
adjusting your code
Your approach is recursing by 1, so it will go through both the even and odd numbers down to zero (at which point it must stop recursing and simply return zero).
Here's how you can adjust it:
Return a value of zero when you are given zero as input
Make sure to return the computed value that comes from the next level of recursion (you are missing return in front of your call to even(a-1)
Add the parameter value when it is even but don't add it when it is odd
...
def even(a):
if a == 0 : return 0 # base case, no further recusion
if a%2 == 1 : return even(a-1) # odd number: skip to even number
return a + even(a-1) # even number: add with recursion
# a+even(a-2) would be better
A trick to create a recursive function
An easy way to come up with the structure of a recursive function is to be very optimistic and imagine that you already have one that works. Then determine how you would use the result of that imaginary function to produce the next result. That will be the recursive part of the function.
Finally, find a case where you would know the answer without using the function. That will be your exit condition.
In this case (sum of even numbers), imagine you already have a function magic(x) that gives you the answer for x. How would you use it to find a solution for n given the result of magic(n-1) ?
If n is even, add it to magic(n-1). If n is odd, use magic(n-1) directly.
Now, to find a smaller n where we know the answer without using magic(). Well if n is less than 2 (or zero) we know that magic(n) will return zero so we can give that result without calling it.
So our recursion is "n+magic(n-1) if n is even, else magic(n-1)"
and our stop condition is "zero if n < 2"
Now substitute magic with the name of your function and the magic is done.
For an O(1) solution:
Given that the sum of numbers from 1 to N can be calculated with N*(N+1)//2, you can get half of the sum of even numbers if you use N//2 in the formula. Then multiply the result by 2 to obtain the sum of even numbers.
so (N//2)*(N//2+1) will give the answer directly in O(1) time:
N = 8
print((N//2)*(N//2+1))
# 20
# other examples:
for N in range(10):
print(N,N//2*(N//2+1))
# 0 0
# 1 0
# 2 2
# 3 2
# 4 6
# 5 6
# 6 12
# 7 12
# 8 20
# 9 20
Visually, you can see the progression like this:
1..n : 1 2 3 4 5 6 7 8
∑n : 1 3 6 10 15 21 28 36 n(n+1)/2
n/2 : 0 1 1 2 2 3 3 4
1..n/2 : 1 2 3 4
∑n/2 : 1 3 5 10 half of the result
2∑n/2 : 2 6 10 20 sum of even numbers
So we simply replace N with N//2 in the formula and multiply the result by 2:
N*(N+1)//2 --> replace N with N//2 --> N//2*(N//2+1)//2
N//2*(N//2+1)//2 --> multiply by 2 --> N//2*(N//2+1)
Another way to see it is using Gauss's visualisation of the sum of numbers but using even numbers:
ascending 2 4 6 8 ... N-6 N-4 N-2 N (where N is even)
descending N N-2 N-4 N-6 ... 8 6 4 2
--- --- --- --- --- --- --- ---
totals N+2 N+2 N+2 N+2 ... N+2 N+2 N+2 N+2 (N/2 times N+2)
Because we added the even numbers twice, once in ascending order and once in descending order, the sum of all the totals will be twice the sum of even numbers (we need to divide that sum by 2 to get what we are looking for).
sum of evens: N/2*(N+2)/2 --> N/2*(N/2+1)
The N/2(N/2+1) formulation allows us to supply the formula with an odd number and get the right result by using integer division which absorbs the 'odd bit': N//2(N//2+1)
Recursive O(1) solution
Instead of using the integer division to absorb the odd bit, you could use recursion with the polynomial form of N/2*(N+2)/2: N^2/4 + N/2
def sumEven(n):
if n%2 == 0 : return n**2/4 + n/2 # exit condition
return sumEven(n-1) # recursion
Technically this is recursive although in practice it will never go deeper than 1 level

Try out this.
>>> n = 5
>>> sum(range(0, n+1, 2))

with minimum complexity
# include <stdio.h>
void main()
{
int num, sum, i;
printf("Number: ");
scanf("%d", &num);
i = num;
if (num % 2 != 0)
num = num -1;
sum = (num * (num + 2)) / 4;
printf("The sum of even numbers upto %d is %d\n\n", i, sum);
}
It is a C program and could be used in any language with respective syntax.

And it needs to be based on recursion.
Though you want a recursion one, I still want to share this dp solution with detailed steps to solve this problem.
Dynamic Programming
dp[i] represents the even sum among [0, i] which I denote as nums.
Case1: When i is 0, there is one number 0 in nums. dp[0] is 0.
Case2: When i is 1, there are two numbers 0 and 1 in nums. dp[1] is still 0.
Case3: When i is 2, there are three numbers 0, 1 and 2 in nums. dp[2] is 2.
Case4: When i is greater than 2, there are two more cases
If i is odd, dp[i] = dp[i-1]. Since i is odd, it is the same with [0, i-1].
If i is even, dp[i] = dp[i-2] + i by adding the current even number to the even sum among [0, i-2] (i-1 is odd, so won't be added).
PS. dp[i] = dp[i-1] + i is also ok. The difference is how you initialize dp.
Since we want the even sum among [0, n], we return dp[n]. You can conclude this from the first three cases.
def even_sum(n):
dp = []
# Init
dp.append(0) # dp[0] = 0
dp.append(0) # dp[1] = 0
# DP
for i in range(2, n+1): # n+1 because range(i, j) results [i, j) and you take n into account
if i % 2 == 1: # n is odd
dp.append(dp[i-1]) # dp[i] = dp[i-1]
else: # n is even
dp.append(dp[i-2] + i) # dp[i] = dp[i-2] + i
return dp[-1]

How to write 2**n - 1 as a recursive function?

I need a function that takes n and returns 2n - 1 . It sounds simple enough, but the function has to be recursive. So far I have just 2n:
def required_steps(n):
if n == 0:
return 1
return 2 * req_steps(n-1)
The exercise states: "You can assume that the parameter n is always a positive integer and greater than 0"

2**n -1 is also 1+2+4+...+2n-1 which can made into a single recursive function (without the second one to subtract 1 from the power of 2).
Hint: 1+2*(1+2*(...))
Solution below, don't look if you want to try the hint first.
This works if n is guaranteed to be greater than zero (as was actually promised in the problem statement):
def required_steps(n):
if n == 1: # changed because we need one less going down
return 1
return 1 + 2 * required_steps(n-1)
A more robust version would handle zero and negative values too:
def required_steps(n):
if n < 0:
raise ValueError("n must be non-negative")
if n == 0:
return 0
return 1 + 2 * required_steps(n-1)
(Adding a check for non-integers is left as an exercise.)

To solve a problem with a recursive approach you would have to find out how you can define the function with a given input in terms of the same function with a different input. In this case, since f(n) = 2 * f(n - 1) + 1, you can do:
def required_steps(n):
return n and 2 * required_steps(n - 1) + 1
so that:
for i in range(5):
print(required_steps(i))
outputs:
0
1
3
7
15

You can extract the really recursive part to another function
def f(n):
return required_steps(n) - 1
Or you can set a flag and define just when to subtract
def required_steps(n, sub=True):
if n == 0: return 1
return 2 * required_steps(n-1, False) - sub
>>> print(required_steps(10))
1023

Using an additional parameter for the result, r -
def required_steps (n = 0, r = 1):
if n == 0:
return r - 1
else:
return required_steps(n - 1, r * 2)
for x in range(6):
print(f"f({x}) = {required_steps(x)}")
# f(0) = 0
# f(1) = 1
# f(2) = 3
# f(3) = 7
# f(4) = 15
# f(5) = 31
You can also write it using bitwise left shift, << -
def required_steps (n = 0, r = 1):
if n == 0:
return r - 1
else:
return required_steps(n - 1, r << 1)
The output is the same

Have a placeholder to remember original value of n and then for the very first step i.e. n == N, return 2^n-1
n = 10
# constant to hold initial value of n
N = n
def required_steps(n, N):
if n == 0:
return 1
elif n == N:
return 2 * required_steps(n-1, N) - 1
return 2 * required_steps(n-1, N)
required_steps(n, N)

One way to get the offset of "-1" is to apply it in the return from the first function call using an argument with a default value, then explicitly set the offset argument to zero during the recursive calls.
def required_steps(n, offset = -1):
if n == 0:
return 1
return offset + 2 * required_steps(n-1,0)

On top of all the awesome answers given earlier, below will show its implementation with inner functions.
def outer(n):
k=n
def p(n):
if n==1:
return 2
if n==k:
return 2*p(n-1)-1
return 2*p(n-1)
return p(n)
n=5
print(outer(n))
Basically, it is assigning a global value of n to k and recursing through it with appropriate comparisons.

Efficiently generating Stern's Diatomic Sequence

Stern's Diatomic Sequence can be read about in more details over here; however, for my purpose I will define it now.
Definition of Stern's Diatomic Sequence
Let n be a number to generate the fusc function out of. Denoted fusc(n).
If n is 0 then the returned value is 0.
If n is 1 then the returned value is 1.
If n is even then the returned value is fusc(n / 2).
If n is odd then the returned value is fusc((n - 1) / 2) + fusc((n + 1) / 2).
Currently, my Python code brute forces through most of the generation, other than the dividing by two part since it will always yield no change.
def fusc (n):
if n <= 1:
return n
while n > 2 and n % 2 == 0:
n /= 2
return fusc((n - 1) / 2) + fusc((n + 1) / 2)
However, my code must be able to handle digits in the magnitude of 1000s millions of bits, and recursively running through the function thousands millions of times does not seem very efficient or practical.
Is there any way I could algorithmically improve my code such that massive numbers can be passed through without having to recursively call the function so many times?

With memoization for a million bits, the recursion stack would be extremely large. We can first try to look at a sufficiently large number which we can work by hand, fusc(71) in this case:
fusc(71) = fusc(35) + fusc(36)
fusc(35) = fusc(17) + fusc(18)
fusc(36) = fusc(18)
fusc(71) = 1 * fusc(17) + 2 * fusc(18)
fusc(17) = fusc(8) + fusc(9)
fusc(18) = fusc(9)
fusc(71) = 1 * fusc(8) + 3 * fusc(9)
fusc(8) = fusc(4)
fusc(9) = fusc(4) + fusc(5)
fusc(71) = 4 * fusc(4) + 3 * fusc(5)
fusc(4) = fusc(2)
fusc(3) = fusc(1) + fusc(2)
fusc(71) = 7 * fusc(2) + 3 * fusc(3)
fusc(2) = fusc(1)
fusc(3) = fusc(1) + fusc(2)
fusc(71) = 11 * fusc(1) + 3 * fusc(2)
fusc(2) = fusc(1)
fusc(71) = 14 * fusc(1) = 14
We realize that we can avoid recursion completely in this case as we can always express fusc(n) in the form a * fusc(m) + b * fusc(m+1) while reducing the value of m to 0. From the example above, you may find the following pattern:
if m is odd:
a * fusc(m) + b * fusc(m+1) = a * fusc((m-1)/2) + (b+a) * fusc((m+1)/2)
if m is even:
a * fusc(m) + b * fusc(m+1) = (a+b) * fusc(m/2) + b * fusc((m/2)+1)
Therefore, you may use a simple loop function to solve the problem in O(lg(n)) time
def fusc(n):
if n == 0: return 0
a = 1
b = 0
while n > 0:
if n%2:
b = b + a
n = (n-1)/2
else:
a = a + b
n = n/2
return b

lru_cache works wonders in your case. make sure maxsize is a power of 2. may need to fiddle a bit with that size for your application. cache_info() will help with that.
also use // instead of / for integer division.
from functools import lru_cache
#lru_cache(maxsize=512, typed=False)
def fusc(n):
if n <= 1:
return n
while n > 2 and n % 2 == 0:
n //= 2
return fusc((n - 1) // 2) + fusc((n + 1) // 2)
print(fusc(1000000000078093254329870980000043298))
print(fusc.cache_info())
and yes, this is just meomization as proposed by Filip Malczak.
you might gain an additional tiny speedup using bit-operations in the while loop:
while not n & 1: # as long as the lowest bit is not 1
n >>= 1 # shift n right by one
UPDATE:
here is a simple way of doing meomzation 'by hand':
def fusc(n, _mem={}): # _mem will be the cache of the values
# that have been calculated before
if n in _mem: # if we know that one: just return the value
return _mem[n]
if n <= 1:
return n
while not n & 1:
n >>= 1
if n == 1:
return 1
ret = fusc((n - 1) // 2) + fusc((n + 1) // 2)
_mem[n] = ret # store the value for next time
return ret
UPDATE
after reading a short article by dijkstra himself a minor update.
the article states, that f(n) = f(m) if the fist and last bit of m are the same as those of n and the bits in between are inverted. the idea is to get n as small as possible.
that is what the bitmask (1<<n.bit_length()-1)-2 is for (first and last bits are 0; those in the middle 1; xoring n with that gives m as described above).
i was only able to do small benchmarks; i'm interested if this is any help at all for the magitude of your input... this will reduce the memory for the cache and hopefully bring some speedup.
def fusc_ed(n, _mem={}):
if n <= 1:
return n
while not n & 1:
n >>= 1
if n == 1:
return 1
# https://www.cs.utexas.edu/users/EWD/transcriptions/EWD05xx/EWD578.html
# bit invert the middle bits and check if this is smaller than n
m = n ^ (1<<n.bit_length()-1)-2
n = m if m < n else n
if n in _mem:
return _mem[n]
ret = fusc(n >> 1) + fusc((n >> 1) + 1)
_mem[n] = ret
return ret
i had to increase the recursion limit:
import sys
sys.setrecursionlimit(10000) # default limit was 1000
benchmarking gave strange results; using the code below and making sure that i always started a fresh interperter (having an empty _mem) i sometimes got significantly better runtimes; on other occasions the new code was slower...
benchmarking code:
print(n.bit_length())
ti = timeit('fusc(n)', setup='from __main__ import fusc, n', number=1)
print(ti)
ti = timeit('fusc_ed(n)', setup='from __main__ import fusc_ed, n', number=1)
print(ti)
and these are three random results i got:
6959
24.117448464001427
0.013900151001507766
6989
23.92404893300045
0.013844672999766772
7038
24.33894686200074
24.685758719999285
that is where i stopped...

How to calculate no. of palindroms in a large number interval?

I want to calculate how many numbers are palindrome in large interval data say 10^15
My simple code (python) snippet is:
def count_palindromes(start, end):
count = 0
for i in range(start, end + 1):
if str(i) == str(i)[::-1]:
count += 1
return count
start = 1000 #some initial number
end = 10000000000000 #some other large number
if __name__ == "__main__":
print count_palindromes(start, end)
Its a simple program which checks each number one by one. Its vary time consuming and takes a lot of computer resources.
Is there any other method/technique by which we can count Palindrome numbers? Any Algorithm to use for this?
I want to minimize time taken in producing the output.

When you want to count the numbers having some given property between two limits, it is often useful to solve the somewhat simpler problem
How many numbers with the given property are there between 0 and n?
Keeping one limit fixed can make the problem significantly simpler to tackle. When the simpler problem is solved, you can get the solution to the original problem with a simple subtraction:
countBetween(a,b) = countTo(b) - countTo(a)
or countTo(b ± 1) - countTo(a ± 1), depending on whether the limit is included in countTo and which limits shall be included in countBetween.
If negative limits can occur (not for palindromes, I presume), countTo(n) should be <= 0 for negative n (one can regard the function as an integral with respect to the counting measure).
So let us determine
palindromes_below(n) = #{ k : 0 <= k < n, k is a palindrome }
We get more uniform formulae for the first part if we pretend that 0 is not a palindrome, so for the first part, we do that.
Part 1: How many palindromes with a given number d of digits are there?
The first digit cannot be 0, otherwise it's unrestricted, hence there are 9 possible choices (b-1 for palindromes in an arbitrary base b).
The last digit is equal to the first by the fact that it shall be a palindrome.
The second digit - if d >= 3 - can be chosen arbitrarily and independently from the first. That also determines the penultimate digit.
If d >= 5, one can also freely choose the third digit, and so on.
A moment's thought shows that for d = 2*k + 1 or d = 2*k + 2, there are k digits that can be chosen without restriction, and one digit (the first) that is subject to the restriction that it be non-zero. So there are
9 * 10**k
d-digit palindromes then ((b-1) * b**k for base b).
That's a nice and simple formula. From that, using the formula for a geometric sum, we can easily obtain the number of palindromes smaller than 10n (that is, with at most n digits):
if n is even, the number is
n/2-1 n/2-1
2 * ∑ 9*10**k = 18 * ∑ 10**k = 18 * (10**(n/2) - 1) / (10 - 1) = 2 * (10**(n/2) - 1)
k=0 k=0
if n is odd, the number is
2 * (10**((n-1)/2) - 1) + 9 * 10**((n-1)/2) = 11 * (10**((n-1)/2) - 2
(for general base b, the numbers are 2 * (b**(n/2) - 1) resp. (b+1) * b**((n-1)/2) - 2).
That's not quite as uniform anymore, but still simple enough:
def palindromes_up_to_n_digits(n):
if n < 1:
return 0
if n % 2 == 0:
return 2*10**(n//2) - 2
else:
return 11*10**(n//2) - 2
(remember, we don't count 0 yet).
Now for the remaining part. Given n > 0 with k digits, the palindromes < n are either
palindromes with fewer than k digits, there are palindromes_up_to_n_digits(k-1) of them, or
palindromes with exactly k digits that are smaller than n.
So it remains to count the latter.
Part 2:
Letm = (k-1)//2 and
d[1] d[2] ... d[m] d[m+1] ... d[k]
the decimal representation of n (the whole thing works with the same principle for other bases, but I don't explicitly mention that in the following), so
k
n = ∑ d[j]*10**(k-j)
j=1
For each 1 <= c[1] < d[1], we can choose the m digits c[2], ..., c[m+1] freely to obtain a palindrome
p = c[1] c[2] ... c[m+1] {c[m+1]} c[m] ... c[2] c[1]
(the digit c[m+1] appears once for odd k and twice for even k). Now,
c[1]*(10**(k-1) + 1) <= p < (c[1] + 1)*10**(k-1) <= d[1]*10**(k-1) <= n,
so all these 10**m palindromes (for a given choice of c[1]!) are smaller than n.
Thus there are (d[1] - 1) * 10**m k-digit palindromes whose first digit is smaller than the first digit of n.
Now let us consider the k-digit palindromes with first digit d[1] that are smaller than n.
If k == 2, there is one if d[1] < d[2] and none otherwise. If k >= 3, for each 0 <= c[2] < d[2], we can freely choose the m-1 digits c[3] ... c[m+1] to obtain a palindrome
p = d[1] c[2] c[3] ... c[m] c[m+1] {c[m+1]} c[m] ... c[3] c[2] d[1]
We see p < n:
d[1]*(10**(k-1) + 1) + c[2]*(10**(k-2) + 10)
<= p < d[1]*(10**(k-1) + 1) + (c[2] + 1)*(10**(k-2) + 10)
<= d[1]*(10**(k-1) + 1) + d[2]*(10**(k-2) + 10) <= n
(assuming k > 3, for k == 3 replace 10**(k-2) + 10 with 10).
So that makes d[2]*10**(m-1) k-digit palindromes with first digit d[1] and second digit smaller than d[2].
Continuing, for 1 <= r <= m, there are
d[m+1]*10**(m-r)
k-digit palindromes whose first r digits are d[1] ... d[r] and whose r+1st digit is smaller than d[r+1].
Summing up, there are
(d[1]-1])*10**m + d[2]*10**(m-1) + ... + d[m]*10 + d[m+1]
k-digit palindromes that have one of the first m+1 digits smaller than the corresponding digit of n and all preceding digits equal to the corresponding digit of n. Obviously, these are all smaller than n.
There is one k-digit palindrome p whose first m+1 digits are d[1] .. d[m+1], we must count that too if p < n.
So, wrapping up, and now incorporating 0 too, we get
def palindromes_below(n):
if n < 1:
return 0
if n < 10:
return n # 0, 1, ..., n-1
# General case
dec = str(n)
digits = len(dec)
count = palindromes_up_to_n_digits(digits-1) + 1 # + 1 for 0
half_length = (digits-1) // 2
front_part = dec[0:half_length + 1]
count += int(front_part) - 10**half_length
i, j = half_length, half_length+1
if digits % 2 == 1:
i -= 1
while i >= 0 and dec[i] == dec[j]:
i -= 1
j += 1
if i >= 0 and dec[i] < dec[j]:
count += 1
return count
Since the limits are both to be included in the count for the given problem (unless the OP misunderstood), we then have
def count_palindromes(start, end):
return palindromes_below(end+1) - palindromes_below(start)
for a fast solution:
>>> bench(10**100,10**101-1)
900000000000000000000000000000000000000000000000000 palindromes between
10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
and
99999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999
in 0.000186920166016 seconds

Actually, it's a problem for Google Codejam (which I'm pretty sure you're not supposed to get outside help on) but alas, I'll throw in my 2 cents.
The idea I came up with (but failed to implement) for the large problem was to precompile (generated at runtime, not hardcoded into the source) a list of all palindromic numbers less than 10^15 (there's not very many, it takes like ~60 seconds) then find out how many of those numbers lie between the bounds of each input.
EDIT: This won't work on the 10^100 problem, like you said, that would be a mathematical solution (although there is a pattern if you look, so you'd just need an algorithm to generate all numbers with that pattern)

I presume this is for something like Project Euler... my rough idea would be to generate all numbers up to half the length of your limit (like, if you're going to 99999, go up to 99). Then reverse them, append them to the unreversed one, and potentially add a digit in the middle (for the numbers with odd lengths). You'll might have to do some filtering for duplicates, or weird ones (like if you had a zero at the beginning of the number or sommat) but that should be a lot faster than what you were doing.