Overall Problem: Project Euler 12 - What is the value of the first triangle number to have over five hundred divisors?
Focus of problem: The divisor function
Language: Python
Description: The function I used is brute and the time it take for the program to find a number with more divisors than x increases almost exponentially with each 10 or 20 numbers highers. I need to get to 500 or more divisors. I've identified that the divisor function is what is hogging down the program. The research I did lead me to divisor functions and specifically the divisor function which is supposed to be a function that will count all the divisors of any integer. Every page I've looked at seems to be directed toward mathematics majors and I only have high-school maths. Although I did come across some page that mentioned allot about primes and the Sieve of Atkins but I could not make the connection between primes and finding all the divisors of any integer nor find anything on the net about it.
Main Question: Could someone explain how to code the divisor function or even provide a sample? Maths concepts make more sense to me when I look at them with code. So much appreciated.
brute force divisor function:
def countdiv(a):
count = 0
for i in range(1,(a/2)+1):
if a % i == 0:
count += 1
return count + 1 # +1 to account for number itself as a divisor
If you need a bruteforce function to calculate Number of Divisors (also known as tau(n))
Here's what it looks like
def tau(n):
sqroot,t = int(n**0.5),0
for factor in range(1,sqroot+1):
if n % factor == 0:
t += 2 # both factor and N/factor
if sqroot*sqroot == n: t = t - 1 # if sqroot is a factor then we counted it twice, so subtract 1
return t
The second method involves a decomposing n into its prime factors (and its exponents).
tau(n) = (e1+1)(e2+1)....(em+1) where n = p1^e1 * p2^e2 .... pm^em and p1,p2..pm are primes
More info here
The third method and much more simpler to understand is simply using a Sieve to calculate tau.
def sieve(N):
t = [0]*(N+1)
for factor in range(1,N+1):
for multiple in range(factor,N+1,factor):
t[multiple]+=1
return t[1:]
Here's it in action at ideone
I agree with the two other answers submitted here in that you will only need to search up to the square root of the number. I have one thing to add to this however. The solutions offered will get you the correct answer in a reasonable amount of time. But when the problems start getting tougher, you will need an even more powerful function.
Take a look at Euler's Totient function. Though it only indirectly applies here, it is incredibly useful in later problems. Another related concept is that of Prime Factorization.
A quick way to improve your algorithm is to find the prime factorization of the number. In the Wikipedia article, they use 36 as an example, whose prime factorization is 2^2 * 3^2. Therefore, knowing this, you can use combinatorics to find the number of factors of 36. With this, you will not actually be computing each factor, plus you'd only have to check divisors 2 and 3 before you're complete.
When searching for divisors of n you never have to search beyond the square root of the number n. Whenever you find a divisor that's less than sqrt(n) there is exactly one matching divisor which is greater than the root, so you can increment your count by 2 (if you find divisor d of n then n/d will be the counterpart).
Watch out for square numbers, though. :) The root will be a divisor that doesn't count twice, of course.
If you're going to solve the Project Euler problems you need some functions that deal with prime numbers and integer factorization. Here is my modest library, which provides primes(n), is_prime(n) and factors(n); the focus is on simplicity, clarity and brevity at the expense of speed, though these functions should be sufficient for Project Euler:
def primes(n):
"""
list of primes not exceeding n in ascending
order; assumes n is an integer greater than
1; uses Sieve of Eratosthenes
"""
m = (n-1) // 2
b = [True] * m
i, p, ps = 0, 3, [2]
while p*p < n:
if b[i]:
ps.append(p)
j = 2*i*i + 6*i + 3
while j < m:
b[j] = False
j = j + 2*i + 3
i += 1; p += 2
while i < m:
if b[i]:
ps.append(p)
i += 1; p += 2
return ps
def is_prime(n):
"""
False if n is provably composite, else
True if n is probably prime; assumes n
is an integer greater than 1; uses
Miller-Rabin test on prime bases < 100
"""
ps = [2,3,5,7,11,13,17,19,23,29,31,37,41,
43,47,53,59,61,67,71,73,79,83,89,97]
def is_spsp(n, a):
d, s = n-1, 0
while d%2 == 0:
d /= 2; s += 1
if pow(a,d,n) == 1:
return True
for r in xrange(s):
if pow(a, d*pow(2,r), n) == n-1:
return True
return False
if n in ps: return True
for p in ps:
if not is_spsp(n,p):
return False
return True
def factors(n):
"""
list of prime factors of n in ascending
order; assumes n is an integer, may be
positive, zero or negative; uses Pollard's
rho algorithm with Floyd's cycle finder
"""
def gcd(a,b):
while b: a, b = b, a%b
return abs(a)
def facts(n,c,fs):
f = lambda(x): (x*x+c) % n
if is_prime(n): return fs+[n]
t, h, d = 2, 2, 1
while d == 1:
t = f(t); h = f(f(h))
d = gcd(t-h, n)
if d == n:
return facts(n, c+1, fs)
if is_prime(d):
return facts(n//d, c+1, fs+[d])
return facts(n, c+1, fs)
if -1 <= n <= 1: return [n]
if n < -1: return [-1] + factors(-n)
fs = []
while n%2 == 0:
n = n//2; fs = fs+[2]
if n == 1: return fs
return sorted(facts(n,1,fs))
Once you know how to factor a number, it is easy to count the number of divisors. Consider 76576500 = 2^2 * 3^2 * 5^3 * 7^1 * 11^1 * 13^1 * 17^1. Ignore the bases and look at the exponents, which are 2, 2, 3, 1, 1, 1, and 1. Add 1 to each exponent, giving 3, 3, 4, 2, 2, 2, and 2. Now multiply that list to get the number of divisors of the original number 76576500: 3 * 3 * 4 * 2 * 2 * 2 * 2 = 576. Here's the function:
def numdiv(n):
fs = factors(n)
f = fs.pop(0); d = 1; x = 2
while fs:
if f == fs[0]:
x += 1
else:
d *= x; x = 2
f = fs.pop(0)
return d * x
You can see these functions at work at http://codepad.org/4j8qp60u, and learn more about how they work at my blog. I'll leave it to you to work out the solution to Problem 12.
Related
I'm writing a script to crack small RSA keys. I've thought of a way to save massive amounts of time in the methods I'm using. To do this, I need to know how to find the largest possible prime factor of a number, of a certain size. For example:
N = 9868690954602957859
N_MAX_FACTOR_BITSIZE = 64
Then, I run N through this algorithm:
p = floor(sqrt(n))
if p mod 2 == 0:
p += 1
while p < N: # Referenced above
if p mod n == 0:
return p
p += 2
This algorithm works on the principle that there are only two prime numbers above floor(sqrt(N)) that will divide N evenly. Seeing as both numbers will be prime, the algorithm only checks odd numbers.
To shorten the amount of time this algorithm takes, I would like to swap the N in while p < N with the largest odd 64 bit number that could multiply into N.
EG an algorithim that takes N and N_MAX_FACTOR_BITSIZE as arguments and returns the largest odd factor of N that is N_MAX_FACTOR_BITSIZE long.
The number it returns must be N_MAX_FACTOR_BITSIZE bits long.
Any help would be appreciated.
def prime_factors(n):
"""Returns all the prime factors of a positive integer"""
factors = []
d = 2
while n > 1:
while n % d == 0:
factors.append(d)
n /= d
d = d + 1
return factors
n = 9868690954602957859
primes =prime_factors(n)[-1]
print(primes)
My code is very slow when it comes to very large numbers.
def divisors(num):
divs = 1
if num == 1:
return 1
for i in range(1, int(num/2)):
if num % i == 0:
divs += 1
elif int(num/2) == i:
break
else:
pass
return divs
For 10^9 i get a run time of 381.63s.
Here is an approach that determines the multiplicities of the various distinct prime factors of n. Each such power, k, contributes a factor of k+1 to the total number of divisors.
import math
def smallest_divisor(p,n):
#returns the smallest divisor of n which is greater than p
for d in range(p+1,1+math.ceil(math.sqrt(n))):
if n % d == 0:
return d
return n
def divisors(n):
divs = 1
p = 1
while p < n:
p = smallest_divisor(p,n)
k = 0
while n % p == 0:
k += 1
n //= p
divs *= (k+1)
return divs - 1
It returns the number of proper divisors (so not counting the number itself). If you want to count the number itself, don't subtract 1 from the result.
It works rapidly with numbers of the size 10**9, though will slow down dramatically with even larger numbers.
Division is expensive, multiplication is cheap.
Factorize the number into primes. (Download the list of primes, keep dividing from the <= sqrt(num).)
Then count all the permutations.
If your number is a power of exactly one prime, p^n, you obvioulsy have n divisors for it, excluding 1; 8 = 2^3 has 3 divisors: 8, 4, 2.
In general case, your number factors into k primes: p0^n0 * p1^n1 * ... * pk^nk. It has (n0 + 1) * (n1 + 1) * .. * (nk + 1) divisors. The "+1" term counts for the case when all other powers are 0, that is, contribute a 1 to the multiplication.
Alternatively, you could just google and RTFM.
Here is an improved version of my code in the question. The running time is better, 0.008s for 10^9 now.
def divisors(num):
ceiling = int(sqrt(num))
divs = []
if num == 1:
return [1]
for i in range(1, ceiling + 1):
if num % i == 0:
divs.append(num / i)
if i != num // i:
divs.append(i)
return divs
It is important for me to also keep the divisors, so if this can still
be improved I'd be glad.
Consider this:
import math
def num_of_divisors(n):
ct = 1
rest = n
for i in range(2, int(math.ceil(math.sqrt(n)))+1):
while rest%i==0:
ct += 1
rest /= i
print i # the factors
if rest == 1:
break
if rest != 1:
print rest # the last factor
ct += 1
return ct
def main():
number = 2**31 * 3**13
print '{} divisors in {}'.format(num_of_divisors(number), number)
if __name__ == '__main__':
main()
We can stop searching for factors at the square root of n. Multiple factors are found in the while loop. And when a factor is found we divide it out from the number.
edit:
#Mark Ransom is right, the factor count was 1 too small for numbers where one factor was greater than the square root of the number, for instance 3*47*149*6991. The last check for rest != 1 accounts for that.
The number of factors is indeed correct then - you don't have to check beyond sqrt(n) for this.
Both statements where a number is printed can be used to append this number to the number of factors, if desired.
I was working on project euler question 23 with python. For this question, I have to find sum of any numbers <28124 that cannot be made by sum of two abundant numbers. abundant numbers are numbers that are smaller then its own sum of proper divisors.
my apporach was : https://gist.github.com/anonymous/373f23098aeb5fea3b12fdc45142e8f7
from math import sqrt
def dSum(n): #find sum of proper divisors
lst = set([])
if n%2 == 0:
step = 1
else:
step = 2
for i in range(1, int(sqrt(n))+1, step):
if n % i == 0:
lst.add(i)
lst.add(int(n/i))
llst = list(lst)
lst.remove(n)
sum = 0
for j in lst:
sum += j
return sum
#any numbers greater than 28123 can be written as the sum of two abundant numbers.
#thus, only have to find abundant numbers up to 28124 / 2 = 14062
abnum = [] #list of abundant numbers
sum = 0
can = set([])
for i in range(1,14062):
if i < dSum(i):
abnum.append(i)
for i in abnum:
for j in abnum:
can.add(i + j)
print (abnum)
print (can)
cannot = set(range(1,28124))
cannot = cannot - can
cannot = list(cannot)
cannot.sort ()
result = 0
print (cannot)
for i in cannot:
result += i
print (result)
which gave me answer of 31531501, which is wrong.
I googled the answer and answer should be 4179871.
theres like 1 million difference between the answers, so it should mean that I'm removing numbers that cannot be written as sum of two abundant numbers. But when I re-read the code it looks fine logically...
Please save from this despair
Just for some experience you really should look at comprehensions and leveraging the builtins (vs. hiding them):
You loops outside of dSum() (which can also be simplified) could look like:
import itertools as it
abnum = [i for i in range(1,28124) if i < dSum(i)]
can = {i+j for i, j in it.product(abnum, repeat=2)}
cannot = set(range(1,28124)) - can
print(sum(cannot)) # 4179871
There are a few ways to improve your code.
Firstly, here's a more compact version of dSum that's fairly close to your code. Operators are generally faster than function calls, so I use ** .5 instead of calling math.sqrt. I use a conditional expression instead of an if...else block to compute the step size. I use the built-in sum function instead of a for loop to add up the divisors; also, I use integer subtraction to remove n from the total because that's more efficient than calling the set.remove method.
def dSum(n):
lst = set()
for i in range(1, int(n ** .5) + 1, 2 if n % 2 else 1):
if n % i == 0:
lst.add(i)
lst.add(n // i)
return sum(lst) - n
However, we don't really need to use a set here. We can just add the divisor pairs as we find them, if we're careful not to add any divisor twice.
def dSum(n):
total = 0
for i in range(1, int(n ** .5) + 1, 2 if n % 2 else 1):
if n % i == 0:
j = n // i
if i < j:
total += i + j
else:
if i == j:
total += i
break
return total - n
This is slightly faster, and uses less RAM, at the expense of added code complexity. However, there's a more efficient approach to this problem.
Instead of finding the divisors (and hence the divisor sum) of each number individually, it's better to use a sieving approach that finds the divisors of all the numbers in the required range. Here's a simple example.
num = 28124
# Build a table of divisor sums.
table = [1] * num
for i in range(2, num):
for j in range(2 * i, num, i):
table[j] += i
# Collect abundant numbers
abnum = [i for i in range(2, num) if i < table[i]]
print(len(abnum), abnum[0], abnum[-1])
output
6965 12 28122
If we need to find divisor sums for a very large num a good approach is to find the prime power factors of each number, since there's an efficient way to compute the sum of the divisors from the prime power factorization. However, for numbers this small the minor time saving doesn't warrant the extra code complexity. (But I can add some prime power sieve code if you're curious; for finding divisor sums for all numbers < 28124, the prime power sieve technique is about twice as fast as the above code).
AChampion's answer shows a very compact way to find the sum of the numbers that cannot be written as the sum of two abundant numbers. However, it's a bit slow, mostly because it loops over all pairs of abundant numbers in abnum. Here's a faster way.
def main():
num = 28124
# Build a table of divisor sums. table[0] should be 0, but we ignore it.
table = [1] * num
for i in range(2, num):
for j in range(2 * i, num, i):
table[j] += i
# Collect abundant numbers
abnum = [i for i in range(2, num) if i < table[i]]
del table
# Create a set for fast searching
abset = set(abnum)
print(len(abset), abnum[0], abnum[-1])
total = 0
for i in range(1, num):
# Search for pairs of abundant numbers j <= d: j + d == i
for j in abnum:
d = i - j
if d < j:
# No pairs were found
total += i
break
if d in abset:
break
print(total)
if __name__ == "__main__":
main()
output
6965 12 28122
4179871
This code runs in around 2.7 seconds on my old 32bit single core 2GHz machine running Python 3.6.0. On Python 2, it's about 10% faster; I think that's because list comprehensions have less overhead in Python 2 (the run in the current scope rather than creating a new scope).
The divisor function is the sum of divisors of a natural number.
Making a little research I found this to be a very good method if you want to find the divisor function of a given natural number N, so I tried to code it in Python:
def divisor_function(n):
"Returns the sum of divisors of n"
checked = [False]*100000
factors = prime_factors(n)
sum_of_divisors = 1 # It's = 1 because it will be the result of a product
for x in factors:
if checked[x]:
continue
else:
count = factors.count(x)
tmp = (x**(count+1)-1)//(x-1)
sum_of_divisors*=tmp
checked[x]=True
return sum_of_divisors
It works pretty well,but I am sure that it can be improved(e.g. : I create a list with 100000 elements,but I am not using most of them).
How would you improve/implement it?
P.S. This is prime_factors:
def prime_factors(n):
"Returns all the prime factors of a positive integer"
factors = []
d = 2
while (n > 1):
while (n%d==0):
factors.append(d)
n /= d
d = d + 1
if (d*d>n):
if (n>1): factors.append(int(n));
break;
return factors
When computing the sum of divisors, you need the factorization of n in the form p1k1 p2k2 ... — that is, you need the exponent of each prime in the factorization. At the moment you are doing this by computing a flat list of prime factors, and then calling count to work out the exponent. This is a waste of time because you can easily generate the prime factorization in the format you need in the first place, like this:
def factorization(n):
"""
Generate the prime factorization of n in the form of pairs (p, k)
where the prime p appears k times in the factorization.
>>> list(factorization(1))
[]
>>> list(factorization(24))
[(2, 3), (3, 1)]
>>> list(factorization(1001))
[(7, 1), (11, 1), (13, 1)]
"""
p = 1
while p * p < n:
p += 1
k = 0
while n % p == 0:
k += 1
n //= p
if k:
yield p, k
if n != 1:
yield n, 1
Notes on the code above:
I've transformed this code so that it generates the factorization, instead of constructing a list (by repeated calls to append) and returning it. In Python, this transformation is nearly always an improvement, because it allows you to consume elements one by one as they are generated, without having to store the whole sequence in memory.
This is the kind of function for which doctests work well.
Now computing the sum of divisors is really simple: there's no need to store the set of checked factors or to count the number of times each factor occurs. In fact you can do it in just one line:
from operator import mul
def sum_of_divisors(n):
"""
Return the sum of divisors of n.
>>> sum_of_divisors(1)
1
>>> sum_of_divisors(33550336) // 2
33550336
"""
return reduce(mul, ((p**(k+1)-1) // (p-1) for p, k in factorization(n)), 1)
You need to change two lines only:
def divisor_function(n):
"Returns the sum of divisors of n"
checked = {}
factors = prime_factors(n)
sum_of_divisors = 1 # It's = 1 because it will be the result of a product
for x in factors:
if checked.get(x,False):
continue
else:
count = factors.count(x)
tmp = (x**(count+1)-1)//(x-1)
sum_of_divisors*=tmp
checked[x]=True
return sum_of_divisors
why use dict or set - or count() - at all, when prime_factors() is guaranteed to return the factors in ascending order? You only ever deal with a previous factor. Counting can just be a part of iteration:
def divisor_function(n):
"Returns the sum of divisors of n"
factors = prime_factors(n)
sum_of_divisors = 1
count = 0; prev = 0;
for x in factors:
if x==prev:
count += 1
else:
if prev: sum_of_divisors *= (prev**(count+1)-1)//(prev-1)
count = 1; prev = x;
if prev: sum_of_divisors *= (prev**(count+1)-1)//(prev-1)
return sum_of_divisors
def sum_divisors(n):
assert n > 0
if n == 1:
return 0
sum = 1
if n % 2 == 0: # if n is even there is a need to go over even numbers
i = 2
while i < sqrt (n):
if n % i == 0:
sum = sum + i + (n//i) # if i|n then n/i is an integer so we want to add it as well
i = i + 1
if type (sqrt (n)) == int: # if sqrt(n)|n we would like to avoid adding it twice
sum = sum + sqrt (n)
else:
i = 3
while i < sqrt (n): # this loop will only go over odd numbers since 2 is not a factor
if n % i == 0:
sum = sum + i + (n//i) # if i|n then n/i is an integer so we want to add it as well
i = i + 2
if type (sqrt (n)) == int: # if sqrt(n)|n we would like to avoid adding it twice
sum = sum + sqrt (n)
return sum
Here is what I do in my Java number utilities (extensively used for Project Euler):
Generate the prime factorization with explicit exponents (see the answer by Gareth Rees).
Unfold the prime factorization in the various functions based on it. I.e., use the same algorithm as for prime factorization but directly compute the desire value instead of storing the factors and exponents.
By default test only divisors two and odd numbers. I have methods firstDivisor(n) and nextDivisor(n,d) for that.
Optionally precompute a table of least divisors for all numbers below a bound. This is very useful if you need to factorize all or most numbers below the bound (improves speed by about sqrt(limit)). I hook the table into the firstDivisor(n) and nextDivisor(n,d) methods, so this doesn't change the factorization algorithms.
I'm attempting to implement the Sieve of Eratosthenes. The output seems to be correct (minus "2" that needs to be added) but if the input to the function is larger than 100k or so it seems to take an inordinate amount of time. What are ways that I can optimize this function?
def sieveErato(n):
numberList = range(3,n,2)
for item in range(int(math.sqrt(len(numberList)))):
divisor = numberList[item]
for thing in numberList:
if(thing % divisor == 0) and thing != divisor:
numberList.remove(thing)
return numberList
Your algorithm is not the Sieve of Eratosthenes. You perform trial division (the modulus operator) instead of crossing-off multiples, as Eratosthenes did over two thousand years ago. Here is an explanation of the true sieving algorithm, and shown below is my simple, straight forward implementation, which returns a list of primes not exceeding n:
def sieve(n):
m = (n-1) // 2
b = [True]*m
i,p,ps = 0,3,[2]
while p*p < n:
if b[i]:
ps.append(p)
j = 2*i*i + 6*i + 3
while j < m:
b[j] = False
j = j + 2*i + 3
i+=1; p+=2
while i < m:
if b[i]:
ps.append(p)
i+=1; p+=2
return ps
We sieve only on the odd numbers, stopping at the square root of n. The odd-looking calculations on j map between the integers being sieved 3, 5, 7, 9, ... and indexes 0, 1, 2, 3, ... in the b array of bits.
You can see this function in action at http://ideone.com/YTaMB, where it computes the primes to a million in less than a second.
You can try the same way Eratosthenes did. Take an array with all numbers you need to check order ascending, go to number 2 and mark it. Now scratch every second number till the end of the array. Then go to 3 and mark it. After that scratch every third number . Then go to 4 - it is already scratched, so skip it. Repeat this for every n+1 which is not already scratched.
In the end, the marked numbers are the prime one. This algorithm is faster, but sometimes need lots of memory. You can optimize it a little by drop all even numbers (cause they are not prime) and add 2 manually to the list. This will twist the logic a little, but will take half the memory.
Here is an illustration of what I'm talking about: http://en.wikipedia.org/wiki/Sieve_of_Eratosthenes
Warning: removing elements from an iterator while iterating on it can be dengerous...
You could make the
if(thing % divisor == 0) and thing != divisor:
test lighter by splitting it in the loop that breaks when you arrive to the index of 'divisor' and then the test:
for thing in numberList_fromDivisorOn:
if(thing % divisor == 0):
numberList.remove(thing)
This code takes 2 seconds to generate primes less than 10M
(it is not mine, i found it somewer on google)
def erat_sieve(bound):
if bound < 2:
return []
max_ndx = (bound - 1) // 2
sieve = [True] * (max_ndx + 1)
#loop up to square root
for ndx in range(int(bound ** 0.5) // 2):
# check for prime
if sieve[ndx]:
# unmark all odd multiples of the prime
num = ndx * 2 + 3
sieve[ndx+num:max_ndx:num] = [False] * ((max_ndx-ndx-num-1)//num + 1)
# translate into numbers
return [2] + [ndx * 2 + 3 for ndx in range(max_ndx) if sieve[ndx]]
I followed this link: Sieve of Eratosthenes - Finding Primes Python as suggested by #MAK and I've found that the accepted answer could be improved with an idea I've found in your code:
def primes_sieve2(limit):
a = [True] * limit # Initialize the primality list
a[0] = a[1] = False
sqrt = int(math.sqrt(limit))+1
for i in xrange(sqrt):
isprime = a[i]
if isprime:
yield i
for n in xrange(i*i, limit, i): # Mark factors non-prime
a[n] = False
for (i, isprime) in enumerate(a[sqrt:]):
if isprime:
yield i+sqrt
if given unlimited memory and time, the following code will print all the prime numbers. and it'll do it without using trial division. it is based on the haskell code in the paper: The Genuine Sieve of Eratosthenes by Melissa E. O'Neill
from heapq import heappush, heappop, heapreplace
def sieve():
w = [2,4,2,4,6,2,6,4,2,4,6,6,2,6,4,2,6,4,6,8,4,2,4,2,4,8,6,4,6,2,4,6,2,6,6,4,2,4,6,2,6,4,2,4,2,10,2,10]
for p in [2,3,5,7]: print p
n,o = 11,0
t = []
l = len(w)
p = n
heappush(t, (p*p, n,o,p))
print p
while True:
n,o = n+w[o],(o+1)%l
p = n
if not t[0][0] <= p:
heappush(t, (p*p, n,o,p))
print p
continue
while t[0][0] <= p:
_, b,c,d = t[0]
b,c = b+w[c],(c+1)%l
heapreplace(t, (b*d, b,c,d))
sieve()