I've been struggling with this problem for a couple days now, I'm still new to Python and more math intensive coding so any help would be appreciated, just point me in the right direction :)
So the question was something like:
You have a movie pass which is valid for N days. You can use it in any way you want except for 3 consecutive days or more.
So basically, you can either use your pass on a given day or choose to not, meaning 2 raised to N total possibilities. The valid ways for claiming the pass are then 2 raised to N - invalidCases
You have to find the number of valid cases % (10^9+7)
I found a recurrence relation for the invalid cases which looked like
invalidCases(at_N) = 2^(n-4) + 2*invalidCases(at_N-1) - invalidCases(at_n-4)
So my first impulse was to simply use recursion:
def invalidCases(n):
if(n<3):
return 0;
elif(n==3):
return 1;
else:
return 2**(n-4)+ 2*invalidCases(n-1)- invalidCases(n-4)
Very inefficient, but my equation seemed correct.
My next attempt, I tried memoization, but I kept running into an error at N=1006.
So I changed the recursion limit.
My present attempt(with memoization and increased recursion limit)
import sys
sys.setrecursionlimit(10**6)
T=int(input());
#2**(n-4) + 2*ans(n-1)-ans(n-4)
memo={0:0,1:0,2:0,3:1,4:3,5:8,6:20,7:47} #
def ans(n):
#complete this function
if n not in memo:
memo[n]=(2**(n-4) + 2*ans(n-1)-ans(n-4));
return memo[n];
modulo = 10**9 + 7;
print((2**n-ans(n))%modulo);
Finally, my problem.
I need this code to work for n = 999999.
How do I bring its worst case down to minimum?
Any pointers or tips would be great.
Here's a complete solution which is based on the observation that a valid solution for three or more days must begin with one of the following:
0
10
110
where 1 indicates the pass is used on that day, and 0 indicates it is not.
There are valid(n-1) possibilities for the first form, valid(n-2) possibilities for the second form, and valid(n-3) possibilities for the third form.
The recurrence is then:
valid(n) = valid(n-1) + valid(n-2) + valid(n-3)
The basis cases are valid(0) = 1, valid(1) = 2, and valid(2) = 4. It is important to note that valid(0) is 1, not zero. That's because there is exactly one solution when n=0, namely, the empty sequence. This is not only mathematically correct, but is also needed for the recurrence to work correctly.
The code does three things to make it run fast:
It uses a cache to cache results (memoization), as you had done.
It doesn't store the full results, but instead first applies the modulus, greatly reducing the value ranges.
It preloads the cache, starting with 0 and going up to the desired value. This reduces the maximum recursion depth to one.
Here's the code:
cache = {}
modulus = 10**9 + 7
def valid(n):
if n in cache:
return cache[n]
if n == 0:
v = 1
elif n == 1:
v = 2
elif n == 2:
v = 4
else:
v = valid(n-1) + valid(n-2) + valid(n-3)
v %= modulus
cache[n] = v
return v
def main():
# Preload the cache
for n in range(1000000):
valid(n)
print(valid(999999))
main()
Here's the output:
746580045
It runs in under 2 seconds on my system.
Update: Here's a minimal iterative solution, inspired by the approach used by MFisherKDX. The seed values were constructed in a way that eliminates the need for special-casing (the intial v2 is valid(0)):
modulus = 10**9 + 7
def valid(n):
v0, v1, v2 = 0, 1, 1
for i in range(n):
v0, v1, v2 = v1, v2, (v0 + v1 + v2) % modulus
return v2
print(valid(999999))
This solution is probably as fast as you can get. It discards the intermediate results after using them, which is fine if you're only calling the function once.
Here's my answer. Bottom up solution. Compare with Tom's answer which is top down and equally valid. At each day j it keeps track of the number of possibilities that use the pass on day j as well as the number of possibilities that use the pass on both j and j-1.
def ans(n):
total = 1
tcd = 0 #total used at current day
tcpd = 0 #total used at current and previous day
m = 1000000007
for j in range(0, n):
next_tot = 2*total - tcpd
next_tcd = total - tcpd
next_tcpd = tcd - tcpd
total = next_tot % m
tcd = next_tcd % m
tcpd = next_tcpd % m
return total
print(ans(999999))
Result is 746580045 and takes 400ms on my system.
Related
Write a function named powPosInt. This function will take two integer input parameters, named x and p. This function will compute the value of x to the power of p, where p >= 0, which is defined as x to the power of p = x × · · · × x
| {z }
p times
or 1 if p is 0.
The function will return this computed result. This function should not produce any console output.
For computing x to the power of p
, I am requiring that you implement the calculations using a while loop and an accumulator variable.
Examples of values you should test with: x = 2, p = 3 should return 8. x = 1, p = 4
should return 1. x = 7, p = 0 should return 1. x = −3, p = 2 should return 9.
x = −3, p = 3 should return −27.
This is what I have so far, I am having trouble figuring out how to put a while loop within this function. I am asking on here for help as my TEACHER will not assist, (he wants us to treat his class like the real world where we will have to figure out solutions on our own... why pay his salary right?)
def powPosInt(x,p):
number = p
count = 0
while (number != 0):
answer = (x**p)
count = count + 1
if (p<0):
answer = (x**(1/abs(p)))
return answer
Using accumulation means you need to create a variable that will store the result, and while loop will multiply it by x each step. You shouldn't use the ** operator.
steps = 0
result = 1
while (steps < p):
result *= x
steps += 1
def powPosInt(x,p):
number = p
count = 0
while (number != 0):
answer = (x**p)
count = count + 1
if (p<0):
answer = (x**(1/abs(p)))
return answer
I see a number of problems here:
A loop for calculating powers will be using repeated multiplication, not exponentiation. The intent here is to assume Python doesn't have an exponentiation operator, otherwise this assignment would be futile.
There's no reason to handle negative powers, the specifications clarly call for p >= 0. Hence your final if block is unnecessary.
This is a style issue only: you should get into the habit of using meaningful variable names. The only situation in which I use single-letter variables nowadays is a short-lived, non-nested, loop where the variable name is irrelevant (and I use i for that). For any other situation, I use explanatory names (yes, not even j or k for nested loops). And I'm even starting to move away from using i for these simple cases.
Having said all that, I'd suggest revisiting your solution to take those into account.
Once you've done so, you can refer to the code below for one solution:
def powPosInt(myBase, myPower):
result = 1
for _ in range(myPower):
result *= myBase
return result
And, if you wanted to go for extra credits(a), you could put in place contractual checks to ensure callers are doing the right thing:
def powPosInt(myBase, myPower):
if type(myBase) != int: raise TypeError("Base must be an integer")
if type(myPower) != int: raise TypeError("Power must be an integer")
if myPower < 1: raise ValueError("Power must be >= 1")
result = 1
for _ in range(myPower):
result *= myBase
return result
(a) Or risk losing marks for being a smart-alec. This depends very much on the personality of your teacher :-)
There is a problem which is "Last Digit of the Sum of Fibonacci Numbers". I have already optimised the naive approach but this code is not working for higher values. Example: 613455
Everytime I run this program, the os crashes (memory limit exceeds).
My code is:
def SumFib(n):
result = []
for i in range(0, n+1):
if i <= 1:
result.append(i)
else:
result.append(result[i-1] + result[i-2])
return (sum(result) % 10)
print(SumFib(int(input())))
Need some help to get over this problem.
In order to consume less memory, store less values. For example:
def sum_fib(n):
if n == 0:
return 0
elif n == 1:
return 1
elif n == 2:
return 2
res_sum = 2
a, b = 1, 1
for _ in range(n-2):
a, b = b, a+b
res_sum += b
return res_sum % 10
The memory requirements (at a glance) are constant. Your original snippet has lineal memory requirements (when n grows, the memory requirements grow).
(More intelligent things can be done regarding the modulo operation if n is really huge, but that is off topic to the original question)
Edit:
I was curious and ended up doing some homework.
Lemma 1: The last digit of the fibonacci follows a 60-length cycle.
Demonstration: One can check that, starting by (0,1), element 60 has last digit 0 and element 61 has last digit 1 (this can be checked empirically). From this follows that fib(n) equals fib(n % 60).
Lemma 2: The last digit of the sum of the first 60 digits is 0.
Demonstration: Can be checked empirically too.
Conclusion: sum_fib(n) == sum_fib(n % 60). So the algorithm to solve "last digit of the sum of the n-th first fibonacci numbers can be solved in O(1) by taking n % 60 and evaluating sum_fib of that number, i.e., sum_fib(n % 60). Or one can precreate the digit list of 60 elements and do a simple-lookup.
Instead of using range you can try a custom function that works as generator.
Generators calculate values on the fly instead of loading all at once like iterators.
Replace the below function with range
def generate_value(start, end, step):
count =start
while True:
if count <=end:
yield count
count+=step
break
I have code like below
def yield_multiple():
for prime in prime_list:
for multiple in range(prime+prime, end, prime):
yield multiple
And I use this to get the prime numbers
multiple_set = set(yield_multiple())
result = [v for v in candidate_list if v not in multiple_set]
And I meet the memory error when the set is very large, so I was thinking to use this to save the memory
result = [v for v in candidate_list if v not in yield_multiple()]
But this will get the wrong result. So, How to avoid memory error to get the prime numbers correctly?
Here's my improved solution without too much memory to use.
import math
import sys
import time
from mpi4py import MPI
import eratosthenes
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()
TAG_RESULT = 0
n = sys.argv[1]
if n.isdigit():
start_time = time.time()
n = int(n)
sqrt_n = int(math.sqrt(n))
task_per_block = int(math.ceil((n - 1) / size))
begin = 2 + rank * task_per_block
end = begin + task_per_block if begin + task_per_block <= n + 1 else n + 1
if rank == 0:
begin = sqrt_n if sqrt_n < end else begin
sieve_list = [True] * (end - begin)
prime_list = eratosthenes.sieve(sqrt_n)
if rank == 0:
result = sum(prime_list)
for prime in prime_list:
start = begin if begin % prime == 0 else (int(begin / prime) + 1) * prime
for multiple in range(start, end, prime):
sieve_list[multiple - begin] = False
result += sum(i + begin for i, v in enumerate(sieve_list) if v)
result_received = 0
while result_received < size - 1:
data = comm.recv(source=MPI.ANY_SOURCE, tag=TAG_RESULT)
result += data
result_received += 1
print(result)
print(time.time() - start_time)
else:
for prime in prime_list:
start = begin if begin % prime == 0 else (int(begin / prime) + 1) * prime
for multiple in range(start, end, prime):
sieve_list[multiple - begin] = False
result = sum(i + begin for i, v in enumerate(sieve_list) if v)
comm.send(result, dest=0, tag=TAG_RESULT)
By switching to working by segments between squares of consecutive primes, creating these sets for each segment one after another.
For each segment you'll have to calculate the starting point of the enumeration of a prime's multiples, for each known prime which is not greater than the segment's top value (i.e. the next "core" prime's square).
The "core" primes, to get the squares of, you can get separately, independently, by a recursive application of the same algorithm.
An example of this approach (the separate primes supply that is) is How to implement an efficient infinite generator of prime numbers in Python?
To make it parallel, you'll need to find means to use the set in a shared fashion between all the enumerations, which each will set each of its enumerated multiples off in the same shared set. Order of operations is not important, as long as they are all finished. The access need not be guarded, as setting the same location off twice (or more) is perfectly fine.
This will also be very efficient.
If you want to stay with this approach - which does have a certain simplicity, although it must be terribly inefficient - the simplest way I can see to do it without constructing a large set or re-running yield_multiple for each candidate is to sort of reverse your membership check:
multiples = {c for c in yield_multiple() if c in candidate_list}
result = [c for c in candidate_list if c not in multiples]
However, unless using your own code is the most important factor here, I'd recommend finding a more efficient approach, like for example the one described in this other answer.
I've been using the random_element() function provided by SAGE to generate random integer partitions for a given integer (N) that are a particular length (S). I'm trying to generate unbiased random samples from the set of all partitions for given values of N and S. SAGE's function quickly returns random partitions for N (i.e. Partitions(N).random_element()).
However, it slows immensely when adding S (i.e. Partitions(N,length=S).random_element()). Likewise, filtering out random partitions of N that are of length S is incredibly slow.
However, and I hope this helps someone, I've found that in the case when the function returns a partition of N not matching the length S, that the conjugate partition is often of length S. That is:
S = 10
N = 100
part = list(Partitions(N).random_element())
if len(part) != S:
SAD = list(Partition(part).conjugate())
if len(SAD) != S:
continue
This increases the rate at which partitions of length S are found and appears to produce unbiased samples (I've examined the results against entire sets of partitions for various values of N and S).
However, I'm using values of N (e.g. 10,000) and S (e.g. 300) that make even this approach impractically slow. The comment associated with SAGE's random_element() function admits there is plenty of room for optimization. So, is there a way to more quickly generate unbiased (i.e. random uniform) samples of integer partitions matching given values of N and S, perhaps, by not generating partitions that do not match S? Additionally, using conjugate partitions works well in many cases to produce unbiased samples, but I can't say that I precisely understand why.
Finally, I have a definitively unbiased method that has a zero rejection rate. Of course, I've tested it to make sure the results are representative samples of entire feasible sets. It's very fast and totally unbiased. Enjoy.
from sage.all import *
import random
First, a function to find the smallest maximum addend for a partition of n with s parts
def min_max(n,s):
_min = int(floor(float(n)/float(s)))
if int(n%s) > 0:
_min +=1
return _min
Next, A function that uses a cache and memoiziation to find the number of partitions
of n with s parts having x as the largest part. This is fast, but I think there's
a more elegant solution to be had. e.g., Often: P(N,S,max=K) = P(N-K,S-1)
Thanks to ante (https://stackoverflow.com/users/494076/ante) for helping me with this:
Finding the number of integer partitions given a total, a number of parts, and a maximum summand
D = {}
def P(n,s,x):
if n > s*x or x <= 0: return 0
if n == s*x: return 1
if (n,s,x) not in D:
D[(n,s,x)] = sum(P(n-i*x, s-i, x-1) for i in xrange(s))
return D[(n,s,x)]
Finally, a function to find uniform random partitions of n with s parts, with no rejection rate! Each randomly chosen number codes for a specific partition of n having s parts.
def random_partition(n,s):
S = s
partition = []
_min = min_max(n,S)
_max = n-S+1
total = number_of_partitions(n,S)
which = random.randrange(1,total+1) # random number
while n:
for k in range(_min,_max+1):
count = P(n,S,k)
if count >= which:
count = P(n,S,k-1)
break
partition.append(k)
n -= k
if n == 0: break
S -= 1
which -= count
_min = min_max(n,S)
_max = k
return partition
I ran into a similar problem when I was trying to calculate the probability of the strong birthday problem.
First off, the partition function explodes when given only modest amount of numbers. You'll be returning a LOT of information. No matter which method you're using N = 10000 and S = 300 will generate ridiculous amounts of data. It will be slow. Chances are any pure python implementation you use will be equally slow or slower. Look to making a CModule.
If you want to try python the approach I took as a combination of itertools and generators to keep memory usage down. I don't seem to have my code handy anymore, but here's a good impementation:
http://wordaligned.org/articles/partitioning-with-python
EDIT:
Found my code:
def partition(a, b=-1, limit=365):
if (b == -1):
b = a
if (a == 2 or a == 3):
if (b >= a and limit):
yield [a]
else:
return
elif (a > 3):
if (a <= b):
yield [a]
c = 0
if b > a-2:
c = a-2
else:
c = b
for i in xrange(c, 1, -1):
if (limit):
for j in partition(a-i, i, limit-1):
yield [i] + j
Simple approach: randomly assign the integers:
def random_partition(n, s):
partition = [0] * s
for x in range(n):
partition[random.randrange(s)] += 1
return partition
recently I became interested in the subset-sum problem which is finding a zero-sum subset in a superset. I found some solutions on SO, in addition, I came across a particular solution which uses the dynamic programming approach. I translated his solution in python based on his qualitative descriptions. I'm trying to optimize this for larger lists which eats up a lot of my memory. Can someone recommend optimizations or other techniques to solve this particular problem? Here's my attempt in python:
import random
from time import time
from itertools import product
time0 = time()
# create a zero matrix of size a (row), b(col)
def create_zero_matrix(a,b):
return [[0]*b for x in xrange(a)]
# generate a list of size num with random integers with an upper and lower bound
def random_ints(num, lower=-1000, upper=1000):
return [random.randrange(lower,upper+1) for i in range(num)]
# split a list up into N and P where N be the sum of the negative values and P the sum of the positive values.
# 0 does not count because of additive identity
def split_sum(A):
N_list = []
P_list = []
for x in A:
if x < 0:
N_list.append(x)
elif x > 0:
P_list.append(x)
return [sum(N_list), sum(P_list)]
# since the column indexes are in the range from 0 to P - N
# we would like to retrieve them based on the index in the range N to P
# n := row, m := col
def get_element(table, n, m, N):
if n < 0:
return 0
try:
return table[n][m - N]
except:
return 0
# same definition as above
def set_element(table, n, m, N, value):
table[n][m - N] = value
# input array
#A = [1, -3, 2, 4]
A = random_ints(200)
[N, P] = split_sum(A)
# create a zero matrix of size m (row) by n (col)
#
# m := the number of elements in A
# n := P - N + 1 (by definition N <= s <= P)
#
# each element in the matrix will be a value of either 0 (false) or 1 (true)
m = len(A)
n = P - N + 1;
table = create_zero_matrix(m, n)
# set first element in index (0, A[0]) to be true
# Definition: Q(1,s) := (x1 == s). Note that index starts at 0 instead of 1.
set_element(table, 0, A[0], N, 1)
# iterate through each table element
#for i in xrange(1, m): #row
# for s in xrange(N, P + 1): #col
for i, s in product(xrange(1, m), xrange(N, P + 1)):
if get_element(table, i - 1, s, N) or A[i] == s or get_element(table, i - 1, s - A[i], N):
#set_element(table, i, s, N, 1)
table[i][s - N] = 1
# find zero-sum subset solution
s = 0
solution = []
for i in reversed(xrange(0, m)):
if get_element(table, i - 1, s, N) == 0 and get_element(table, i, s, N) == 1:
s = s - A[i]
solution.append(A[i])
print "Solution: ",solution
time1 = time()
print "Time execution: ", time1 - time0
I'm not quite sure if your solution is exact or a PTA (poly-time approximation).
But, as someone pointed out, this problem is indeed NP-Complete.
Meaning, every known (exact) algorithm has an exponential time behavior on the size of the input.
Meaning, if you can process 1 operation in .01 nanosecond then, for a list of 59 elements it'll take:
2^59 ops --> 2^59 seconds --> 2^26 years --> 1 year
-------------- ---------------
10.000.000.000 3600 x 24 x 365
You can find heuristics, which give you just a CHANCE of finding an exact solution in polynomial time.
On the other side, if you restrict the problem (to another) using bounds for the values of the numbers in the set, then the problem complexity reduces to polynomial time. But even then the memory space consumed will be a polynomial of VERY High Order.
The memory consumed will be much larger than the few gigabytes you have in memory.
And even much larger than the few tera-bytes on your hard drive.
( That's for small values of the bound for the value of the elements in the set )
May be this is the case of your Dynamic programing algorithm.
It seemed to me that you were using a bound of 1000 when building your initialization matrix.
You can try a smaller bound. That is... if your input is consistently consist of small values.
Good Luck!
Someone on Hacker News came up with the following solution to the problem, which I quite liked. It just happens to be in python :):
def subset_summing_to_zero (activities):
subsets = {0: []}
for (activity, cost) in activities.iteritems():
old_subsets = subsets
subsets = {}
for (prev_sum, subset) in old_subsets.iteritems():
subsets[prev_sum] = subset
new_sum = prev_sum + cost
new_subset = subset + [activity]
if 0 == new_sum:
new_subset.sort()
return new_subset
else:
subsets[new_sum] = new_subset
return []
I spent a few minutes with it and it worked very well.
An interesting article on optimizing python code is available here. Basically the main result is that you should inline your frequent loops, so in your case this would mean instead of calling get_element twice per loop, put the actual code of that function inside the loop in order to avoid the function call overhead.
Hope that helps! Cheers
, 1st eye catch
def split_sum(A):
N_list = 0
P_list = 0
for x in A:
if x < 0:
N_list+=x
elif x > 0:
P_list+=x
return [N_list, P_list]
Some advices:
Try to use 1D list and use bitarray to reduce memory footprint at minimum (http://pypi.python.org/pypi/bitarray) so you will just change get / set functon. This should reduce your memory footprint by at lest 64 (integer in list is pointer to integer whit type so it can be factor 3*32)
Avoid using try - catch, but figure out proper ranges at beginning, you might found out that you will gain huge speed.
The following code works for Python 3.3+ , I have used the itertools module in Python that has some great methods to use.
from itertools import chain, combinations
def powerset(iterable):
s = list(iterable)
return chain.from_iterable(combinations(s, r) for r in range(len(s)+1))
nums = input("Enter the Elements").strip().split()
inputSum = int(input("Enter the Sum You want"))
for i, combo in enumerate(powerset(nums), 1):
sum = 0
for num in combo:
sum += int(num)
if sum == inputSum:
print(combo)
The Input Output is as Follows:
Enter the Elements 1 2 3 4
Enter the Sum You want 5
('1', '4')
('2', '3')
Just change the values in your set w and correspondingly make an array x as big as the len of w then pass the last value in the subsetsum function as the sum for which u want subsets and you wl bw done (if u want to check by giving your own values).
def subsetsum(cs,k,r,x,w,d):
x[k]=1
if(cs+w[k]==d):
for i in range(0,k+1):
if x[i]==1:
print (w[i],end=" ")
print()
elif cs+w[k]+w[k+1]<=d :
subsetsum(cs+w[k],k+1,r-w[k],x,w,d)
if((cs +r-w[k]>=d) and (cs+w[k]<=d)) :
x[k]=0
subsetsum(cs,k+1,r-w[k],x,w,d)
#driver for the above code
w=[2,3,4,5,0]
x=[0,0,0,0,0]
subsetsum(0,0,sum(w),x,w,7)