Related
I have some permutations of a list:
>>> import itertools
>>> perms = list(itertools.permutations([0,1,2,3]))
>>> perms
[(0, 1, 2, 3), (0, 1, 3, 2), (0, 2, 1, 3), (0, 2, 3, 1), (0, 3, 1, 2), (0, 3, 2, 1), (1, 0, 2, 3), (1, 0, 3, 2), (1, 2, 0, 3), (1, 2, 3, 0), (1, 3, 0, 2), (1, 3, 2, 0), (2, 0, 1, 3), (2, 0, 3, 1), (2, 1, 0, 3), (2, 1, 3, 0), (2, 3, 0, 1), (2, 3, 1, 0), (3, 0, 1, 2), (3, 0, 2, 1), (3, 1, 0, 2), (3, 1, 2, 0), (3, 2, 0, 1), (3, 2, 1, 0)]
>>> len(perms)
24
What function can I use (without access to the list perm) to get the index of an arbitrary permutation, e.g. (0, 2, 3, 1) -> 3?
(You can assume that permuted elements are always an ascending list of integers, starting at zero.)
Hint: The factorial number system may be involved. https://en.wikipedia.org/wiki/Factorial_number_system
Off the top of my head I came up with the following, didn't test it thoroughly.
from math import factorial
elements = list(range(4))
permutation = (3, 2, 1, 0)
index = 0
nf = factorial(len(elements))
for n in permutation:
nf //= len(elements)
index += elements.index(n) * nf
elements.remove(n)
print(index)
EDIT: replaced nf /= len(elements) with nf //= len(elements)
I suppose this is a challenge, so here is my (recursive) answer:
import math
import itertools
def get_index(l):
# In a real function, there should be more tests to validate that the input is valid, e.g. len(l)>0
# Terminal case
if len(l)==1:
return 0
# Number of possible permutations starting with l[0]
span = math.factorial(len(l)-1)
# Slightly modifying l[1:] to use the function recursively
new_l = [ val if val < l[0] else val-1 for val in l[1:] ]
# Actual solution
return get_index(new_l) + span*l[0]
get_index((0,1,2,3))
# 0
get_index((0,2,3,1))
# 3
get_index((3,2,1,0))
# 23
get_index((4,2,0,1,5,3))
# 529
list(itertools.permutations((0,1,2,3,4,5))).index((4,2,0,1,5,3))
# 529
You need to write your own function. Something like this would work
import math
def perm_loc(P):
N = len(P)
assert set(P) == set(range(N))
def rec(perm):
nums = set(perm)
if not perm:
return 0
else:
sub_res = rec(perm[1:]) # Result for tail of permutation
sub_size = math.factorial(len(nums) - 1) # How many tail permutations exist
sub_index = sorted(nums).index(perm[0]) # Location of first element in permutaiotn
# in the sorted list of number
return sub_index * sub_size + sub_res
return rec(P)
The function that does all the work is rec, with perm_loc just serving as a wrapper around it. Note that this algorithm is based on the nature of the permutation algorithm that itertools.permutation happens to use.
The following code tests the above function. First on your sample, and then on all permutations of range(7):
print perm_loc([0,2,3,1]) # Print the result from the example
import itertools
def test(N):
correct = 0
perms = list(itertools.permutations(range(N)))
for (i, p) in enumerate(perms):
pl = perm_loc(p)
if i == pl:
correct += 1
else:
print ":: Incorrect", p, perms.index(p), perm_loc(N, p)
print ":: Found %d correct results" % correct
test(7) # Test on all permutations of range(7)
from math import factorial
def perm_to_permidx(perm):
# Extract info
n = len(perm)
elements = range(n)
# "Gone"s will be the elements of the given perm
gones = []
# According to each number in perm, we add the repsective offsets
offset = 0
for i, num in enumerate(perm[:-1], start=1):
idx = num - sum(num > gone for gone in gones)
offset += idx * factorial(n - i)
gones.append(num)
return offset
the_perm = (0, 2, 3, 1)
print(perm_to_permidx(the_perm))
# 3
Explanation: All permutations of a given range can be considered as a groups of permutations. So, for example, for the permutations of 0, 1, 2, 3 we first "fix" 0 and permute rest, then fix 1 and permute rest, and so on. Once we fix a number, the rest is again permutations; so we again fix a number at a time from the remaining numbers and permute the rest. This goes on till we are left with one number only. Every level of fixing has a corresponding (n-i)! permutations.
So this code finds the "offsets" for each level of permutation. The offset corresonds to where the given permutation starts when we fix numbers of perm in order. For the given example of (0, 2, 3, 1), we first look at the first number in the given perm which is 0, and figure the offset as 0. Then this goes to gones list (we will see its usage). Then, at the next level of permutation we see 2 as the fixing number. To calculate the offset for this, we need the "order" of this 2 among the remaining three numbers. This is where gones come into play; if an already-fixed and considered number (in this case 0) is less than the current fixer, we subtract 1 to find the new order. Then offset is calculated and accumulated. For the next number 3, the new order is 3 - (1 + 1) = 1 because both previous fixers 0 and 2 are at the "left" of 3.
This goes on till the last number of the given perm since there is no need to look at it; it will have been determined anyway.
I have a list of values and I input a number. The function has to return the indices in the list whose value add up to the number. The problem is the list contains duplicate numbers and the returned indices should not be the same. Here is what I have but the solution does not look clean. Is there a better way?
finalList = []
def getIndices(number):
values = [10,20,20,50,100,200,200,500,1000,2000,2000,5000]
for i in range(len(values)):
if values[i] == number:
if i not in finalList:
finalList.append(i)
else:
finalList.append(i-1)
return values
elif values[i] < number:
continue
else:
number = number - values[i-1]
if i-1 not in finalList:
finalList.append(i-1)
else:
finalList.append(i-2)
if number <= 0:
break
return getIndices(number)
result = getIndices(450)
print(result)
Output
[6, 5, 3]
If I didnt check the list before appending then I would get [6, 6, 3] which is not what I want.
If the list of values is sorted, the following code will do what you want. The trick here is to walk the list in reverse order.
But note the edit and the last test in the function!
Edit: The problem is known as the subset sum problem (see Wikipedia) and is NP-complete. Should have recognized it right away, my bad. This basically means that there is no simple & efficient solution. The simplest solution would be to try all possible combinations, but if your list of values is large, it will simply take too long to complete.
def getIndices(number, values):
'''Return a tuple (n, l) where l is a list of indices in 'values' and the
following condition holds: n + sum(values[i] for i in l) == number.
values -- sorted list of numbers
number -- sum to search for
>>> values=[10,20,20,50,100,200,200,500,1000,2000,2000,5000]
>>> getIndices(18000, values)
(6900, [11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0])
>>> getIndices(450, values)
(0, [6, 5, 3])
>>> getIndices(15, values)
(5, [0])
>>> getIndices(10, values)
(0, [0])
>>> getIndices(5, values)
(5, [])
>>> getIndices(0, values)
(0, [])
This simplicist algorithm does not always find a solution with 'n' == 0,
even if one exists. The following test fails, it returns (10, [2])
instead.
>>> getIndices(40, [20, 20, 30])
(0, [0, 1])
'''
n = number
l = []
for i in range(len(values) - 1, -1, -1):
if values[i] <= n:
l.append(i)
n -= values[i]
assert(n + sum(values[i] for i in l) == number)
return n, l
if __name__ == '__main__':
import doctest
doctest.testmod()
I need to generate permutations of digits, the number can be bigger than the digit count. For my current purpose I need to generate permutations of these digits 0, 1, 2 to get numbers of upto 20 digits length. For example I the first few permutations would be 0, 1, 2, 10, 11, 12, ... 1122, 1211.
There are existing answers using Iterator in Python here or here and those gives the full permutation directly.
But I need to perform some tests over each permutations and if I keep the entire permutations list in memory it becomes too big, especially for 20 digits it comes to 320 permutations.
So my question is can it be done without recursion, so that I can perform the tests over each permutations.
Edit:
I'm looking at permutations with repetitions. So for a number of say 20 digits, each digit can take value from [0, 1, 2]. That's why the number of permutations in that case will come to 320.
What you are looking is called a Cartesian product, not a permutation. Python itertools have a method itertools.product() to produce the desired result:
import itertools
for p in itertools.product(range(3), repeat=4):
print p
Output is 3^4 lines:
(0, 0, 0, 0)
(0, 0, 0, 1)
(0, 0, 0, 2)
(0, 0, 1, 0)
(0, 0, 1, 1)
...
(2, 2, 2, 1)
(2, 2, 2, 2)
To produce output tuples with length form 1 to 4, use an additional iteration:
for l in range(1, 5):
for p in itertools.product(range(3), repeat=l):
print p
Finally, this works for string elements, too:
for i in range(5):
for p in itertools.product(('0', '1', '2'), repeat=i):
print ''.join(p),
print
Output:
0 1 2 00 01 02 10 11 12 20 21 22 000 001 002 010 [...] 2220 2221 2222
Yes, your program could like like this:
import itertools
def perform_test(permutation):
pass
# permutations() does not construct entire list, but yields
# results one by on.
for permutation in itertools.permutations([1, 2, 3, 4, 5], 2):
perform_test(permutation)
While there are ways to do this using itertools etc, here is a way that is a bit different from what you would normally do.
If you were to have a list of these permutations in order, what you would actually have is ternary numbers that represent their place in the list. e.g. list[4] is 11 which is 4 in ternary (3*1+1*1). So you could convert the index value that you want to test into ternary and that would produce the correct value.
While python can do conversion from an integer to its form in that base (e.g. int("11",3) outputs 4) the reverse is not implicitly implemented. There are lots of implementations out there though. Here is a good one (modified for your case):
def digit_to_char(digit):
if digit < 10:
return str(digit)
return chr(ord('a') + digit - 10)
def perm(number):
(d, m) = divmod(number, 3)
if d > 0:
return perm(d) + digit_to_char(m)
return digit_to_char(m)
So if you wanted to find the 20th permutation, you could do perm(20), which would give you 202. So now you can just do a regular loop through the index values that you want. With no storage of big lists in memory.
permutation = 0
i = 0
while len(str(permutation)) < 20:
permutation = perm(i)
do_test(permutation)
i += 1
Lets assume you have a list with y poisitions (0 for sake of this question). If y = 10:
[0,0,0,0,0,0,0,0,0,0]
You want to fill adjacent positions up to a given value x and append it to an empty list. If x = 4:
[[1,1,1,1,0,0,0,0,0,0], [0,1,1,1,1,0,0,0,0,0], [0,0,1,1,1,1,0,0,0,0], ... , [0,0,0,0,0,0,1,1,1,1]]
I made that occur through this function:
def permutations(number=4, limit=10):
perms = []
if type(number) == int:
a = -1
b = a + number
while b < limit:
a+=1
b = a + number
start = [0 for x in range(limit)]
for i in range(a, b):
start[i] = 1
perms.append(start)
This is fine, but if I want to do the same thing, but pass a tuple instead of an integer I'd like the output to be:
if number = (4,3):
[[1,1,1,1,0,1,1,1,0,0], [1,1,1,1,0,0,1,1,1,0], [1,1,1,1,0,0,0,1,1,1],
[0,1,1,1,1,0,1,1,1,0], [0,1,1,1,1,0,0,1,1,1],
[0,0,1,1,1,1,0,1,1,1]]
The 0 between the two groupings of 1's is necessary the first value of the tuple corresponds to the number of 1's in the first grouping, and the second value of the tuple corresponds to the number of 1's in the second grouping. Ideally this function would work with tuples that have more than 2 values.
This idea is a little challenging to get across so please let me know if you need any clarification.
Thank you for your help!
The simplest approach I can think of is to generate all possible combinations of 1 and 0, and filter out all of the ones that don't have the right grouping lengths.
import itertools
def permutations(tup, limit=10):
for candidate in itertools.product([0,1], repeat=limit):
segment_lengths = [len(list(b)) for a,b in itertools.groupby(candidate) if a == 1]
if tup == tuple(segment_lengths):
yield candidate
for seq in permutations((4, 3), 10):
print seq
Result:
(0, 0, 1, 1, 1, 1, 0, 1, 1, 1)
(0, 1, 1, 1, 1, 0, 0, 1, 1, 1)
(0, 1, 1, 1, 1, 0, 1, 1, 1, 0)
(1, 1, 1, 1, 0, 0, 0, 1, 1, 1)
(1, 1, 1, 1, 0, 0, 1, 1, 1, 0)
(1, 1, 1, 1, 0, 1, 1, 1, 0, 0)
Note that this is very slow for large values of limit - it has to evaluate 2^limit candidate sequences. Not bad for limit = 10; only 1024 candidates need to be evaluated. But it quickly grows into the millions and beyond for larger limits.
Edit: Inspired by user2097159's excellent comment, here's an approach with better run time.
import itertools
"""Finds all non-negative integer sequences whose sum equals `total`, and who have `size` elements."""
def possible_sums(total, size):
if total == 0:
yield [0]*size
return
if size == 1:
yield [total]
return
for i in range(total+1):
left = [i]
for right in possible_sums(total-i, size-1):
yield left + right
"""
combines two lists a and b in order like:
[a[0], b[0], a[1], b[1]...]
"""
def interleave(a,b):
result = []
for pair in itertools.izip_longest(a,b):
for item in pair:
if item is not None:
result.append(item)
return result
"""flattens a list of lists into a one dimensional list"""
def flatten(seq):
return [x for item in seq for x in item]
def permutations(tup, limit):
one_segments = [[1]*size for size in tup]
for i in range(len(tup)-1):
one_segments[i].append(0)
remaining_zeroes = limit - sum(tup) - len(tup) + 1
assert remaining_zeroes >= 0, "not enough room to separate ranges!"
for gap_sizes in possible_sums(remaining_zeroes, len(tup)+1):
zero_segments = [[0]*size for size in gap_sizes]
yield flatten(interleave(zero_segments, one_segments))
for seq in permutations((4, 3), 10):
print seq
You can generate all list recursively.
F(tup, limit) =
[1, 1, ...1, 0] combine with all solutions of F(tup[1:], limit - len(tup[1]) - 1)
[0, 1 ,1 , ... 1, 0] combine with all solutions of F(tup[1:], limit - len(tup[1]) - 2)
.
.
.
if tup is empty return a list of zero
if sum(tup) + len(tup) - 1 > limit, return an empty list since there is no solution.
e.g. permutations((4,3,2), 10) shall return []
Otherwise, enumerating how many prefix zero there will be:
Generate prefix list which is [0, 0, 0 .. 0, 1, 1, ... 1, 0] The number of 1s is the value of first item in the tuple. Append additional 0 if it's not the last item of the tuple.
Call the function recursively for the rest element in the tuple to solve the similar sub-problem
Combine the prefix list with each solution of the sub-problem
Here is the code:
def permutations(tup, limit=100):
if len(tup) <= 0:
return [[0] * limit]
minimum_len = sum(tup) + len(tup) - 1
if minimum_len > limit:
return []
perms = []
for prefix_zero in range(0, limit - minimum_len + 1):
prefix = [0] * prefix_zero + [1] * tup[0]
if len(tup) > 1:
prefix += [0]
suffix_list = permutations(tup[1:], limit - len(prefix))
perms += [prefix + suffix for suffix in suffix_list] #combine the solutions
return perms
This solution creates all permutations of blocks of ones (a list defined by each entry in the tuple) with blocks of zeros (lists of length one) for the extra padding.
import itertools as it
spec = (1,2,3)
nBlocks = len(spec)
nZeros = 5
totalSize = sum(spec) + nZeros+1-nBlocks
blocks = [[1,]*s + [0,] for s in spec]
zeros = [[0,],]*(nZeros+1-nBlocks)
a = list(it.permutations(blocks + zeros, nZeros+1))
b = [list(it.chain.from_iterable(l))[:-1] for l in a]
for l in b:
print l
Without using itertools.
My shot at this, should be fairly quick, but uses a recursive generator (python recursion depth limit, here I come...).
# simple test case
seqs = (1, 2, 3)
length = 10
# '0' spots count
zeros = length - (sum(seqs))
# partitions count
partitions = len(seqs) + 1
# first and last can partitions have 0 zeros
# so use a flag when we call the function or check if it's the last partition
def generate_gaps(zeros_left, partition, first=False):
"""
:param zeros_left: how many zeros we can still use
:param partition: what partition is this
:param first: is this the first gap
:return: all possible gaps
"""
for gap in range((0 if first or partition == 0 else 1), zeros_left + 1):
if partition == 0:
if (zeros_left - gap) == 0:
yield [gap]
else:
for rest in generate_gaps(zeros_left - gap, partition - 1):
yield [gap] + rest
for gaps in generate_gaps(zeros, partitions - 1, True):
print "using gaps: " + str(gaps)
# merge lists
# zip gaps (0's) and sequences (1's) - all but last gap (added to result)
gaps_seqs = zip(gaps, seqs)
# expand everything... magic (could be done explicitly trivially).
result = sum(map(lambda x: [0] * x[0] + [1] * x[1], gaps_seqs)
# last gap (truncated from zip)
result = result + [[0] * gaps[-1]], [])
A simple non-recursive generator solution without itertools:
def fill_sequence(sequence, size):
n_slots = size - len(sequence)
for start in xrange(n_slots + 1):
yield [0]*start + sequence + [0]*(n_slots - start)
def all_placements(inner_sizes, outer_size):
x, y = inner_sizes
for margin in xrange(1, outer_size - sum(block_sizes) + 1):
sequence = [1]*x + [0]*margin + [1]*y
for p in fill_sequence(sequence, outer_size):
yield p
So that:
>>> list(all_placements((4,3), 10))
[[1, 1, 1, 1, 0, 1, 1, 1, 0, 0],
[0, 1, 1, 1, 1, 0, 1, 1, 1, 0],
[0, 0, 1, 1, 1, 1, 0, 1, 1, 1],
[1, 1, 1, 1, 0, 0, 1, 1, 1, 0],
[0, 1, 1, 1, 1, 0, 0, 1, 1, 1],
[1, 1, 1, 1, 0, 0, 0, 1, 1, 1]]
The idea is quite simple. Suppose you fix the number of zeros between your two blocks of ones, call it the margin. This gives you a 4 + margin + 3 sequence. You can easily place this sequence in the larger list of zeros using the approach you took in your post. Then simply iteratively increase the margin, yielding all possible placements.
Given a list of data as follows:
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
I would like to create an algorithm that is able to offset the list of certain number of steps. For example, if the offset = -1:
def offsetFunc(inputList, offsetList):
#make something
return output
where:
output = [0,0,0,0,1,1,5,5,5,5,5,5,3,3,3,2,2]
Important Note: The elements of the list are float numbers and they are not in any progression. So I actually need to shift them, I cannot use any work-around for getting the result.
So basically, the algorithm should replace the first set of values (the 4 "1", basically) with the 0 and then it should:
Detect the lenght of the next range of values
Create a parallel output vectors with the values delayed by one set
The way I have roughly described the algorithm above is how I would do it. However I'm a newbie to Python (and even beginner in general programming) and I have figured out time by time that Python has a lot of built-in functions that could make the algorithm less heavy and iterating. Does anyone have any suggestion to better develop a script to make this kind of job? This is the code I have written so far (assuming a static offset at -1):
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
output = []
PrevVal = 0
NextVal = input[0]
i = 0
while input[i] == NextVal:
output.append(PrevVal)
i += 1
while i < len(input):
PrevVal = NextVal
NextVal = input[i]
while input[i] == NextVal:
output.append(PrevVal)
i += 1
if i >= len(input):
break
print output
Thanks in advance for any help!
BETTER DESCRIPTION
My list will always be composed of "sets" of values. They are usually float numbers, and they take values such as this short example below:
Sample = [1.236,1.236,1.236,1.236,1.863,1.863,1.863,1.863,1.863,1.863]
In this example, the first set (the one with value "1.236") is long 4 while the second one is long 6. What I would like to get as an output, when the offset = -1, is:
The value "0.000" in the first 4 elements;
The value "1.236" in the second 6 elements.
So basically, this "offset" function is creating the list with the same "structure" (ranges of lengths) but with the values delayed by "offset" times.
I hope it's clear now, unfortunately the problem itself is still a bit silly to me (plus I don't even speak good English :) )
Please don't hesitate to ask any additional info to complete the question and make it clearer.
How about this:
def generateOutput(input, value=0, offset=-1):
values = []
for i in range(len(input)):
if i < 1 or input[i] == input[i-1]:
yield value
else: # value change in input detected
values.append(input[i-1])
if len(values) >= -offset:
value = values.pop(0)
yield value
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
print list(generateOutput(input))
It will print this:
[0, 0, 0, 0, 1, 1, 5, 5, 5, 5, 5, 5, 3, 3, 3, 2, 2]
And in case you just want to iterate, you do not even need to build the list. Just use for i in generateOutput(input): … then.
For other offsets, use this:
print list(generateOutput(input, 0, -2))
prints:
[0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 5, 5, 5, 3, 3]
Using deque as the queue, and using maxlen to define the shift length. Only holding unique values. pushing inn new values at the end, pushes out old values at the start of the queue, when the shift length has been reached.
from collections import deque
def shift(it, shift=1):
q = deque(maxlen=shift+1)
q.append(0)
for i in it:
if q[-1] != i:
q.append(i)
yield q[0]
Sample = [1.236,1.236,1.236,1.236,1.863,1.863,1.863,1.863,1.863,1.863]
print list(shift(Sample))
#[0, 0, 0, 0, 1.236, 1.236, 1.236, 1.236, 1.236, 1.236]
My try:
#Input
input = [1,1,1,1,5,5,3,3,3,3,3,3,2,2,2,5,5]
shift = -1
#Build service structures: for each 'set of data' store its length and its value
set_lengths = []
set_values = []
prev_value = None
set_length = 0
for value in input:
if prev_value is not None and value != prev_value:
set_lengths.append(set_length)
set_values.append(prev_value)
set_length = 0
set_length += 1
prev_value = value
else:
set_lengths.append(set_length)
set_values.append(prev_value)
#Output the result, shifting the values
output = []
for i, l in enumerate(set_lengths):
j = i + shift
if j < 0:
output += [0] * l
else:
output += [set_values[j]] * l
print input
print output
gives:
[1, 1, 1, 1, 5, 5, 3, 3, 3, 3, 3, 3, 2, 2, 2, 5, 5]
[0, 0, 0, 0, 1, 1, 5, 5, 5, 5, 5, 5, 3, 3, 3, 2, 2]
def x(list, offset):
return [el + offset for el in list]
A completely different approach than my first answer is this:
import itertools
First analyze the input:
values, amounts = zip(*((n, len(list(g))) for n, g in itertools.groupby(input)))
We now have (1, 5, 3, 2, 5) and (4, 2, 6, 3, 2). Now apply the offset:
values = (0,) * (-offset) + values # nevermind that it is longer now.
And synthesize it again:
output = sum([ [v] * a for v, a in zip(values, amounts) ], [])
This is way more elegant, way less understandable and probably way more expensive than my other answer, but I didn't want to hide it from you.