Develop Reference

Python, permutation to permuation-index function

I have some permutations of a list:
>>> import itertools
>>> perms = list(itertools.permutations([0,1,2,3]))
>>> perms
[(0, 1, 2, 3), (0, 1, 3, 2), (0, 2, 1, 3), (0, 2, 3, 1), (0, 3, 1, 2), (0, 3, 2, 1), (1, 0, 2, 3), (1, 0, 3, 2), (1, 2, 0, 3), (1, 2, 3, 0), (1, 3, 0, 2), (1, 3, 2, 0), (2, 0, 1, 3), (2, 0, 3, 1), (2, 1, 0, 3), (2, 1, 3, 0), (2, 3, 0, 1), (2, 3, 1, 0), (3, 0, 1, 2), (3, 0, 2, 1), (3, 1, 0, 2), (3, 1, 2, 0), (3, 2, 0, 1), (3, 2, 1, 0)]
>>> len(perms)
24
What function can I use (without access to the list perm) to get the index of an arbitrary permutation, e.g. (0, 2, 3, 1) -> 3?
(You can assume that permuted elements are always an ascending list of integers, starting at zero.)
Hint: The factorial number system may be involved. https://en.wikipedia.org/wiki/Factorial_number_system

Off the top of my head I came up with the following, didn't test it thoroughly.
from math import factorial
elements = list(range(4))
permutation = (3, 2, 1, 0)
index = 0
nf = factorial(len(elements))
for n in permutation:
nf //= len(elements)
index += elements.index(n) * nf
elements.remove(n)
print(index)
EDIT: replaced nf /= len(elements) with nf //= len(elements)

I suppose this is a challenge, so here is my (recursive) answer:
import math
import itertools
def get_index(l):
# In a real function, there should be more tests to validate that the input is valid, e.g. len(l)>0
# Terminal case
if len(l)==1:
return 0
# Number of possible permutations starting with l[0]
span = math.factorial(len(l)-1)
# Slightly modifying l[1:] to use the function recursively
new_l = [ val if val < l[0] else val-1 for val in l[1:] ]
# Actual solution
return get_index(new_l) + span*l[0]
get_index((0,1,2,3))
# 0
get_index((0,2,3,1))
# 3
get_index((3,2,1,0))
# 23
get_index((4,2,0,1,5,3))
# 529
list(itertools.permutations((0,1,2,3,4,5))).index((4,2,0,1,5,3))
# 529

You need to write your own function. Something like this would work
import math
def perm_loc(P):
N = len(P)
assert set(P) == set(range(N))
def rec(perm):
nums = set(perm)
if not perm:
return 0
else:
sub_res = rec(perm[1:]) # Result for tail of permutation
sub_size = math.factorial(len(nums) - 1) # How many tail permutations exist
sub_index = sorted(nums).index(perm[0]) # Location of first element in permutaiotn
# in the sorted list of number
return sub_index * sub_size + sub_res
return rec(P)
The function that does all the work is rec, with perm_loc just serving as a wrapper around it. Note that this algorithm is based on the nature of the permutation algorithm that itertools.permutation happens to use.
The following code tests the above function. First on your sample, and then on all permutations of range(7):
print perm_loc([0,2,3,1]) # Print the result from the example
import itertools
def test(N):
correct = 0
perms = list(itertools.permutations(range(N)))
for (i, p) in enumerate(perms):
pl = perm_loc(p)
if i == pl:
correct += 1
else:
print ":: Incorrect", p, perms.index(p), perm_loc(N, p)
print ":: Found %d correct results" % correct
test(7) # Test on all permutations of range(7)

from math import factorial
def perm_to_permidx(perm):
# Extract info
n = len(perm)
elements = range(n)
# "Gone"s will be the elements of the given perm
gones = []
# According to each number in perm, we add the repsective offsets
offset = 0
for i, num in enumerate(perm[:-1], start=1):
idx = num - sum(num > gone for gone in gones)
offset += idx * factorial(n - i)
gones.append(num)
return offset
the_perm = (0, 2, 3, 1)
print(perm_to_permidx(the_perm))
# 3
Explanation: All permutations of a given range can be considered as a groups of permutations. So, for example, for the permutations of 0, 1, 2, 3 we first "fix" 0 and permute rest, then fix 1 and permute rest, and so on. Once we fix a number, the rest is again permutations; so we again fix a number at a time from the remaining numbers and permute the rest. This goes on till we are left with one number only. Every level of fixing has a corresponding (n-i)! permutations.
So this code finds the "offsets" for each level of permutation. The offset corresonds to where the given permutation starts when we fix numbers of perm in order. For the given example of (0, 2, 3, 1), we first look at the first number in the given perm which is 0, and figure the offset as 0. Then this goes to gones list (we will see its usage). Then, at the next level of permutation we see 2 as the fixing number. To calculate the offset for this, we need the "order" of this 2 among the remaining three numbers. This is where gones come into play; if an already-fixed and considered number (in this case 0) is less than the current fixer, we subtract 1 to find the new order. Then offset is calculated and accumulated. For the next number 3, the new order is 3 - (1 + 1) = 1 because both previous fixers 0 and 2 are at the "left" of 3.
This goes on till the last number of the given perm since there is no need to look at it; it will have been determined anyway.

Create all permutations with repeated elements, without repeating the permutation itself [duplicate]

itertools.permutations generates where its elements are treated as unique based on their position, not on their value. So basically I want to avoid duplicates like this:
>>> list(itertools.permutations([1, 1, 1]))
[(1, 1, 1), (1, 1, 1), (1, 1, 1), (1, 1, 1), (1, 1, 1), (1, 1, 1)]
Filtering afterwards is not possible because the amount of permutations is too large in my case.
Does anybody know of a suitable algorithm for this?
Thank you very much!
EDIT:
What I basically want is the following:
x = itertools.product((0, 1, 'x'), repeat=X)
x = sorted(x, key=functools.partial(count_elements, elem='x'))
which is not possible because sorted creates a list and the output of itertools.product is too large.
Sorry, I should have described the actual problem.

class unique_element:
def __init__(self,value,occurrences):
self.value = value
self.occurrences = occurrences
def perm_unique(elements):
eset=set(elements)
listunique = [unique_element(i,elements.count(i)) for i in eset]
u=len(elements)
return perm_unique_helper(listunique,[0]*u,u-1)
def perm_unique_helper(listunique,result_list,d):
if d < 0:
yield tuple(result_list)
else:
for i in listunique:
if i.occurrences > 0:
result_list[d]=i.value
i.occurrences-=1
for g in perm_unique_helper(listunique,result_list,d-1):
yield g
i.occurrences+=1
a = list(perm_unique([1,1,2]))
print(a)
result:
[(2, 1, 1), (1, 2, 1), (1, 1, 2)]
EDIT (how this works):
I rewrote the above program to be longer but more readable.
I usually have a hard time explaining how something works, but let me try.
In order to understand how this works, you have to understand a similar but simpler program that would yield all permutations with repetitions.
def permutations_with_replacement(elements,n):
return permutations_helper(elements,[0]*n,n-1)#this is generator
def permutations_helper(elements,result_list,d):
if d<0:
yield tuple(result_list)
else:
for i in elements:
result_list[d]=i
all_permutations = permutations_helper(elements,result_list,d-1)#this is generator
for g in all_permutations:
yield g
This program is obviously much simpler:
d stands for depth in permutations_helper and has two functions. One function is the stopping condition of our recursive algorithm, and the other is for the result list that is passed around.
Instead of returning each result, we yield it. If there were no function/operator yield we would have to push the result in some queue at the point of the stopping condition. But this way, once the stopping condition is met, the result is propagated through all stacks up to the caller. That is the purpose of
for g in perm_unique_helper(listunique,result_list,d-1): yield g
so each result is propagated up to caller.
Back to the original program:
we have a list of unique elements. Before we can use each element, we have to check how many of them are still available to push onto result_list. Working with this program is very similar to permutations_with_replacement. The difference is that each element cannot be repeated more times than it is in perm_unique_helper.

Because sometimes new questions are marked as duplicates and their authors are referred to this question it may be important to mention that sympy has an iterator for this purpose.
>>> from sympy.utilities.iterables import multiset_permutations
>>> list(multiset_permutations([1,1,1]))
[[1, 1, 1]]
>>> list(multiset_permutations([1,1,2]))
[[1, 1, 2], [1, 2, 1], [2, 1, 1]]

This relies on the implementation detail that any permutation of a sorted iterable are in sorted order unless they are duplicates of prior permutations.
from itertools import permutations
def unique_permutations(iterable, r=None):
previous = tuple()
for p in permutations(sorted(iterable), r):
if p > previous:
previous = p
yield p
for p in unique_permutations('cabcab', 2):
print p
gives
('a', 'a')
('a', 'b')
('a', 'c')
('b', 'a')
('b', 'b')
('b', 'c')
('c', 'a')
('c', 'b')
('c', 'c')

Roughly as fast as Luka Rahne's answer, but shorter & simpler, IMHO.
def unique_permutations(elements):
if len(elements) == 1:
yield (elements[0],)
else:
unique_elements = set(elements)
for first_element in unique_elements:
remaining_elements = list(elements)
remaining_elements.remove(first_element)
for sub_permutation in unique_permutations(remaining_elements):
yield (first_element,) + sub_permutation
>>> list(unique_permutations((1,2,3,1)))
[(1, 1, 2, 3), (1, 1, 3, 2), (1, 2, 1, 3), ... , (3, 1, 2, 1), (3, 2, 1, 1)]
It works recursively by setting the first element (iterating through all unique elements), and iterating through the permutations for all remaining elements.
Let's go through the unique_permutations of (1,2,3,1) to see how it works:
unique_elements are 1,2,3
Let's iterate through them: first_element starts with 1.
remaining_elements are [2,3,1] (ie. 1,2,3,1 minus the first 1)
We iterate (recursively) through the permutations of the remaining elements: (1, 2, 3), (1, 3, 2), (2, 1, 3), (2, 3, 1), (3, 1, 2), (3, 2, 1)
For each sub_permutation, we insert the first_element: (1,1,2,3), (1,1,3,2), ... and yield the result.
Now we iterate to first_element = 2, and do the same as above.
remaining_elements are [1,3,1] (ie. 1,2,3,1 minus the first 2)
We iterate through the permutations of the remaining elements: (1, 1, 3), (1, 3, 1), (3, 1, 1)
For each sub_permutation, we insert the first_element: (2, 1, 1, 3), (2, 1, 3, 1), (2, 3, 1, 1)... and yield the result.
Finally, we do the same with first_element = 3.

You could try using set:
>>> list(itertools.permutations(set([1,1,2,2])))
[(1, 2), (2, 1)]
The call to set removed duplicates

A naive approach might be to take the set of the permutations:
list(set(it.permutations([1, 1, 1])))
# [(1, 1, 1)]
However, this technique wastefully computes replicate permutations and discards them. A more efficient approach would be more_itertools.distinct_permutations, a third-party tool.
Code
import itertools as it
import more_itertools as mit
list(mit.distinct_permutations([1, 1, 1]))
# [(1, 1, 1)]
Performance
Using a larger iterable, we will compare the performances between the naive and third-party techniques.
iterable = [1, 1, 1, 1, 1, 1]
len(list(it.permutations(iterable)))
# 720
%timeit -n 10000 list(set(it.permutations(iterable)))
# 10000 loops, best of 3: 111 µs per loop
%timeit -n 10000 list(mit.distinct_permutations(iterable))
# 10000 loops, best of 3: 16.7 µs per loop
We see more_itertools.distinct_permutations is an order of magnitude faster.
Details
From the source, a recursion algorithm (as seen in the accepted answer) is used to compute distinct permutations, thereby obviating wasteful computations. See the source code for more details.

This is my solution with 10 lines:
class Solution(object):
def permute_unique(self, nums):
perms = [[]]
for n in nums:
new_perm = []
for perm in perms:
for i in range(len(perm) + 1):
new_perm.append(perm[:i] + [n] + perm[i:])
# handle duplication
if i < len(perm) and perm[i] == n: break
perms = new_perm
return perms
if __name__ == '__main__':
s = Solution()
print s.permute_unique([1, 1, 1])
print s.permute_unique([1, 2, 1])
print s.permute_unique([1, 2, 3])
--- Result ----
[[1, 1, 1]]
[[1, 2, 1], [2, 1, 1], [1, 1, 2]]
[[3, 2, 1], [2, 3, 1], [2, 1, 3], [3, 1, 2], [1, 3, 2], [1, 2, 3]]

Here is a recursive solution to the problem.
def permutation(num_array):
res=[]
if len(num_array) <= 1:
return [num_array]
for num in set(num_array):
temp_array = num_array.copy()
temp_array.remove(num)
res += [[num] + perm for perm in permutation(temp_array)]
return res
arr=[1,2,2]
print(permutation(arr))

The best solution to this problem I have seen uses Knuth's "Algorithm L" (as noted previously by Gerrat in the comments to the original post):
http://stackoverflow.com/questions/12836385/how-can-i-interleave-or-create-unique-permutations-of-two-stings-without-recurs/12837695
Some timings:
Sorting [1]*12+[0]*12 (2,704,156 unique permutations):
Algorithm L → 2.43 s
Luke Rahne's solution → 8.56 s
scipy.multiset_permutations() → 16.8 s

To generate unique permutations of ["A","B","C","D"] I use the following:
from itertools import combinations,chain
l = ["A","B","C","D"]
combs = (combinations(l, r) for r in range(1, len(l) + 1))
list_combinations = list(chain.from_iterable(combs))
Which generates:
[('A',),
('B',),
('C',),
('D',),
('A', 'B'),
('A', 'C'),
('A', 'D'),
('B', 'C'),
('B', 'D'),
('C', 'D'),
('A', 'B', 'C'),
('A', 'B', 'D'),
('A', 'C', 'D'),
('B', 'C', 'D'),
('A', 'B', 'C', 'D')]
Notice, duplicates are not created (e.g. items in combination with D are not generated, as they already exist).
Example: This can then be used in generating terms of higher or lower order for OLS models via data in a Pandas dataframe.
import statsmodels.formula.api as smf
import pandas as pd
# create some data
pd_dataframe = pd.Dataframe(somedata)
response_column = "Y"
# generate combinations of column/variable names
l = [col for col in pd_dataframe.columns if col!=response_column]
combs = (combinations(l, r) for r in range(1, len(l) + 1))
list_combinations = list(chain.from_iterable(combs))
# generate OLS input string
formula_base = '{} ~ '.format(response_column)
list_for_ols = [":".join(list(item)) for item in list_combinations]
string_for_ols = formula_base + ' + '.join(list_for_ols)
Creates...
Y ~ A + B + C + D + A:B + A:C + A:D + B:C + B:D + C:D + A:B:C + A:B:D + A:C:D + B:C:D + A:B:C:D'
Which can then be piped to your OLS regression
model = smf.ols(string_for_ols, pd_dataframe).fit()
model.summary()

It sound like you are looking for itertools.combinations() docs.python.org
list(itertools.combinations([1, 1, 1],3))
[(1, 1, 1)]

Bumped into this question while looking for something myself !
Here's what I did:
def dont_repeat(x=[0,1,1,2]): # Pass a list
from itertools import permutations as per
uniq_set = set()
for byt_grp in per(x, 4):
if byt_grp not in uniq_set:
yield byt_grp
uniq_set.update([byt_grp])
print uniq_set
for i in dont_repeat(): print i
(0, 1, 1, 2)
(0, 1, 2, 1)
(0, 2, 1, 1)
(1, 0, 1, 2)
(1, 0, 2, 1)
(1, 1, 0, 2)
(1, 1, 2, 0)
(1, 2, 0, 1)
(1, 2, 1, 0)
(2, 0, 1, 1)
(2, 1, 0, 1)
(2, 1, 1, 0)
set([(0, 1, 1, 2), (1, 0, 1, 2), (2, 1, 0, 1), (1, 2, 0, 1), (0, 1, 2, 1), (0, 2, 1, 1), (1, 1, 2, 0), (1, 2, 1, 0), (2, 1, 1, 0), (1, 0, 2, 1), (2, 0, 1, 1), (1, 1, 0, 2)])
Basically, make a set and keep adding to it. Better than making lists etc. that take too much memory..
Hope it helps the next person looking out :-) Comment out the set 'update' in the function to see the difference.

You can make a function that uses collections.Counter to get unique items and their counts from the given sequence, and uses itertools.combinations to pick combinations of indices for each unique item in each recursive call, and map the indices back to a list when all indices are picked:
from collections import Counter
from itertools import combinations
def unique_permutations(seq):
def index_permutations(counts, index_pool):
if not counts:
yield {}
return
(item, count), *rest = counts.items()
rest = dict(rest)
for indices in combinations(index_pool, count):
mapping = dict.fromkeys(indices, item)
for others in index_permutations(rest, index_pool.difference(indices)):
yield {**mapping, **others}
indices = set(range(len(seq)))
for mapping in index_permutations(Counter(seq), indices):
yield [mapping[i] for i in indices]
so that [''.join(i) for i in unique_permutations('moon')] returns:
['moon', 'mono', 'mnoo', 'omon', 'omno', 'nmoo', 'oomn', 'onmo', 'nomo', 'oonm', 'onom', 'noom']

This is my attempt without resorting to set / dict, as a generator using recursion, but using string as input. Output is also ordered in natural order:
def perm_helper(head: str, tail: str):
if len(tail) == 0:
yield head
else:
last_c = None
for index, c in enumerate(tail):
if last_c != c:
last_c = c
yield from perm_helper(
head + c, tail[:index] + tail[index + 1:]
)
def perm_generator(word):
yield from perm_helper("", sorted(word))
example:
from itertools import takewhile
word = "POOL"
list(takewhile(lambda w: w != word, (x for x in perm_generator(word))))
# output
# ['LOOP', 'LOPO', 'LPOO', 'OLOP', 'OLPO', 'OOLP', 'OOPL', 'OPLO', 'OPOL', 'PLOO', 'POLO']

ans=[]
def fn(a, size):
if (size == 1):
if a.copy() not in ans:
ans.append(a.copy())
return
for i in range(size):
fn(a,size-1);
if size&1:
a[0], a[size-1] = a[size-1],a[0]
else:
a[i], a[size-1] = a[size-1],a[i]
https://www.geeksforgeeks.org/heaps-algorithm-for-generating-permutations/

Came across this the other day while working on a problem of my own. I like Luka Rahne's approach, but I thought that using the Counter class in the collections library seemed like a modest improvement. Here's my code:
def unique_permutations(elements):
"Returns a list of lists; each sublist is a unique permutations of elements."
ctr = collections.Counter(elements)
# Base case with one element: just return the element
if len(ctr.keys())==1 and ctr[ctr.keys()[0]] == 1:
return [[ctr.keys()[0]]]
perms = []
# For each counter key, find the unique permutations of the set with
# one member of that key removed, and append the key to the front of
# each of those permutations.
for k in ctr.keys():
ctr_k = ctr.copy()
ctr_k[k] -= 1
if ctr_k[k]==0:
ctr_k.pop(k)
perms_k = [[k] + p for p in unique_permutations(ctr_k)]
perms.extend(perms_k)
return perms
This code returns each permutation as a list. If you feed it a string, it'll give you a list of permutations where each one is a list of characters. If you want the output as a list of strings instead (for example, if you're a terrible person and you want to abuse my code to help you cheat in Scrabble), just do the following:
[''.join(perm) for perm in unique_permutations('abunchofletters')]

I came up with a very suitable implementation using itertools.product in this case (this is an implementation where you want all combinations
unique_perm_list = [''.join(p) for p in itertools.product(['0', '1'], repeat = X) if ''.join(p).count() == somenumber]
this is essentially a combination (n over k) with n = X and somenumber = k
itertools.product() iterates from k = 0 to k = X subsequent filtering with count ensures that just the permutations with the right number of ones are cast into a list. you can easily see that it works when you calculate n over k and compare it to the len(unique_perm_list)

Adapted to remove recursion, use a dictionary and numba for high performance but not using yield/generator style so memory usage is not limited:
import numba
#numba.njit
def perm_unique_fast(elements): #memory usage too high for large permutations
eset = set(elements)
dictunique = dict()
for i in eset: dictunique[i] = elements.count(i)
result_list = numba.typed.List()
u = len(elements)
for _ in range(u): result_list.append(0)
s = numba.typed.List()
results = numba.typed.List()
d = u
while True:
if d > 0:
for i in dictunique:
if dictunique[i] > 0: s.append((i, d - 1))
i, d = s.pop()
if d == -1:
dictunique[i] += 1
if len(s) == 0: break
continue
result_list[d] = i
if d == 0: results.append(result_list[:])
dictunique[i] -= 1
s.append((i, -1))
return results
import timeit
l = [2, 2, 3, 3, 4, 4, 5, 5, 6, 6]
%timeit list(perm_unique(l))
#377 ms ± 26 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
ltyp = numba.typed.List()
for x in l: ltyp.append(x)
%timeit perm_unique_fast(ltyp)
#293 ms ± 3.37 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
assert list(sorted(perm_unique(l))) == list(sorted([tuple(x) for x in perm_unique_fast(ltyp)]))
About 30% faster but still suffers a bit due to list copying and management.
Alternatively without numba but still without recursion and using a generator to avoid memory issues:
def perm_unique_fast_gen(elements):
eset = set(elements)
dictunique = dict()
for i in eset: dictunique[i] = elements.count(i)
result_list = list() #numba.typed.List()
u = len(elements)
for _ in range(u): result_list.append(0)
s = list()
d = u
while True:
if d > 0:
for i in dictunique:
if dictunique[i] > 0: s.append((i, d - 1))
i, d = s.pop()
if d == -1:
dictunique[i] += 1
if len(s) == 0: break
continue
result_list[d] = i
if d == 0: yield result_list
dictunique[i] -= 1
s.append((i, -1))

May be we can use set here to obtain unique permutations
import itertools
print('unique perms >> ', set(itertools.permutations(A)))

What about
np.unique(itertools.permutations([1, 1, 1]))
The problem is the permutations are now rows of a Numpy array, thus using more memory, but you can cycle through them as before
perms = np.unique(itertools.permutations([1, 1, 1]))
for p in perms:
print p

Efficient permutation of large arrays under symmetry constraints in Python [duplicate]

itertools.permutations generates where its elements are treated as unique based on their position, not on their value. So basically I want to avoid duplicates like this:
>>> list(itertools.permutations([1, 1, 1]))
[(1, 1, 1), (1, 1, 1), (1, 1, 1), (1, 1, 1), (1, 1, 1), (1, 1, 1)]
Filtering afterwards is not possible because the amount of permutations is too large in my case.
Does anybody know of a suitable algorithm for this?
Thank you very much!
EDIT:
What I basically want is the following:
x = itertools.product((0, 1, 'x'), repeat=X)
x = sorted(x, key=functools.partial(count_elements, elem='x'))
which is not possible because sorted creates a list and the output of itertools.product is too large.
Sorry, I should have described the actual problem.

class unique_element:
def __init__(self,value,occurrences):
self.value = value
self.occurrences = occurrences
def perm_unique(elements):
eset=set(elements)
listunique = [unique_element(i,elements.count(i)) for i in eset]
u=len(elements)
return perm_unique_helper(listunique,[0]*u,u-1)
def perm_unique_helper(listunique,result_list,d):
if d < 0:
yield tuple(result_list)
else:
for i in listunique:
if i.occurrences > 0:
result_list[d]=i.value
i.occurrences-=1
for g in perm_unique_helper(listunique,result_list,d-1):
yield g
i.occurrences+=1
a = list(perm_unique([1,1,2]))
print(a)
result:
[(2, 1, 1), (1, 2, 1), (1, 1, 2)]
EDIT (how this works):
I rewrote the above program to be longer but more readable.
I usually have a hard time explaining how something works, but let me try.
In order to understand how this works, you have to understand a similar but simpler program that would yield all permutations with repetitions.
def permutations_with_replacement(elements,n):
return permutations_helper(elements,[0]*n,n-1)#this is generator
def permutations_helper(elements,result_list,d):
if d<0:
yield tuple(result_list)
else:
for i in elements:
result_list[d]=i
all_permutations = permutations_helper(elements,result_list,d-1)#this is generator
for g in all_permutations:
yield g
This program is obviously much simpler:
d stands for depth in permutations_helper and has two functions. One function is the stopping condition of our recursive algorithm, and the other is for the result list that is passed around.
Instead of returning each result, we yield it. If there were no function/operator yield we would have to push the result in some queue at the point of the stopping condition. But this way, once the stopping condition is met, the result is propagated through all stacks up to the caller. That is the purpose of
for g in perm_unique_helper(listunique,result_list,d-1): yield g
so each result is propagated up to caller.
Back to the original program:
we have a list of unique elements. Before we can use each element, we have to check how many of them are still available to push onto result_list. Working with this program is very similar to permutations_with_replacement. The difference is that each element cannot be repeated more times than it is in perm_unique_helper.

Because sometimes new questions are marked as duplicates and their authors are referred to this question it may be important to mention that sympy has an iterator for this purpose.
>>> from sympy.utilities.iterables import multiset_permutations
>>> list(multiset_permutations([1,1,1]))
[[1, 1, 1]]
>>> list(multiset_permutations([1,1,2]))
[[1, 1, 2], [1, 2, 1], [2, 1, 1]]

This relies on the implementation detail that any permutation of a sorted iterable are in sorted order unless they are duplicates of prior permutations.
from itertools import permutations
def unique_permutations(iterable, r=None):
previous = tuple()
for p in permutations(sorted(iterable), r):
if p > previous:
previous = p
yield p
for p in unique_permutations('cabcab', 2):
print p
gives
('a', 'a')
('a', 'b')
('a', 'c')
('b', 'a')
('b', 'b')
('b', 'c')
('c', 'a')
('c', 'b')
('c', 'c')

Roughly as fast as Luka Rahne's answer, but shorter & simpler, IMHO.
def unique_permutations(elements):
if len(elements) == 1:
yield (elements[0],)
else:
unique_elements = set(elements)
for first_element in unique_elements:
remaining_elements = list(elements)
remaining_elements.remove(first_element)
for sub_permutation in unique_permutations(remaining_elements):
yield (first_element,) + sub_permutation
>>> list(unique_permutations((1,2,3,1)))
[(1, 1, 2, 3), (1, 1, 3, 2), (1, 2, 1, 3), ... , (3, 1, 2, 1), (3, 2, 1, 1)]
It works recursively by setting the first element (iterating through all unique elements), and iterating through the permutations for all remaining elements.
Let's go through the unique_permutations of (1,2,3,1) to see how it works:
unique_elements are 1,2,3
Let's iterate through them: first_element starts with 1.
remaining_elements are [2,3,1] (ie. 1,2,3,1 minus the first 1)
We iterate (recursively) through the permutations of the remaining elements: (1, 2, 3), (1, 3, 2), (2, 1, 3), (2, 3, 1), (3, 1, 2), (3, 2, 1)
For each sub_permutation, we insert the first_element: (1,1,2,3), (1,1,3,2), ... and yield the result.
Now we iterate to first_element = 2, and do the same as above.
remaining_elements are [1,3,1] (ie. 1,2,3,1 minus the first 2)
We iterate through the permutations of the remaining elements: (1, 1, 3), (1, 3, 1), (3, 1, 1)
For each sub_permutation, we insert the first_element: (2, 1, 1, 3), (2, 1, 3, 1), (2, 3, 1, 1)... and yield the result.
Finally, we do the same with first_element = 3.

You could try using set:
>>> list(itertools.permutations(set([1,1,2,2])))
[(1, 2), (2, 1)]
The call to set removed duplicates

A naive approach might be to take the set of the permutations:
list(set(it.permutations([1, 1, 1])))
# [(1, 1, 1)]
However, this technique wastefully computes replicate permutations and discards them. A more efficient approach would be more_itertools.distinct_permutations, a third-party tool.
Code
import itertools as it
import more_itertools as mit
list(mit.distinct_permutations([1, 1, 1]))
# [(1, 1, 1)]
Performance
Using a larger iterable, we will compare the performances between the naive and third-party techniques.
iterable = [1, 1, 1, 1, 1, 1]
len(list(it.permutations(iterable)))
# 720
%timeit -n 10000 list(set(it.permutations(iterable)))
# 10000 loops, best of 3: 111 µs per loop
%timeit -n 10000 list(mit.distinct_permutations(iterable))
# 10000 loops, best of 3: 16.7 µs per loop
We see more_itertools.distinct_permutations is an order of magnitude faster.
Details
From the source, a recursion algorithm (as seen in the accepted answer) is used to compute distinct permutations, thereby obviating wasteful computations. See the source code for more details.

This is my solution with 10 lines:
class Solution(object):
def permute_unique(self, nums):
perms = [[]]
for n in nums:
new_perm = []
for perm in perms:
for i in range(len(perm) + 1):
new_perm.append(perm[:i] + [n] + perm[i:])
# handle duplication
if i < len(perm) and perm[i] == n: break
perms = new_perm
return perms
if __name__ == '__main__':
s = Solution()
print s.permute_unique([1, 1, 1])
print s.permute_unique([1, 2, 1])
print s.permute_unique([1, 2, 3])
--- Result ----
[[1, 1, 1]]
[[1, 2, 1], [2, 1, 1], [1, 1, 2]]
[[3, 2, 1], [2, 3, 1], [2, 1, 3], [3, 1, 2], [1, 3, 2], [1, 2, 3]]

Here is a recursive solution to the problem.
def permutation(num_array):
res=[]
if len(num_array) <= 1:
return [num_array]
for num in set(num_array):
temp_array = num_array.copy()
temp_array.remove(num)
res += [[num] + perm for perm in permutation(temp_array)]
return res
arr=[1,2,2]
print(permutation(arr))

The best solution to this problem I have seen uses Knuth's "Algorithm L" (as noted previously by Gerrat in the comments to the original post):
http://stackoverflow.com/questions/12836385/how-can-i-interleave-or-create-unique-permutations-of-two-stings-without-recurs/12837695
Some timings:
Sorting [1]*12+[0]*12 (2,704,156 unique permutations):
Algorithm L → 2.43 s
Luke Rahne's solution → 8.56 s
scipy.multiset_permutations() → 16.8 s

To generate unique permutations of ["A","B","C","D"] I use the following:
from itertools import combinations,chain
l = ["A","B","C","D"]
combs = (combinations(l, r) for r in range(1, len(l) + 1))
list_combinations = list(chain.from_iterable(combs))
Which generates:
[('A',),
('B',),
('C',),
('D',),
('A', 'B'),
('A', 'C'),
('A', 'D'),
('B', 'C'),
('B', 'D'),
('C', 'D'),
('A', 'B', 'C'),
('A', 'B', 'D'),
('A', 'C', 'D'),
('B', 'C', 'D'),
('A', 'B', 'C', 'D')]
Notice, duplicates are not created (e.g. items in combination with D are not generated, as they already exist).
Example: This can then be used in generating terms of higher or lower order for OLS models via data in a Pandas dataframe.
import statsmodels.formula.api as smf
import pandas as pd
# create some data
pd_dataframe = pd.Dataframe(somedata)
response_column = "Y"
# generate combinations of column/variable names
l = [col for col in pd_dataframe.columns if col!=response_column]
combs = (combinations(l, r) for r in range(1, len(l) + 1))
list_combinations = list(chain.from_iterable(combs))
# generate OLS input string
formula_base = '{} ~ '.format(response_column)
list_for_ols = [":".join(list(item)) for item in list_combinations]
string_for_ols = formula_base + ' + '.join(list_for_ols)
Creates...
Y ~ A + B + C + D + A:B + A:C + A:D + B:C + B:D + C:D + A:B:C + A:B:D + A:C:D + B:C:D + A:B:C:D'
Which can then be piped to your OLS regression
model = smf.ols(string_for_ols, pd_dataframe).fit()
model.summary()

It sound like you are looking for itertools.combinations() docs.python.org
list(itertools.combinations([1, 1, 1],3))
[(1, 1, 1)]

Bumped into this question while looking for something myself !
Here's what I did:
def dont_repeat(x=[0,1,1,2]): # Pass a list
from itertools import permutations as per
uniq_set = set()
for byt_grp in per(x, 4):
if byt_grp not in uniq_set:
yield byt_grp
uniq_set.update([byt_grp])
print uniq_set
for i in dont_repeat(): print i
(0, 1, 1, 2)
(0, 1, 2, 1)
(0, 2, 1, 1)
(1, 0, 1, 2)
(1, 0, 2, 1)
(1, 1, 0, 2)
(1, 1, 2, 0)
(1, 2, 0, 1)
(1, 2, 1, 0)
(2, 0, 1, 1)
(2, 1, 0, 1)
(2, 1, 1, 0)
set([(0, 1, 1, 2), (1, 0, 1, 2), (2, 1, 0, 1), (1, 2, 0, 1), (0, 1, 2, 1), (0, 2, 1, 1), (1, 1, 2, 0), (1, 2, 1, 0), (2, 1, 1, 0), (1, 0, 2, 1), (2, 0, 1, 1), (1, 1, 0, 2)])
Basically, make a set and keep adding to it. Better than making lists etc. that take too much memory..
Hope it helps the next person looking out :-) Comment out the set 'update' in the function to see the difference.

You can make a function that uses collections.Counter to get unique items and their counts from the given sequence, and uses itertools.combinations to pick combinations of indices for each unique item in each recursive call, and map the indices back to a list when all indices are picked:
from collections import Counter
from itertools import combinations
def unique_permutations(seq):
def index_permutations(counts, index_pool):
if not counts:
yield {}
return
(item, count), *rest = counts.items()
rest = dict(rest)
for indices in combinations(index_pool, count):
mapping = dict.fromkeys(indices, item)
for others in index_permutations(rest, index_pool.difference(indices)):
yield {**mapping, **others}
indices = set(range(len(seq)))
for mapping in index_permutations(Counter(seq), indices):
yield [mapping[i] for i in indices]
so that [''.join(i) for i in unique_permutations('moon')] returns:
['moon', 'mono', 'mnoo', 'omon', 'omno', 'nmoo', 'oomn', 'onmo', 'nomo', 'oonm', 'onom', 'noom']

This is my attempt without resorting to set / dict, as a generator using recursion, but using string as input. Output is also ordered in natural order:
def perm_helper(head: str, tail: str):
if len(tail) == 0:
yield head
else:
last_c = None
for index, c in enumerate(tail):
if last_c != c:
last_c = c
yield from perm_helper(
head + c, tail[:index] + tail[index + 1:]
)
def perm_generator(word):
yield from perm_helper("", sorted(word))
example:
from itertools import takewhile
word = "POOL"
list(takewhile(lambda w: w != word, (x for x in perm_generator(word))))
# output
# ['LOOP', 'LOPO', 'LPOO', 'OLOP', 'OLPO', 'OOLP', 'OOPL', 'OPLO', 'OPOL', 'PLOO', 'POLO']

ans=[]
def fn(a, size):
if (size == 1):
if a.copy() not in ans:
ans.append(a.copy())
return
for i in range(size):
fn(a,size-1);
if size&1:
a[0], a[size-1] = a[size-1],a[0]
else:
a[i], a[size-1] = a[size-1],a[i]
https://www.geeksforgeeks.org/heaps-algorithm-for-generating-permutations/

Came across this the other day while working on a problem of my own. I like Luka Rahne's approach, but I thought that using the Counter class in the collections library seemed like a modest improvement. Here's my code:
def unique_permutations(elements):
"Returns a list of lists; each sublist is a unique permutations of elements."
ctr = collections.Counter(elements)
# Base case with one element: just return the element
if len(ctr.keys())==1 and ctr[ctr.keys()[0]] == 1:
return [[ctr.keys()[0]]]
perms = []
# For each counter key, find the unique permutations of the set with
# one member of that key removed, and append the key to the front of
# each of those permutations.
for k in ctr.keys():
ctr_k = ctr.copy()
ctr_k[k] -= 1
if ctr_k[k]==0:
ctr_k.pop(k)
perms_k = [[k] + p for p in unique_permutations(ctr_k)]
perms.extend(perms_k)
return perms
This code returns each permutation as a list. If you feed it a string, it'll give you a list of permutations where each one is a list of characters. If you want the output as a list of strings instead (for example, if you're a terrible person and you want to abuse my code to help you cheat in Scrabble), just do the following:
[''.join(perm) for perm in unique_permutations('abunchofletters')]

I came up with a very suitable implementation using itertools.product in this case (this is an implementation where you want all combinations
unique_perm_list = [''.join(p) for p in itertools.product(['0', '1'], repeat = X) if ''.join(p).count() == somenumber]
this is essentially a combination (n over k) with n = X and somenumber = k
itertools.product() iterates from k = 0 to k = X subsequent filtering with count ensures that just the permutations with the right number of ones are cast into a list. you can easily see that it works when you calculate n over k and compare it to the len(unique_perm_list)

Adapted to remove recursion, use a dictionary and numba for high performance but not using yield/generator style so memory usage is not limited:
import numba
#numba.njit
def perm_unique_fast(elements): #memory usage too high for large permutations
eset = set(elements)
dictunique = dict()
for i in eset: dictunique[i] = elements.count(i)
result_list = numba.typed.List()
u = len(elements)
for _ in range(u): result_list.append(0)
s = numba.typed.List()
results = numba.typed.List()
d = u
while True:
if d > 0:
for i in dictunique:
if dictunique[i] > 0: s.append((i, d - 1))
i, d = s.pop()
if d == -1:
dictunique[i] += 1
if len(s) == 0: break
continue
result_list[d] = i
if d == 0: results.append(result_list[:])
dictunique[i] -= 1
s.append((i, -1))
return results
import timeit
l = [2, 2, 3, 3, 4, 4, 5, 5, 6, 6]
%timeit list(perm_unique(l))
#377 ms ± 26 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
ltyp = numba.typed.List()
for x in l: ltyp.append(x)
%timeit perm_unique_fast(ltyp)
#293 ms ± 3.37 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
assert list(sorted(perm_unique(l))) == list(sorted([tuple(x) for x in perm_unique_fast(ltyp)]))
About 30% faster but still suffers a bit due to list copying and management.
Alternatively without numba but still without recursion and using a generator to avoid memory issues:
def perm_unique_fast_gen(elements):
eset = set(elements)
dictunique = dict()
for i in eset: dictunique[i] = elements.count(i)
result_list = list() #numba.typed.List()
u = len(elements)
for _ in range(u): result_list.append(0)
s = list()
d = u
while True:
if d > 0:
for i in dictunique:
if dictunique[i] > 0: s.append((i, d - 1))
i, d = s.pop()
if d == -1:
dictunique[i] += 1
if len(s) == 0: break
continue
result_list[d] = i
if d == 0: yield result_list
dictunique[i] -= 1
s.append((i, -1))

May be we can use set here to obtain unique permutations
import itertools
print('unique perms >> ', set(itertools.permutations(A)))

What about
np.unique(itertools.permutations([1, 1, 1]))
The problem is the permutations are now rows of a Numpy array, thus using more memory, but you can cycle through them as before
perms = np.unique(itertools.permutations([1, 1, 1]))
for p in perms:
print p

Making number from elements of list

How can i check that can i create a number from elements of list?
For example:
list=[1,1,3,3,3,3,5,10,23,53]
And now we can make 9 from [1,3,5] or [3,3,3]
i tried something like that:
list=[1,1,3,3,3,3,5,10,23,53]
tmp=[]
sum=0
for i in range(len(list)):
tmpChange=9
tmpChange -= list[i]+sum
if tmpChange == 0:
break
elif tmpChange > 0:
tmp.append(list[i])
sum += list[i]
print(tmpChange)
print(tmp)
else:
tmp.pop(i)

A naive way to approach this is to find all of the subsets of your original list, which you can do using itertools.combinations. Then you can check if the subset sums to your original value, then add them to a set.
import itertools
l = [1,1,3,3,3,3,5,10,23,53]
total = 9
values = set()
for r in range(1, len(l)):
for c in itertools.combinations(l, r):
if sum(c) == total:
values.add(tuple(c))
The result is then
>>> values
{(1, 3, 5), (3, 3, 3)}
As another example using the following data
l = [1,1,3,3,3,3,4,5,9,10,23,53]
The result would be
>>> values
{(4, 5), (3, 3, 3), (1, 1, 3, 4), (1, 3, 5), (9,)}

Python: Generating All Pairwise-Unique Pairings

I am looking for a Pythonic method to generate all pairwise-unique unique pairings (where a pairing is a system consisting of pairs, and pairwise-unique indicates that (a,b) ≠ (b,a)) for a set containing even number n items.
I like the code from here:
for perm in itertools.permutations(range(n)):
print zip(perm[::2], perm[1::2])
except that it generates all order-unique, pairwise-unique pairings, or (n/2)! times more pairings than I want (redundancy), which, although I can filter out, really bog down my program at large n.
That is, for n = 4, I am looking for the following output (12 unique pairings):
[(0, 1), (2, 3)]
[(0, 1), (3, 2)]
[(1, 0), (2, 3)]
[(1, 0), (3, 2)]
[(1, 2), (0, 3)]
[(1, 2), (3, 0)]
[(1, 3), (0, 2)]
[(2, 0), (1, 3)]
[(2, 0), (3, 1)]
[(3, 1), (0, 2)]
[(0, 3), (2, 1)]
[(3, 0), (2, 1)]
Note that (a,b) ≠ (b,a).
Is this possible? I am also okay with a function that generates the 3 non–pairwise-unique pairings for n = 4 where (a,b) = (b,a), as it is straightforward to permute what I need from there. My main goal is to avoid the superfluous permutations on the order of the pairs in the pairing.
Thanks in advance for your help and suggestions—I really appreciate it.

I think this gives you the fundamental pairings that you need: 1 when N=2; 3 when N=4; 15 when N=6; 105 when n=8, etc.
import sys
def pairings(remainder, partial = None):
partial = partial or []
if len(remainder) == 0:
yield partial
else:
for i in xrange(1, len(remainder)):
pair = [[remainder[0], remainder[i]]]
r1 = remainder[1:i]
r2 = remainder[i+1:]
for p in pairings(r1 + r2, partial + pair):
yield p
def main():
n = int(sys.argv[1])
items = list(range(n))
for p in pairings(items):
print p
main()

In the linked question "Generating all unique pair permutations", (here), an algorithm is given to generate a round-robin schedule for any given n. That is, each possible set of matchups/pairings for n teams.
So for n = 4 (assuming exclusive), that would be:
[0, 3], [1, 2]
[0, 2], [3, 1]
[0, 1], [2, 3]
Now we've got each of these partitions, we just need to find their permutations in order to get the full list of pairings. i.e [0, 3], [1, 2] is a member of a group of four: [0, 3], [1, 2] (itself) and [3, 0], [1, 2] and [0, 3], [2, 1] and [3, 0], [2, 1].
To get all the members of a group from one member, you take the permutation where each pair can be either flipped or not flipped (if they were, for example, n-tuples instead of pairs, then there would be n! options for each one). So because you have two pairs and options, each partition yields 2 ^ 2 pairings. So you have 12 altogether.
Code to do this, where round_robin(n) returns a list of lists of pairs. So round_robin(4) --> [[[0, 3], [1, 2]], [[0, 2], [3, 1]], [[0, 1], [2, 3]]].
def pairs(n):
for elem in round_robin(n):
for first in [elem[0], elem[0][::-1]]:
for second in [elem[1], elem[1][::-1]]:
print (first, second)
This method generates less than you want and then goes up instead of generating more than you want and getting rid of a bunch, so it should be more efficient. ([::-1] is voodoo for reversing a list immutably).
And here's the round-robin algorithm from the other posting (written by Theodros Zelleke)
from collections import deque
def round_robin_even(d, n):
for i in range(n - 1):
yield [[d[j], d[-j-1]] for j in range(n/2)]
d[0], d[-1] = d[-1], d[0]
d.rotate()
def round_robin_odd(d, n):
for i in range(n):
yield [[d[j], d[-j-1]] for j in range(n/2)]
d.rotate()
def round_robin(n):
d = deque(range(n))
if n % 2 == 0:
return list(round_robin_even(d, n))
else:
return list(round_robin_odd(d, n))

I'm not sure if I understood the problem well, anyway here is my solution:
import itertools
n = 4
out = set()
for perm in itertools.permutations(range(n)):
pairs = tuple(sorted(zip(perm[::2], perm[1::2])))
out.add(pairs)
for i, p in enumerate(sorted(out)):
print i,p
it returns 12 unique pairs for n=4, and 120 for n=6.