Develop Reference

Numba save permutations with duplicates

I have a function that gets called extremely often and so to speed it up i want to use numbas #njit decorator. However in this function i need to calculate the permutations of an array and numba does not play nice with itertools.
I found this for a numba save version to produce permutations however this implementation does not deal with duplicates in the input in the way i need it to.
array1 = [9,9,21]
def permutations(A, k):
r = [[i for i in range(0)]]
for i in range(k):
r = [[a] + b for a in A for b in r if (a in b)==False]
return r
print(permutations(array1,3))
print(list(itertools.permutations(array1,3)))
[]
[(9, 9, 21), (9, 21, 9), (9, 9, 21), (9, 21, 9), (21, 9, 9), (21, 9, 9)]
What i want is the second result, not the first

I've created your "ideal world" permutations function, it recursively sends one permutation of the original list with one member short.
However, don't expect as fast results as in itertools.
array1 = [9, 9, 21]
array2 = [1, 2, 3]
array3 = [1, 2, 3, 4]
def permutations(A):
r = []
for i in range(len(A)):
a, b = A[i], A[: i] + A[i + 1:]
if b:
for c in permutations(b):
if [a] + c in r:
continue
r.append([a] + c)
else:
r.append([a])
return r
print(permutations(array1))
print(permutations(array2))
print(permutations(array3))
OUTPUT:
[[9, 9, 21], [9, 21, 9], [21, 9, 9]]
[[1, 2, 3], [1, 3, 2], [2, 1, 3], [2, 3, 1], [3, 1, 2], [3, 2, 1]]
[[1, 2, 3, 4], [1, 2, 4, 3], [1, 3, 2, 4], [1, 3, 4, 2], [1, 4, 2, 3], [1, 4, 3, 2],
[2, 1, 3, 4], [2, 1, 4, 3], [2, 3, 1, 4], [2, 3, 4, 1], [2, 4, 1, 3], [2, 4, 3, 1],
[3, 1, 2, 4], [3, 1, 4, 2], [3, 2, 1, 4], [3, 2, 4, 1], [3, 4, 1, 2], [3, 4, 2, 1],
[4, 1, 2, 3], [4, 1, 3, 2], [4, 2, 1, 3], [4, 2, 3, 1], [4, 3, 1, 2], [4, 3, 2, 1]]

How to create a list from tuples with sub-tuples?

I have two lists a and b. Then I create all combinations taking all two-length combinations of a plus one each of b:
import itertools as it
a = [1,2,3,4]
b = [5,6]
for i in it.product(it.combinations(a, 2), b):
print (i)
# output:
((1, 2), 5)
((1, 2), 6)
((1, 3), 5)
...
# expected output:
[1, 2, 5]
[1, 2, 6]
[1, 3, 5]
...
How can the tuples be transformed at the stage of the loop operation into lists?

The following comprehensions will work:
>>> [[*x, y] for x, y in it.product(it.combinations(a, 2), b)] # Py3
>>> [list(x) + [y] for x, y in it.product(it.combinations(a, 2), b)] # all Py versions
[[1, 2, 5],
[1, 2, 6],
[1, 3, 5],
[1, 3, 6],
[1, 4, 5],
[1, 4, 6],
[2, 3, 5],
[2, 3, 6],
[2, 4, 5],
[2, 4, 6],
[3, 4, 5],
[3, 4, 6]]

Simplified approach:
a = [1,2,3,4]
b = [5,6]
l = len(a)
print(sorted([a[i], a[i_n], j] for i in range(l) for j in b
for i_n in range(i+1, l) if i < l-1))
The output:
[[1, 2, 5], [1, 2, 6], [1, 3, 5], [1, 3, 6], [1, 4, 5], [1, 4, 6], [2, 3, 5], [2, 3, 6], [2, 4, 5], [2, 4, 6], [3, 4, 5], [3, 4, 6]]

Identify vectors with same value in one column with numpy in python

I have a large 2d array of vectors. I want to split this array into several arrays according to one of the vectors' elements or dimensions. I would like to receive one such small array if the values along this column are consecutively identical. For example considering the third dimension or column:
orig = np.array([[1, 2, 3],
[3, 4, 3],
[5, 6, 4],
[7, 8, 4],
[9, 0, 4],
[8, 7, 3],
[6, 5, 3]])
I want to turn into three arrays consisting of rows 1,2 and 3,4,5 and 6,7:
>>> a
array([[1, 2, 3],
[3, 4, 3]])
>>> b
array([[5, 6, 4],
[7, 8, 4],
[9, 0, 4]])
>>> c
array([[8, 7, 3],
[6, 5, 3]])
I'm new to python and numpy. Any help would be greatly appreciated.
Regards
Mat
Edit: I reformatted the arrays to clarify the problem

Using np.split:
>>> a, b, c = np.split(orig, np.where(orig[:-1, 2] != orig[1:, 2])[0]+1)
>>> a
array([[1, 2, 3],
[1, 2, 3]])
>>> b
array([[1, 2, 4],
[1, 2, 4],
[1, 2, 4]])
>>> c
array([[1, 2, 3],
[1, 2, 3]])

Nothing fancy here, but this good old-fashioned loop should do the trick
import numpy as np
a = np.array([[1, 2, 3],
[1, 2, 3],
[1, 2, 4],
[1, 2, 4],
[1, 2, 4],
[1, 2, 3],
[1, 2, 3]])
groups = []
rows = a[0]
prev = a[0][-1] # here i assume that the grouping is based on the last column, change the index accordingly if that is not the case.
for row in a[1:]:
if row[-1] == prev:
rows = np.vstack((rows, row))
else:
groups.append(rows)
rows = [row]
prev = row[-1]
groups.append(rows)
print groups
## [array([[1, 2, 3],
## [1, 2, 3]]),
## array([[1, 2, 4],
## [1, 2, 4],
## [1, 2, 4]]),
## array([[1, 2, 3],
## [1, 2, 3]])]

if a looks like this:
array([[1, 1, 2, 3],
[2, 1, 2, 3],
[3, 1, 2, 4],
[4, 1, 2, 4],
[5, 1, 2, 4],
[6, 1, 2, 3],
[7, 1, 2, 3]])
than this
col = a[:, -1]
indices = np.where(col[:-1] != col[1:])[0] + 1
indices = np.concatenate(([0], indices, [len(a)]))
res = [a[start:end] for start, end in zip(indices[:-1], indices[1:])]
print(res)
results in:
[array([[1, 2, 3],
[1, 2, 3]]), array([[1, 2, 4],
[1, 2, 4],
[1, 2, 4]]), array([[1, 2, 3],
[1, 2, 3]])]
Update: np.split() is much nicer. No need to add first and last index:
col = a[:, -1]
indices = np.where(col[:-1] != col[1:])[0] + 1
res = np.split(a, indices)

How to make a set of lists

I have a list of lists like this:
i = [[1, 2, 3], [2, 4, 5], [1, 2, 3], [2, 4, 5]]
I would like to get a list containing "unique" lists (based on their elements) like:
o = [[1, 2, 3], [2, 4, 5]]
I cannot use set() as there are non-hashable elements in the list. Instead, I am doing this:
o = []
for e in i:
if e not in o:
o.append(e)
Is there an easier way to do this?

You can create a set of tuples, a set of lists will not be possible because of non hashable elements as you mentioned.
>>> l = [[1, 2, 3], [2, 4, 5], [1, 2, 3], [2, 4, 5]]
>>> set(tuple(i) for i in l)
{(1, 2, 3), (2, 4, 5)}

i = [[1, 2, 3], [2, 4, 5], [1, 2, 3], [2, 4, 5]]
print([ele for ind, ele in enumerate(i) if ele not in i[:ind]])
[[1, 2, 3], [2, 4, 5]]
If you consider [2, 4, 5] to be equal to [2, 5, 4] then you will need to do further checks

You can convert each element to a tuple and then insert it in a set.
Here's some code with your example:
tmp = set()
a = [[1, 2, 3], [2, 4, 5], [1, 2, 3], [2, 4, 5]]
for i in a:
tmp.add(tuple(i))
tmp will be like this:
{(1, 2, 3), (2, 4, 5)}

Here's another way to do it:
I = [[1, 2, 3], [2, 4, 5], [1, 2, 3], [2, 4, 5]]
mySet = set()
for j in range(len(I)):
mySet = mySet | set([tuple(I[j])])
print(mySet)

numpy searchsorted lexicographic on slice

Suppose I have a numpy array that is already sorted.
>>> x
array([[1, 2, 3, 1],
[2, 2, 3, 2],
[2, 3, 3, 4],
[3, 4, 4, 4],
[4, 5, 5, 5]])
I want to do a righthand bisection based on the first two columns.
Something like (quasicode) numpy.searchsorted(x, [2, 2], side='right') that should return 2, the index between [2, 2, 3, 2] and [2, 3, 3, 4].

How about this:
>>> i1 = np.searchsorted(x[:, 0], 2, side='l')
>>> i2 = np.searchsorted(x[i1:, 0], 2, side='r')
>>> i3 = np.searchsorted(x[i1:i1+i2, 1], 2, side='r')
>>> i1 + i3
2