I have a problem I've been trying to think through. Say I have a numpy array that looks like this (in the actual implementation, len(array) will be around 4500):
array = np.repeat([0, 1, 2], 2)
array >> [0, 0, 1, 1, 2, 2]
From this, I'm trying to generate a new (shuffled) array where the proportion of values that randomly agree with array is a particular proportion p. So let's say p = .5. Then, an example new array would be something like
array = [0, 0, 1, 1, 2, 2]
new_array = [0, 1, 2, 1, 0, 2]
where you can see that exactly 50% of the values in new_array agree with the values in array. The requirements of the output array are:
np.count_nonzero(array - new_array) / len(array) = p, and
set(np.unique(array)) == set(np.unique(new_array)).
By "agree" I mean array[i] == new_array[i] for agreeing indices i. All values in new_array should be the same as array, just shuffled.
I'm sure there's an elegant way of doing this -- can anybody think of something?
Thanks!
You can try something like
import random
p = 0.5
arr = np.array([0, 0, 1, 1, 2, 2])
# number of similar elements required
num_sim_element = round(len(arr)*p)
# creating indeices of similar element
hp = {}
for i,e in enumerate(arr):
if(e in hp):
hp[e].append(i)
else:
hp[e] = [i]
#print(hp)
out_map = []
k = list(hp.keys())
v = list(hp.values())
index = 0
while(len(out_map) != num_sim_element):
if(len(v[index]) > 0):
k_ = k[index]
random.shuffle(v[index])
v_ = v[index].pop()
out_map.append((k_,v_))
index += 1
index %= len(k)
#print(out_map)
out_unique = set([i[0] for i in out_map])
out_indices = [i[-1] for i in out_map]
out_arr = arr.copy()
#for i in out_map:
# out_arr[i[-1]] = i[0]
for i in set(range(len(arr))).difference(out_indices):
out_arr[i] = random.choice(list(out_unique.difference([out_arr[i]])))
print(arr)
print(out_arr)
assert 1 - (np.count_nonzero(arr - out_arr) / len(arr)) == p
assert set(np.unique(arr)) == set(np.unique(out_arr))
[0 0 1 1 2 2]
[1 0 1 0 0 2]
Here's a version that might be a little easier to follow:
import math, random
# generate array of random values
a = np.random.rand(4500)
# make a utility list of every position in that array, and shuffle it
indices = [i for i in range(0, len(a))]
random.shuffle(indices)
# set the proportion you want to keep the same
proportion = 0.5
# make two lists of indices, the ones that stay the same and the ones that get shuffled
anchors = indices[0:math.floor(len(a)*proportion)]
not_anchors = indices[math.floor(len(a)*proportion):]
# get values of non-anchor indices, and shuffle them
not_anchor_values = [a[i] for i in not_anchors]
random.shuffle(not_anchor_values)
# loop original array, if an anchor position, keep original value
# if not an anchor, draw value from shuffle non-anchor value list and increment the count
final_list = []
count = 0
for e,i in enumerate(a):
if e in not_anchors:
final_list.append(i)
else:
final_list.append(not_anchor_values[count])
count +=1
# test proportion of matches and non-matches in output
match = []
not_match = []
for e,i in enumerate(a):
if i == final_list[e]:
match.append(True)
else:
not_match.append(True)
len(match)/(len(match)+len(not_match))
Comments in the code explain the approach.
(EDITED to include a different and more accurate approach)
One should note that not all values of the shuffled fraction p (number of shuffled elements divided by the total number of elements) is accessible.
The possible value of p depend on the size of the input and on the number of repeated elements.
That said, I can suggest two possible approaches:
split your input into pinned and unpinned indices of the correct size and then shuffle the unpinned indices.
import numpy as np
def partial_shuffle(arr, p=1.0):
n = arr.size
k = round(n * p)
shuffling = np.arange(n)
shuffled = np.random.choice(n, k, replace=False)
shuffling[shuffled] = np.sort(shuffled)
return arr[shuffling]
The main advantage of approach (1) is that it can be easily implemented in a vectorized form using np.random.choice() and advanced indexing.
On the other hand, this works well as long as you are willing to accept that some shuffling may return you some elements unshuffled because of repeating values or simply because the shuffling indexes are accidentally coinciding with the unshuffled ones.
This causes the requested value of p to be typically larger than the actual value observed.
If one needs a relatively more accurate value of p, one could just try performing a search on the p parameter giving the desired value on the output, or go by trial-and-error.
implement a variation of the Fisher-Yates shuffle where you: (a) reject swappings of positions whose value is identical and (b) pick only random positions to swap that were not already visited.
def partial_shuffle_eff(arr, p=1.0, inplace=False, tries=2.0):
if not inplace:
arr = arr.copy()
n = arr.size
k = round(n * p)
tries = round(n * tries)
seen = set()
i = l = t = 0
while i < n and l < k:
seen.add(i)
j = np.random.randint(i, n)
while j in seen and t < tries:
j = np.random.randint(i, n)
t += 1
if arr[i] != arr[j]:
arr[i], arr[j] = arr[j], arr[i]
l += 2
seen.add(j)
while i in seen:
i += 1
return arr
While this approach gets to a more accurate value of p, it is still limited by the fact that the target number of swaps must be even.
Also, for inputs with lots of uniques the second while (while j in seen ...) is potentially an infinite loop so a cap on the number of tries should be set.
Finally, you would need to go with explicit looping, resulting in a much slower execution speed, unless you can use Numba's JIT compilation, which would speed up your execution significantly.
import numba as nb
partial_shuffle_eff_nb = nb.njit(partial_shuffle_eff)
partial_shuffle_eff_nb.__name__ = 'partial_shuffle_eff_nb'
To test the accuracy of the partial shuffling we may use the (percent) Hamming distance:
def hamming_distance(a, b):
assert(a.shape == b.shape)
return np.count_nonzero(a == b)
def percent_hamming_distance(a, b):
return hamming_distance(a, b) / len(a)
def shuffling_fraction(a, b):
return 1 - percent_hamming_distance(a, b)
And we may observe a behavior similar to this:
funcs = (
partial_shuffle,
partial_shuffle_eff,
partial_shuffle_eff_nb
)
n = 12
m = 3
arrs = (
np.zeros(n, dtype=int),
np.arange(n),
np.repeat(np.arange(m), n // m),
np.repeat(np.arange(3), 2),
np.repeat(np.arange(3), 3),
)
np.random.seed(0)
for arr in arrs:
print(" " * 24, arr)
for func in funcs:
shuffled = func(arr, 0.5)
print(f"{func.__name__:>24s}", shuffled, shuffling_fraction(arr, shuffled))
# [0 0 0 0 0 0 0 0 0 0 0 0]
# partial_shuffle [0 0 0 0 0 0 0 0 0 0 0 0] 0.0
# partial_shuffle_eff [0 0 0 0 0 0 0 0 0 0 0 0] 0.0
# partial_shuffle_eff_nb [0 0 0 0 0 0 0 0 0 0 0 0] 0.0
# [ 0 1 2 3 4 5 6 7 8 9 10 11]
# partial_shuffle [ 0 8 2 3 6 5 7 4 9 1 10 11] 0.5
# partial_shuffle_eff [ 3 8 11 0 4 5 6 7 1 9 10 2] 0.5
# partial_shuffle_eff_nb [ 9 10 11 3 4 5 6 7 8 0 1 2] 0.5
# [0 0 0 0 1 1 1 1 2 2 2 2]
# partial_shuffle [0 0 2 0 1 2 1 1 2 2 1 0] 0.33333333333333337
# partial_shuffle_eff [1 1 1 0 0 1 0 0 2 2 2 2] 0.5
# partial_shuffle_eff_nb [1 2 1 0 1 0 0 1 0 2 2 2] 0.5
# [0 0 1 1 2 2]
# partial_shuffle [0 0 1 1 2 2] 0.0
# partial_shuffle_eff [1 1 0 0 2 2] 0.6666666666666667
# partial_shuffle_eff_nb [1 2 0 1 0 2] 0.6666666666666667
# [0 0 0 1 1 1 2 2 2]
# partial_shuffle [0 0 1 1 0 1 2 2 2] 0.2222222222222222
# partial_shuffle_eff [0 1 2 1 0 1 2 2 0] 0.4444444444444444
# partial_shuffle_eff_nb [0 0 1 0 2 1 2 1 2] 0.4444444444444444
or, for an input closer to your use-case:
n = 4500
m = 3
arr = np.repeat(np.arange(m), n // m)
np.random.seed(0)
for func in funcs:
shuffled = func(arr, 0.5)
print(f"{func.__name__:>24s}", shuffling_fraction(arr, shuffled))
# partial_shuffle 0.33777777777777773
# partial_shuffle_eff 0.5
# partial_shuffle_eff_nb 0.5
Finally some small benchmarking:
n = 4500
m = 3
arr = np.repeat(np.arange(m), n // m)
np.random.seed(0)
for func in funcs:
print(f"{func.__name__:>24s}", end=" ")
%timeit func(arr, 0.5)
# partial_shuffle 213 µs ± 6.36 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
# partial_shuffle_eff 10.9 ms ± 194 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
# partial_shuffle_eff_nb 172 µs ± 1.79 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
Related
I want to create a 64 components array showing all the squares in which the two rooks of an empty chessboard could move from their current position. So far I am doing it with for and while loops.
I first create a function just to better visualize the board:
import numpy as np
def from_array_to_matrix(v):
m=np.zeros((8,8)).astype('int')
for row in range(8):
for column in range(8):
m[row,column]=v[row*8+column]
return m
and here I show how I actually build the array:
# positions of the two rooks
a=np.zeros(64).astype('int')
a[15] = 1
a[25] = 1
print from_array_to_matrix(a)
# attack_a will be all the squares where they could move in the empty board
attack_a=np.zeros(64).astype('int')
for piece in np.where(a)[0]:
j=0
square=piece+j*8
while square<64:
attack_a[square]=1
j+=1
square=piece+j*8
j=0
square=piece-j*8
while square>=0:
attack_a[square]=1
j+=1
square=piece-j*8
j=0
square=piece+j
while square<8*(1+piece//8):
attack_a[square]=1
j+=1
square=piece+j
j=0
square=piece-j
while square>=8*(piece//8):
attack_a[square]=1
j+=1
square=piece-j
print attack_a
print from_array_to_matrix(attack_a)
I have been advised to avoid for and while loops whenever it is possible to use other ways, because they tend to be time consuming. Is there any way to achieve the same result without iterating the process with for and while loops ?
Perhaps using the fact that the indices to which I want to assign the value 1 can be determined by a function.
There are a couple of different ways to do this. The simplest thing is of course to work with matrices.
But you can vectorize operations on the raveled array as well. For example, say you had a rook at position 0 <= n < 64 in the linear array. To set the row to one, use integer division:
array[8 * (n // 8):8 * (n // 8 + 1)] = True
To set the column, use modulo:
array[n % 8::8] = True
You can convert to a matrix using reshape:
matrix = array.reshape(8, 8)
And back using ravel:
array = martix.ravel()
Or reshape:
array = matrix.reshape(-1)
Setting ones in a matrix is even simpler, given a specific row 0 <= m < 8 and column 0 <= n < 8:
matrix[m, :] = matrix[:, n] = True
Now the only question is how to vectorize multiple indices simultaneously. As it happens, you can use a fancy index in one axis. I.e, the expression above can be used with an m and n containing multiple elements:
m, n = np.nonzero(matrix)
matrix[m, :] = matrix[:, n] = True
You could even play games and do this with the array, also using fancy indexing:
n = np.nonzero(array)[0]
r = np.linspace(8 * (n // 8), 8 * (n // 8 + 1), 8, False).T.ravel()
c = np.linspace(n % 8, n % 8 + 64, 8, False)
array[r] = array[c] = True
Using linspace allows you to generate multiple sequences of the same size simultaneously. Each sequence is a column, so we transpose before raveling, although this is not required.
Use reshaping to convert 1-D array to 8x8 2-D matrix and then numpy advance indexing to select rows and columns to set to 1:
import numpy as np
def from_array_to_matrix(v):
return v.reshape(8,8)
# positions of the two rooks
a=np.zeros(64).astype('int')
a[15] = 1
a[25] = 1
a = from_array_to_matrix(a)
# attack_a will be all the squares where they could move in the empty board
attack_a=np.zeros(64).astype('int')
attack_a = from_array_to_matrix(attack_a)
#these two lines replace your for and while loops
attack_a[np.where(a)[0],:] = 1
attack_a[:,np.where(a)[1]] = 1
output:
a:
[[0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1]
[0 0 0 0 0 0 0 0]
[0 1 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0]]
attack_a:
[[0 1 0 0 0 0 0 1]
[1 1 1 1 1 1 1 1]
[0 1 0 0 0 0 0 1]
[1 1 1 1 1 1 1 1]
[0 1 0 0 0 0 0 1]
[0 1 0 0 0 0 0 1]
[0 1 0 0 0 0 0 1]
[0 1 0 0 0 0 0 1]]
I have to handle a huge amount of data. Every row starts with 1 or 0. I need a dataframe where every rows start with 1, so I have to step left all rows values till the first value is 1.
For example:
0 1 0 0 1 0 0
1 0 0 0 0 1 1
0 0 0 1 0 0 1
0 0 0 0 0 1 1
The result has to be this:
1 0 0 1 0 0 0
1 0 0 0 0 1 1
1 0 0 1 0 0 0
1 1 0 0 0 0 0
I don't want to use for, while, etc., because I need some faster methods with pandas or numpy.
Do you have idea for this problem?
You may using with cummax to mask all position need to shift as NaN and sorted
df[df.cummax(1).ne(0)].apply(lambda x : sorted(x,key=pd.isnull),1).fillna(0).astype(int)
Out[310]:
1 2 3 4 5 6 7
0 1 0 0 1 0 0 0
1 1 0 0 0 0 1 1
2 1 0 0 1 0 0 0
3 1 1 0 0 0 0 0
Or we using the function justify write by Divakar(much faster than the apply sorted)
pd.DataFrame(justify(df[df.cummax(1).ne(0)].values, invalid_val=np.nan, axis=1, side='left')).fillna(0).astype(int)
Out[314]:
0 1 2 3 4 5 6
0 1 0 0 1 0 0 0
1 1 0 0 0 0 1 1
2 1 0 0 1 0 0 0
3 1 1 0 0 0 0 0
You can make use of numpy.ogrid here:
a = df.values
s = a.argmax(1) * - 1
m, n = a.shape
r, c = np.ogrid[:m, :n]
s[s < 0] += n
c = c - s[:, None]
a[r, c]
array([[1, 0, 0, 1, 0, 0, 0],
[1, 0, 0, 0, 0, 1, 1],
[1, 0, 0, 1, 0, 0, 0],
[1, 1, 0, 0, 0, 0, 0]], dtype=int64)
Timings
In [35]: df = pd.DataFrame(np.random.randint(0, 2, (1000, 1000)))
In [36]: %timeit pd.DataFrame(justify(df[df.cummax(1).ne(0)].values, invalid_val=np.nan, axis=1, side='left')).fillna(0).a
...: stype(int)
116 ms ± 640 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [37]: %%timeit
...: a = df.values
...: s = a.argmax(1) * - 1
...: m, n = a.shape
...: r, c = np.ogrid[:m, :n]
...: s[s < 0] += n
...: c = c - s[:, None]
...: pd.DataFrame(a[r, c])
...:
...:
11.3 ms ± 18.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
For performance, you can use numba. An elementary loop, but effective given JIT-compilation and use of more basic objects at C-level:
from numba import njit
#njit
def shifter(A):
res = np.zeros(A.shape)
for i in range(res.shape[0]):
start, end = 0, 0
for j in range(res.shape[1]):
if A[i, j] != 0:
start = j
break
res[i, :res.shape[1]-start] = A[i, start:]
return res
Performance benchmarking
def jpp(df):
return pd.DataFrame(shifter(df.values).astype(int))
def user348(df):
a = df.values
s = a.argmax(1) * - 1
m, n = a.shape
r, c = np.ogrid[:m, :n]
s[s < 0] += n
c = c - s[:, None]
return pd.DataFrame(a[r, c])
np.random.seed(0)
df = pd.DataFrame(np.random.randint(0, 2, (1000, 1000)))
assert np.array_equal(jpp(df).values, user348(df).values)
%timeit jpp(df) # 9.2 ms per loop
%timeit user348(df) # 18.5 ms per loop
Here is a stride_tricks solution, which is fast because it enables slice-wise copying.
def pp(x):
n, m = x.shape
am = x.argmax(-1)
mam = am.max()
xx = np.empty((n, m + mam), x.dtype)
xx[:, :m] = x
xx[:, m:] = 0
xx = np.lib.stride_tricks.as_strided(xx, (n, mam+1, m), (*xx.strides, xx.strides[-1]))
return xx[np.arange(x.shape[0]), am]
It pads the input with the required number of zeros and then creates a sliding window view using as_strided. This is addressed using fancy indexing, but necause the last dimension is not indexed copying of lines is optimized and fast.
How fast? For large enough inputs on par with numba:
x = np.random.randint(0, 2, (10000, 10))
from timeit import timeit
shifter(x) # that should compile it, right?
print(timeit(lambda:shifter(x).astype(x.dtype), number=1000))
print(timeit(lambda:pp(x), number=1000))
Sample output:
0.8630472810036736
0.7336142909916816
I've tried to create an algorithm for finding all paths of length 2, but it doesn't seem to work properly:
input_split = input().split(' ')
node_count = int(input_split[0])
input_count = int(input_split[1])
items = np.zeros((node_count, node_count), dtype=np.int32) # matrix of adjacency
for j in range(input_count):
split = input().split(' ')
x = int(split[0]) - 1 # convert 1 based coordinates to 0 based
y = int(split[1]) - 1
items[x][y] = 1
items[y][x] = 1
result = np.linalg.matrix_power(items, 2)
result_sum = int(np.sum(result) / 2) # reverse paths are counted only once
print(result_sum)
Sample input:
6 7
1 2
2 3
3 1
2 4
4 5
5 6
6 2
The result should be 11, but it prints 18.
You're on the right track when calculating the square of the adjacency matrix. After the exponentiation you would get result matrix that looks like this:
[[2 1 1 1 0 1]
[1 4 1 0 2 0]
[1 1 2 1 0 1]
[1 0 1 2 0 2]
[0 2 0 0 2 0]
[1 0 1 2 0 2]]
First you need to exclude all diagonal entries from this matrix, because those denote walks that are not paths, as their starting and ending node is the same. Note that for length 2 that is the only way how nodes can be repeating.
The other entries need to be counted only once, because of symmetry. So only look at the upper right triangle of the matrix.
One way to do it is:
result_sum = 0
for i in range(input_count - 1):
for j in range(i + 1, input_count - 1):
result_sum += result[i][j]
print(result_sum) # prints 11
More Pythonic way, one-liner using numpy.trace():
result_sum = (np.sum(result) - np.trace(result)) // 2
You are calculating walks, which would include walks 6-7-6 (which is not a P2)
this discussion might help:
https://math.stackexchange.com/questions/1890620/finding-path-lengths-by-the-power-of-adjacency-matrix-of-an-undirected-graph
I am currently computing a function that contains a summation over an index. The index is between 0 and the integer part of T; ideally I would like to be able to compute this summation quickly for several values of T.
In a real-life case, most of the values of T are small, but a small percentage can be one or two orders of magnitude larger than the average.
What I am doing now is:
1) I define the vector T, e.g. (my real-life data have a much larger number of entries, it is just to give an idea):
import numpy as np
T = np.random.exponential(5, 10)
2) I create a matrix containing the factors between 0 and int(T), and then zeroes:
n = int(T.max())
j = ((np.arange(n) < T[:,np.newaxis])*np.arange(1,n+1)).astype(int).transpose()
print(j)
[[ 1 1 1 1 1 1 1 1 1 1]
[ 2 0 2 2 2 0 2 0 2 2]
[ 0 0 3 0 3 0 3 0 3 3]
[ 0 0 4 0 4 0 0 0 4 4]
[ 0 0 5 0 5 0 0 0 5 5]
[ 0 0 6 0 6 0 0 0 6 6]
[ 0 0 7 0 7 0 0 0 0 7]
[ 0 0 8 0 8 0 0 0 0 8]
[ 0 0 9 0 9 0 0 0 0 9]
[ 0 0 0 0 10 0 0 0 0 10]
[ 0 0 0 0 11 0 0 0 0 0]
[ 0 0 0 0 12 0 0 0 0 0]]
3) I generate the single elements of the summation, using a mask to avoid applying the function to the elements that are zero:
A = np.log(1 + (1 + j) * 5)* (j>0)
4) I sum along the columns:
A.sum(axis=0)
Obtaining:
array([ 5.170484 , 2.39789527, 29.96464821, 5.170484 ,
42.29052851, 2.39789527, 8.21500643, 2.39789527,
18.49060911, 33.9899999 ])
Is there a fastest/better way to vectorize that? I have the feeling that it is very slow due to the large amount of zeroes that do not contribute to the sum, but since I am a beginner with NumPy I couldn't figure out a better way of writing it.
EDIT: in my actual problem, the function applied to j depends also on a second parameter tau (in a vector of the same size of T). So the items contained in every column are not the same.
Looking at your j, for each column it has numbers going from 1 to N, where N is being decided based on each T element. Then, you are summing along each column, which is the same as summing until N because rest of the elements are zeros anyway. Those summed values could be calculated with np.cumsum and those N values that are basically the limits of each column in j could be directly calculated from T. These N values are then used as indices to index into the cumsum-ed values to give us the final output.
This should be pretty fast and memory efficient, given that cumsum is the only computation done and that too on a 1D array, as compared to the summation done in the original approach on a 2D array along each column. Thus, we have a vectorized approach like so -
n = int(T.max())
vals = (np.log(1 + (1 + np.arange(1,n+1)) * 5)).cumsum()
out = vals[(T.astype(int)).clip(max=n-1)]
In terms of memory usage, we are generating three variables -
n : Scalar
vals : 1D array of n elements
out : 1D array of T.size elements (this is the output anyway)
Runtime test and verify output -
In [5]: def original_app(T):
...: n = int(T.max())
...: j = ((np.arange(n) < T[:,None])*np.arange(1,n+1)).astype(int).transpose()
...: A = np.log(1 + (1 + j) * 5)* (j>0)
...: return A.sum(axis=0)
...:
...: def vectorized_app(T):
...: n = int(T.max())
...: vals = (np.log(1 + (1 + np.arange(1,n+1)) * 5)).cumsum()
...: return vals[(T.astype(int)).clip(max=n-1)]
...:
In [6]: # Input array
...: T = np.random.exponential(5, 10000)
In [7]: %timeit original_app(T)
100 loops, best of 3: 9.62 ms per loop
In [8]: %timeit vectorized_app(T)
10000 loops, best of 3: 50.1 µs per loop
In [9]: np.allclose(original_app(T),vectorized_app(T)) # Verify outputs
Out[9]: True
I'm looking for a code that speed up a creation of permutation matrix.
i.e., I want to create a matrix of n columns, where each column value iterates over m values, creating a n^m combinations on each row.
On the example below, there are 2 methods to create the matrix, in this case n=7 and m=5 that creates a matrix similar to:
1 1 1 1 1 1 1
1 1 1 1 1 1 2
1 1 1 1 1 1 3
1 1 1 1 1 1 4
1 1 1 1 1 1 5
1 1 1 1 1 2 1
...
5 5 5 5 5 5 5
The order of the rows doesn't matter, only that all combinations are created.
I have written the following 2 methods to create the arrays, but the metod1 is very slow (although very verbose and clear to understand) and method2 is faster using numpy functions. But I still need to find a faster methodology to create the matrix.
import numpy as np
############################################
def permArray_meth1():
allArray = np.zeros((1,7))
for a1 in range(1,6):
for a2 in range(1,6):
for a3 in range(1,6):
for a4 in range(1,6):
for a5 in range(1,6):
for a6 in range(1,6):
for a7 in range(1,6):
allArray = np.append(allArray, np.array([a1,a2,a3,a4,a5,a6,a7]).reshape(1,7), axis=0)
allArray = np.delete(allArray, 0, 0)
return allArray
############################################
def permArray_meth2():
##### Create permutation matrix #####
a = np.arange(np.power(5,7)).reshape(5,5,5,5,5,5,5)
allArray = [(a1,a2,a3,a4,a5,a6,a7) for a1,a2,a3,a4,a5,a6,a7 in np.ndindex(a.shape)]
##### Convert list to array #####
allArray = np.asarray(allArray)+1
return allArray
############################################
if __name__ == "__main__":
allArray = permArray_meth1() # (50sec)
print 'allArray1', np.shape(allArray)
allArray = permArray_meth2() # (1sec)
print 'allArray2', np.shape(allArray)
I know that the speed is dependent also on the used CPU hardware, but I'm looking for a relatively faster code thatn the shown above.
Is there any other method/code?
You could do this by creating an (n, m, m, ..., m) array of indices for column 1, column 2, ..., column n using np.indices(), then reshaping the output into an (n ** m, n) array:
import numpy as np
def permgrid(m, n):
inds = np.indices((m,) * n)
return inds.reshape(n, -1).T
For example:
print(permgrid(2, 3))
# [[0 0 0]
# [0 0 1]
# [0 1 0]
# [0 1 1]
# [1 0 0]
# [1 0 1]
# [1 1 0]
# [1 1 1]]