What is the most numerically efficient way to add all the combinations of two arrays of vectors? For example what I want is the following:
a = np.array([[1,2,3], [4,5,6]])
b = np.array([[7,8,9], [10,11,12]])
[ai + bj for ai in a for bj in b]
Gives
[array([ 8, 10, 12]),
array([11, 13, 15]),
array([11, 13, 15]),
array([14, 16, 18])]
It's a meshgrid with vectors instead of primary data types.
I've tried somewhat explicitly constructing the meshgrid results, which is faster than the list comprehension:
a_tile = np.tile(a, (2, 1))
array([[1, 2, 3],
[4, 5, 6],
[1, 2, 3],
[4, 5, 6]])
b_repeat = np.repeat(b, 2, axis=0)
array([[ 7, 8, 9],
[ 7, 8, 9],
[10, 11, 12],
[10, 11, 12]])
a_tile + b_repeat
array([[ 8, 10, 12],
[11, 13, 15],
[11, 13, 15],
[14, 16, 18]])
Is this as efficient as it gets? I was looking for a way to broadcast the arrays so that the grid isn't explicitly constructed.
You can try the following:
import numpy as np
a = np.array([[1,2,3], [4,5,6]])
b = np.array([[7,8,9], [10,11,12]])
(a[..., None] + b.T).transpose(0, 2, 1).reshape(-1, 3)
It gives:
array([[ 8, 10, 12],
[11, 13, 15],
[11, 13, 15],
[14, 16, 18]])
you can use numpy.broadcast_to to broadcast an array
N = 2 #number of repeats
your_req_array = np.broadcast_to(b.T, (N,b.shape[1], b.shape[0])).transpose(2,0,1).reshape(-1,b.shape[1]) + np.broadcast_to(a, (N,a.shape[0], a.shape[1])).reshape(-1,b.shape[1])
Related
I am generating multidimensional array of different sizes, though they'll all have an even number of columns.
>> import numpy as np
>> x = np.arange(24).reshape((3,8))
Which results in:
array([[ 0, 1, 2, 3, 4, 5, 6, 7],
[ 8, 9, 10, 11, 12, 13, 14, 15],
[16, 17, 18, 19, 20, 21, 22, 23]])
I am able to slice with numpy and get the first column in an array:
>> newarr = x[0:,0:2]
array([[ 0, 1],
[ 8, 9],
[16, 17]])
However, I want to have one array that is just a list of the columns where column 1 and 2 are together, 3 and 4 are together, and so on.. For example:
array([[[ 0, 1],
[ 8, 9],
[16, 17]],
[[ 2, 3],
[10, 11],
[18, 19]],
etc....]
)
This code below works but it's clunky and my arrays are not all the same. Some arrays have 16 columns, some have 34, some have 50, etc.
>> newarr = [x[0:,0:2]]+[x[0:,2:4]]+[x[0:,4:6]]
[array([[ 0, 1],
[ 8, 9],
[16, 17]]), array([[ 2, 3],
[10, 11],
[18, 19]])]
There's got to be a better way to do this than
newarr = [x[0:,0:2]]+[x[0:,2:4]]+[x[0:,4:6]]+...+[x[0:,n:n+2]]
Help!
My idea is adding a for loop:
slice_len = 2
x_list = [x[0:, slice_len*i:slice_len*(i+1)] for i in range(x.shape[1] // slice_len)]
Output:
[array([[ 0, 1],
[ 8, 9],
[16, 17]]), array([[ 2, 3],
[10, 11],
[18, 19]]), array([[ 4, 5],
[12, 13],
[20, 21]]), array([[ 6, 7],
[14, 15],
[22, 23]])]
I have a huge (N*20) matrix where every 5 rows is a valid sample, ie. every (5*20) matrix. I'm trying to reshape it into a (N/5,1,20,5) matrix where the dimension 20 is kept unchanged. I could do it in tensroflow using keep_dim, but how can I achieve this in numpy?
Thanks in advance.
Reshape and then swap the axes around:
arr1 = arr.reshape(N/5,5,1,20)
arr2 = arr1.transpose(0,2,3,1)
for example
In [476]: arr = np.arange(24).reshape(6,4)
In [477]: arr
Out[477]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]])
In [478]: arr1 = arr.reshape(2,3,1,4)
In [479]: arr2 = arr1.transpose(0,2,3,1)
In [480]: arr2.shape
Out[480]: (2, 1, 4, 3)
In [482]: arr2
Out[482]:
array([[[[ 0, 4, 8],
[ 1, 5, 9],
[ 2, 6, 10],
[ 3, 7, 11]]],
[[[12, 16, 20],
[13, 17, 21],
[14, 18, 22],
[15, 19, 23]]]])
I have a problem with printing a column in a numpy 3D Matrix.
Here is a simplified version of the problem:
import numpy as np
Matrix = np.zeros(((10,9,3))) # Creates a 10 x 9 x 3 3D matrix
Matrix[2][2][6] = 578
# I want to print Matrix[2][x][6] for x in range(9)
# the purpose of this is that I want to get all the Values in Matrix[2][x][6]
Much appreciated if you guys can help me out.
Thanks in advance.
Slicing would work:
a = np.zeros((10, 9, 3))
a[6, 2, 2] = 578
for x in a[6, :, 2]:
print(x)
Output:
0.0
0.0
578.0
0.0
0.0
0.0
0.0
0.0
0.0
Not sure if Numpy supports this, but you can do it with normal lists this way:
If you have three lists a = [1,2,3], b = [4,5,6], and c = [7,8,9], you can get the second dimension [2,5,8] for example by doing
list(zip(a,b,c))[1]
EDIT:
Turns out this is pretty simple in Numpy. According to this thread you can just do:
Matrix[:,1]
No sure if this is what you want. Here is demo:
In [1]: x = np.random.randint(0, 20, size=(4, 5, 3))
In [2]: x
Out[2]:
array([[[ 5, 13, 9],
[ 8, 16, 5],
[15, 17, 1],
[ 6, 14, 5],
[11, 13, 9]],
[[ 5, 8, 0],
[ 8, 15, 5],
[ 9, 2, 13],
[18, 4, 14],
[ 8, 3, 13]],
[[ 3, 7, 4],
[15, 11, 6],
[ 7, 8, 14],
[12, 8, 18],
[ 4, 2, 8]],
[[10, 1, 16],
[ 5, 2, 1],
[11, 12, 13],
[11, 9, 1],
[14, 5, 1]]])
In [4]: x[:, 2, :]
Out[4]:
array([[15, 17, 1],
[ 9, 2, 13],
[ 7, 8, 14],
[11, 12, 13]])
I have these 4 matrices and I want to dynamically combine them into one big matrix by passing n: number of small matrix and output matrix row and column
example:
[[[ 1 2]
[ 3 4]]
[[ 5 6]
[ 7 8]]
[[ 9 10]
[11 12]]
[[13 14]
[15 16]]]
the output matrix:
[[ 1 2 5 6]
[ 3 4 7 8]
[ 9 10 13 14]
[11 12 15 16]]
I can do it manually using:
M = np.bmat( [[x1], [x2], [x3], [x4]] )
I think (but dont know if its right), that its best to work inplace and avoid to create new objects with new methods each time - specifically when You are doing it in loop multiple times. These examples are only for 2d matrices. But it could be easilly implemented to more dimensions. Best would be to have one big array, if its really big, prolly in numpy.memmap array. Then work on its parts. Fastest indexing (second to pointers) would be on cython memoryviews...
import numpy as np
def combine_matrix(*args):
n=len(args)
rows,cols=args[0].shape
a=np.zeros((n,cols*rows))
m=0
for i in range(n/rows):
for j in range(n/cols):
a[i*rows:(i+1)*rows,j*cols:(j+1)*cols]=args[m]
m+=1
return a
def example1():
print '#'*10
a=np.arange(1,17)
n=4
rows,cols=n/2,n/2
lst=[]
for i in range(n):
ai=a[i*n:(i+1)*n]
ai.shape=rows,cols
lst.append(ai)
print lst
print combine_matrix(*lst)
def example2():
print '#'*10
m=24
a=np.arange(m)
n=6
rows,cols=m/n/2,n/2
lst=[]
for i in range(m/n):
ai=a[i*n:(i+1)*n]
ai.shape=rows,cols
lst.append(ai)
print lst
print combine_matrix(*lst)
def example3():
print '#'*10
m,n=36,6
a=np.arange(m)
arrs=np.array_split(a,n)
for i in range(n):
ln=arrs[i].shape[0]
arrs[i].shape=2,ln/2
print combine_matrix(*arrs)
example1()
example2()
example3()
2 minutes implementation (for question before edition, maybe usefull for someone):
import numpy as np
a=np.ones((10,10))
b=a*3
c=a*1
d=a*1.5
def combine_matrix(*args):
n=len(args)
rows,cols=args[0].shape
a=np.zeros((n,rows,cols))
for i in range(n):
a[i]=args[i]
return a
print combine_matrix(a,b,c,d)
If sizes of arrays are huge there is place for improvement...
You can combine transposition and reshape operations:
In [1878]: x=arange(24).reshape(4,3,2)
In [1879]: (_,n,m)=x.shape
In [1880]: x.reshape(2,2,n,m).transpose(0,2,1,3).reshape(2*n,2*m)
Out[1880]:
array([[ 0, 1, 6, 7],
[ 2, 3, 8, 9],
[ 4, 5, 10, 11],
[12, 13, 18, 19],
[14, 15, 20, 21],
[16, 17, 22, 23]])
[edit - I'm assuming that the small arrays are created independently, though my example is based on splitting a (4,2,2) array. If they really are just planes of a 3d array, then some combination of 'reshape' and 'transpose' will work better. But even such a solution will produce a copy because the original values are rearranged.]
Lets make a list of 2x2 arrays (here from a 3d array). Squeeze is needed because this split produces (1,2,2) arrays:
n = len(A)
E = np.zeros((n,n))
In [330]: X=np.arange(1,17).reshape(4,2,2)
In [331]: xl=[np.squeeze(i) for i in np.split(X,4,0)]
In [332]: xl
Out[332]:
[array([[1, 2],
[3, 4]]), array([[5, 6],
[7, 8]]), array([[ 9, 10],
[11, 12]]), array([[13, 14],
[15, 16]])]
Your bmat approach - corrected to produce the square arrangment
In [333]: np.bmat([[xl[0],xl[1]],[xl[2],xl[3]]])
Out[333]:
matrix([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
A concatenation approach:
In [334]: np.vstack([np.hstack(xl[:2]),np.hstack(xl[2:])])
Out[334]:
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
Since slicing works in hstack I could also use it in the bmat:
In [335]: np.bmat([xl[:2],xl[2:]])
Out[335]:
matrix([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
Internally bmat (check its code) is using a version of the vstack of hstacks (contactenates on first and last axes). Effectively
In [366]: ll=[xl[:2], xl[2:]]
In [367]: np.vstack([np.hstack(row) for row in ll])
Out[367]:
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
Some how you have to specify the arrangement of these n arrays. np.bmat(xl) produces a (2,8) matrix (so does hstack). np.vstack(xl) produces a (8,2) array.
It shouldn't be hard to extend this to work with a 3x3, 2x3, etc layout of subarrays. xl is a list of subarrays. Rework it into the desired list of lists of subarrays and apply bmat or the combination of stacks.
2 quick versions of 2x3 layout (a 4d xl array is easier to construct than a 2x3 nested list, but functionally will be the same:
In [369]: xl=np.arange(3*2*2*2).reshape((3,2,2,2))
In [370]: np.vstack([np.hstack(row) for row in xl])
Out[370]:
array([[ 0, 1, 4, 5],
[ 2, 3, 6, 7],
[ 8, 9, 12, 13],
[10, 11, 14, 15],
[16, 17, 20, 21],
[18, 19, 22, 23]])
In [371]: xl=np.arange(2*3*2*2).reshape((2,3,2,2))
In [372]: np.vstack([np.hstack(row) for row in xl])
Out[372]:
array([[ 0, 1, 4, 5, 8, 9],
[ 2, 3, 6, 7, 10, 11],
[12, 13, 16, 17, 20, 21],
[14, 15, 18, 19, 22, 23]])
If I have a square matrix as a nested list in python I can split it up into several equal sized boxes and create a new list where each element is a list of the numbers in one of those boxes. E.g.
a = [[1, 2, 3, 4],
[5, 6, 7, 8],
[9, 10, 11, 12],
[13, 14, 15 ,16]]
b = [[a[0][0], a[0][1], a[1][0], a[1][1]],
[a[0][2], a[0][3], a[1][2], a[1][3]],
[a[2][0], a[2][1], a[3][0], a[3][1]],
[a[2][2], a[2][3], a[3][2], a[3][3]]]
Is there an easier way to do this? Is there a way to set this up as a function which I can apply to matrices of different sizes and also specify the size of the boxes?
The following is equivalent to what you have and a bit more concise:
b = [a[0][:2] + a[1][:2],
a[0][2:] + a[1][2:],
a[2][:2] + a[3][:2],
a[2][2:] + a[3][2:]]
Or an equivalent list comprehension:
b = [a[i][s] + a[i+1][s] for i in (0,2) for s in (slice(None,2), slice(2,None))]
Using NumPy:
In [31]: import numpy as np
In [32]: a = np.arange(1,17).reshape(4,4)
In [33]: a
Out[33]:
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]])
In [34]: b = a.reshape(-1,2,2,2).swapaxes(1,2).reshape(4,4)
In [35]: b
Out[35]:
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])