Related
What is the most numerically efficient way to add all the combinations of two arrays of vectors? For example what I want is the following:
a = np.array([[1,2,3], [4,5,6]])
b = np.array([[7,8,9], [10,11,12]])
[ai + bj for ai in a for bj in b]
Gives
[array([ 8, 10, 12]),
array([11, 13, 15]),
array([11, 13, 15]),
array([14, 16, 18])]
It's a meshgrid with vectors instead of primary data types.
I've tried somewhat explicitly constructing the meshgrid results, which is faster than the list comprehension:
a_tile = np.tile(a, (2, 1))
array([[1, 2, 3],
[4, 5, 6],
[1, 2, 3],
[4, 5, 6]])
b_repeat = np.repeat(b, 2, axis=0)
array([[ 7, 8, 9],
[ 7, 8, 9],
[10, 11, 12],
[10, 11, 12]])
a_tile + b_repeat
array([[ 8, 10, 12],
[11, 13, 15],
[11, 13, 15],
[14, 16, 18]])
Is this as efficient as it gets? I was looking for a way to broadcast the arrays so that the grid isn't explicitly constructed.
You can try the following:
import numpy as np
a = np.array([[1,2,3], [4,5,6]])
b = np.array([[7,8,9], [10,11,12]])
(a[..., None] + b.T).transpose(0, 2, 1).reshape(-1, 3)
It gives:
array([[ 8, 10, 12],
[11, 13, 15],
[11, 13, 15],
[14, 16, 18]])
you can use numpy.broadcast_to to broadcast an array
N = 2 #number of repeats
your_req_array = np.broadcast_to(b.T, (N,b.shape[1], b.shape[0])).transpose(2,0,1).reshape(-1,b.shape[1]) + np.broadcast_to(a, (N,a.shape[0], a.shape[1])).reshape(-1,b.shape[1])
Imagine I have an n x d python array, e.g. a=np.array([[1,2,3],[4,5,6], [7,8,9], [10,11,12], [13,14,15]])
so in this case n=5, d=3 and imagine I have some number c which is smaller or equal than n and what I want to calculate is the following:
Consider every column independently and calculate the sum of every c values; e.g. if c=2, the solution would be
solution=np.array([[1+4, 2+5, 3+6], [7+10,8+11,9+12]])
The last row is skipped because 5 mod 2 = 1, so we need to leave out one line in the end;
If c=1, the solution would be the original array and if e.g. c=3 the solution would be
solution=np.array([[1+4+7, 2+5+8, 3+6+9]]), while the last two lines are omitted;
Now what would be the most elegant and efficient solution to do that? I have searched a lot online but could not find a similar problem
Here's one way -
def sum_in_blocks(a, c):
# Get extent of each col for summing
l = c*(len(a)//c)
# Reshape to 3D considering first l rows, and "cutting" after each c rows
# Then sum along second axis
return a[:l].reshape(-1,c,a.shape[1]).sum(1)
More info on second step - General idea for nd to nd transformation.
Sample runs -
In [79]: a
Out[79]:
array([[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, 12],
[13, 14, 15]])
In [80]: sum_in_blocks(a, c=1)
Out[80]:
array([[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, 12],
[13, 14, 15]])
In [81]: sum_in_blocks(a, c=2)
Out[81]:
array([[ 5, 7, 9],
[17, 19, 21]])
In [82]: sum_in_blocks(a, c=3)
Out[82]: array([[12, 15, 18]])
Explanation with given sample
In [84]: a
Out[84]:
array([[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, 12],
[13, 14, 15]])
In [85]: c = 2
In [87]: l = c*(len(a)//c) # = 4; Get extent of each col for summing
In [89]: a[:l] # hence not relevant rows are skipped
Out[89]:
array([[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, 12]])
# Reshape to 3D "cutting" after every c=2 rows
In [90]: a[:l].reshape(-1,c,a.shape[1])
Out[90]:
array([[[ 1, 2, 3],
[ 4, 5, 6]],
[[ 7, 8, 9],
[10, 11, 12]]])
# Sum along axis=1 for final o/p
In [91]: a[:l].reshape(-1,c,a.shape[1]).sum(axis=1)
Out[91]:
array([[ 5, 7, 9],
[17, 19, 21]])
I'm looking for a short readable way to select some rows of an 2D numpy.ndarray, where the first number of each row is in some list.
Example:
>>> index
[4, 8]
>>> data
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
So in this case i only need
array([[ 4, 5, 6, 7],
[8, 9, 10, 11]])
because the first numbers of these rows are 4 and 8 which are listed in index.
Basically im looking for something like:
data[data[:,0] == i if i in index]
which of course is not working.
You can use np.isin to check, then index as usual:
idx = [4, 8]
data = np.array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
>>> data[np.isin(data[:,0], idx)]
array([[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
If you want to use a list comprehension, here's how (isin is undoubtably better for this usage case, though):
>>> np.array([e for e in data if e[0] in index])
array([[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
I'm struggling to get each of the four numbers of a python array contained within a 2x2 into their separate arrays much like a Sudoku grid. Order doesn't matter. I would have tried writing a code or something but my mind has gone blank.
example grid
[
[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]
]
I want to be able to get it in the form
[
[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]
]
Here is a pure Python solution. If you are after something more readable consider using NumPy (see below).
>>> from itertools import count, chain
>>>
# create 2x2 blocks of 2x2
>>> c = count(1)
>>> L4D = [[[[next(c) for i in range(2)] for j in range(2)] for k in range(2)] for l in range(2)]
>>> L4D
[[[[1, 2], [3, 4]], [[5, 6], [7, 8]]], [[[9, 10], [11, 12]], [[13, 14], [15, 16]]]]
# swap middle dimensions
>>> L4D = [zip(*i) for i in L4D]
# next line is not necessary, only here so we can see what's going on
>>> L4D = [list(i) for i in L4D]
>>> L4D
[[([1, 2], [5, 6]), ([3, 4], [7, 8])], [([9, 10], [13, 14]), ([11, 12], [15, 16])]]
# join first two and last two dimensions
>>> result = [list(chain.from_iterable(j)) for j in chain.from_iterable(L4D)]
>>> result
[[1, 2, 5, 6], [3, 4, 7, 8], [9, 10, 13, 14], [11, 12, 15, 16]]
If using NumPy is an option this can be simplified. Here are three different possibilities. The first is a direct translation of the pure Python solution:
>>> import numpy as np
>>>
>>> np.arange(1, 17).reshape(2, 2, 2, 2).swapaxes(1, 2).reshape(4, 4)
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
>>> np.block(list(map(list, np.arange(1, 17).reshape(2, 2, 2, 2))))
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
>>> a = np.arange(4).reshape(2, 2)
>>> b = np.ones((2, 2), dtype = int)
>>> 4 * np.kron(a, b) + np.kron(b, a) + 1
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
I finally found a solution to my question with a little modification to this answer
cells = [[] * 4 for x in range(4)]
for row_index, row in enumerate(grid):
for col_index, col in enumerate(row):
i = col_index//2 + 2*(row_index//2)
cells[i].append(col)
I have these 4 matrices and I want to dynamically combine them into one big matrix by passing n: number of small matrix and output matrix row and column
example:
[[[ 1 2]
[ 3 4]]
[[ 5 6]
[ 7 8]]
[[ 9 10]
[11 12]]
[[13 14]
[15 16]]]
the output matrix:
[[ 1 2 5 6]
[ 3 4 7 8]
[ 9 10 13 14]
[11 12 15 16]]
I can do it manually using:
M = np.bmat( [[x1], [x2], [x3], [x4]] )
I think (but dont know if its right), that its best to work inplace and avoid to create new objects with new methods each time - specifically when You are doing it in loop multiple times. These examples are only for 2d matrices. But it could be easilly implemented to more dimensions. Best would be to have one big array, if its really big, prolly in numpy.memmap array. Then work on its parts. Fastest indexing (second to pointers) would be on cython memoryviews...
import numpy as np
def combine_matrix(*args):
n=len(args)
rows,cols=args[0].shape
a=np.zeros((n,cols*rows))
m=0
for i in range(n/rows):
for j in range(n/cols):
a[i*rows:(i+1)*rows,j*cols:(j+1)*cols]=args[m]
m+=1
return a
def example1():
print '#'*10
a=np.arange(1,17)
n=4
rows,cols=n/2,n/2
lst=[]
for i in range(n):
ai=a[i*n:(i+1)*n]
ai.shape=rows,cols
lst.append(ai)
print lst
print combine_matrix(*lst)
def example2():
print '#'*10
m=24
a=np.arange(m)
n=6
rows,cols=m/n/2,n/2
lst=[]
for i in range(m/n):
ai=a[i*n:(i+1)*n]
ai.shape=rows,cols
lst.append(ai)
print lst
print combine_matrix(*lst)
def example3():
print '#'*10
m,n=36,6
a=np.arange(m)
arrs=np.array_split(a,n)
for i in range(n):
ln=arrs[i].shape[0]
arrs[i].shape=2,ln/2
print combine_matrix(*arrs)
example1()
example2()
example3()
2 minutes implementation (for question before edition, maybe usefull for someone):
import numpy as np
a=np.ones((10,10))
b=a*3
c=a*1
d=a*1.5
def combine_matrix(*args):
n=len(args)
rows,cols=args[0].shape
a=np.zeros((n,rows,cols))
for i in range(n):
a[i]=args[i]
return a
print combine_matrix(a,b,c,d)
If sizes of arrays are huge there is place for improvement...
You can combine transposition and reshape operations:
In [1878]: x=arange(24).reshape(4,3,2)
In [1879]: (_,n,m)=x.shape
In [1880]: x.reshape(2,2,n,m).transpose(0,2,1,3).reshape(2*n,2*m)
Out[1880]:
array([[ 0, 1, 6, 7],
[ 2, 3, 8, 9],
[ 4, 5, 10, 11],
[12, 13, 18, 19],
[14, 15, 20, 21],
[16, 17, 22, 23]])
[edit - I'm assuming that the small arrays are created independently, though my example is based on splitting a (4,2,2) array. If they really are just planes of a 3d array, then some combination of 'reshape' and 'transpose' will work better. But even such a solution will produce a copy because the original values are rearranged.]
Lets make a list of 2x2 arrays (here from a 3d array). Squeeze is needed because this split produces (1,2,2) arrays:
n = len(A)
E = np.zeros((n,n))
In [330]: X=np.arange(1,17).reshape(4,2,2)
In [331]: xl=[np.squeeze(i) for i in np.split(X,4,0)]
In [332]: xl
Out[332]:
[array([[1, 2],
[3, 4]]), array([[5, 6],
[7, 8]]), array([[ 9, 10],
[11, 12]]), array([[13, 14],
[15, 16]])]
Your bmat approach - corrected to produce the square arrangment
In [333]: np.bmat([[xl[0],xl[1]],[xl[2],xl[3]]])
Out[333]:
matrix([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
A concatenation approach:
In [334]: np.vstack([np.hstack(xl[:2]),np.hstack(xl[2:])])
Out[334]:
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
Since slicing works in hstack I could also use it in the bmat:
In [335]: np.bmat([xl[:2],xl[2:]])
Out[335]:
matrix([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
Internally bmat (check its code) is using a version of the vstack of hstacks (contactenates on first and last axes). Effectively
In [366]: ll=[xl[:2], xl[2:]]
In [367]: np.vstack([np.hstack(row) for row in ll])
Out[367]:
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
Some how you have to specify the arrangement of these n arrays. np.bmat(xl) produces a (2,8) matrix (so does hstack). np.vstack(xl) produces a (8,2) array.
It shouldn't be hard to extend this to work with a 3x3, 2x3, etc layout of subarrays. xl is a list of subarrays. Rework it into the desired list of lists of subarrays and apply bmat or the combination of stacks.
2 quick versions of 2x3 layout (a 4d xl array is easier to construct than a 2x3 nested list, but functionally will be the same:
In [369]: xl=np.arange(3*2*2*2).reshape((3,2,2,2))
In [370]: np.vstack([np.hstack(row) for row in xl])
Out[370]:
array([[ 0, 1, 4, 5],
[ 2, 3, 6, 7],
[ 8, 9, 12, 13],
[10, 11, 14, 15],
[16, 17, 20, 21],
[18, 19, 22, 23]])
In [371]: xl=np.arange(2*3*2*2).reshape((2,3,2,2))
In [372]: np.vstack([np.hstack(row) for row in xl])
Out[372]:
array([[ 0, 1, 4, 5, 8, 9],
[ 2, 3, 6, 7, 10, 11],
[12, 13, 16, 17, 20, 21],
[14, 15, 18, 19, 22, 23]])