If I define a array X with shape (2, 2):
X = np.array([[1, 2], [3, 4]])
and take the kronecker product, then reshape the output using
np.kron(X, X).reshape((2, 2, 2, 2))
I get a resulting matrix:
array([[[[ 1, 2],
[ 2, 4]],
[[ 3, 4],
[ 6, 8]]],
[[[ 3, 6],
[ 4, 8]],
[[ 9, 12],
[12, 16]]]])
However, when I use np.tensordot(X, X, axes=0) the following matrix is output
array([[[[ 1, 2],
[ 3, 4]],
[[ 2, 4],
[ 6, 8]]],
[[[ 3, 6],
[ 9, 12]],
[[ 4, 8],
[12, 16]]]])
which is different from the first output. Why is this the case? I found this while searching for answers, however I don't understand why that solution works or how to generalise to higher dimensions.
My first question is, why do you expect them to be same?
Let's do the kron without reshaping:
In [403]: X = np.array([[1, 2],
...: [3, 4]])
...:
In [404]: np.kron(X,X)
Out[404]:
array([[ 1, 2, 2, 4],
[ 3, 4, 6, 8],
[ 3, 6, 4, 8],
[ 9, 12, 12, 16]])
It's easy to visualize the action.
[X*1, X*2
X*3, X*4]
tensordot normally is thought of as a generalization of np.dot, able to handle more complex situations than the common matrix product (i.e. sum of products on one or more axes). But here there's no summing.
In [405]: np.tensordot(X,X, axes=0)
Out[405]:
array([[[[ 1, 2],
[ 3, 4]],
[[ 2, 4],
[ 6, 8]]],
[[[ 3, 6],
[ 9, 12]],
[[ 4, 8],
[12, 16]]]])
When axes is an integer rather than a tuple, the action is a little tricky to understand. The docs say:
``axes = 0`` : tensor product :math:`a\otimes b`
I just tried to explain what is happening when axes is a scalar (it's not trivial)
How does numpy.tensordot function works step-by-step?
Specifying axes=0 is equivalent to providing this tuple:
np.tensordot(X,X, axes=([],[]))
In any case it's evident from the output that this tensordot is producing the same numbers - but the layout is different from the kron.
I can replicate the kron layout with
In [424]: np.tensordot(X,X,axes=0).transpose(0,2,1,3).reshape(4,4)
Out[424]:
array([[ 1, 2, 2, 4],
[ 3, 4, 6, 8],
[ 3, 6, 4, 8],
[ 9, 12, 12, 16]])
That is I swap the middle 2 axes.
And omitting the reshape, I get the same (2,2,2,2) you get from kron:
np.tensordot(X,X,axes=0).transpose(0,2,1,3)
I like the explicitness of np.einsum:
np.einsum('ij,kl->ijkl',X,X) # = tensordot(X,X,0)
np.einsum('ij,kl->ikjl',X,X) # = kron(X,X).reshape(2,2,2,2)
Or using broadcasting, the 2 products are:
X[:,:,None,None]*X[None,None,:,:] # tensordot 0
X[:,None,:,None]*X[None,:,None,:] # kron
Related
I have been going over this issue with numpy for a while and cant figure out if there is a intuitive way of converting the array while maintaining the position of the sub-array. The sizes of the array will change depending on the input so doing it manually with concatenate is not an option but i do have the dimensions.
a= np.array([[[0,1],[2,3]],[[4,5],[6,7]],[[8,9],[10,11]],[[12,13],[14,15]]])
reshaping just flattens the array like
[1,2,3,4]
[5,6,7,8]
etc
I have also tried np.block but besides setting the positions manually i have not had any success
The result i would like to get in this case is (4,4):
[[ 0, 1, 4, 5],
[ 2, 3, 6, 7],
[ 8, 9,12,13],
[10,11,14,15]]
Does anyone of you smart people know if there is something in numpy that i could use to get this result?
Your original has the 16 consecutive values reshaped into 4d array:
In [67]: x=np.arange(16).reshape(2,2,2,2)
In [68]: x
Out[68]:
array([[[[ 0, 1],
[ 2, 3]],
[[ 4, 5],
[ 6, 7]]],
[[[ 8, 9],
[10, 11]],
[[12, 13],
[14, 15]]]])
Reshape to (4,4) keeps that original order - see the 0,1,2,3...
In [69]: x.reshape(4,4)
Out[69]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
You want to swap values:
In [70]: x.transpose(0,2,1,3)
Out[70]:
array([[[[ 0, 1],
[ 4, 5]],
[[ 2, 3],
[ 6, 7]]],
[[[ 8, 9],
[12, 13]],
[[10, 11],
[14, 15]]]])
which can then be reshaped to (4,4):
In [71]: x.transpose(0,2,1,3).reshape(4,4)
Out[71]:
array([[ 0, 1, 4, 5],
[ 2, 3, 6, 7],
[ 8, 9, 12, 13],
[10, 11, 14, 15]])
I have a number of row vectors which I would like to batch as column vectors and use as input for Conv1d. As an example I'd like to reshape the tensor x into y i.e. making two groups of two column vectors.
# size = [4, 3]
x = torch.tensor([
[0, 1, 2],
[3, 4, 5],
[6, 7, 8],
[9, 10, 11]
])
# size = [2, 3, 2]
y = torch.tensor([
[[0, 3],
[1, 4],
[2, 5]],
[[6, 9],
[7, 10],
[8, 11]]
])
Is there a way to do this with just reshape and similar functions? The only way I can think of is using loops and copying into a new tensor.
You need to use permute as well as reshape:
x.reshape(2, 2, 3).permute(0, 2, 1)
Out[*]:
tensor([[[ 0, 3],
[ 1, 4],
[ 2, 5]],
[[ 6, 9],
[ 7, 10],
[ 8, 11]]])
First, you split the vectors into 2 x.reshape(2,2,3) placing the extra dimension in the middle. Then using permute you change the order of dimensions to be as you expected.
You can also use torch.split and torch.stack like
torch.stack(x.split(2), dim=2) # or torch.stack(x.T.split(2, dim=1))
tensor([[[ 0, 3],
[ 1, 4],
[ 2, 5]],
[[ 6, 9],
[ 7, 10],
[ 8, 11]]])
I would like to know if there is any fast way to sum each row of a first array with all rows of a second array. In this case both arrays have the same number of colulmns. For instance if array1.shape = (n,c) and array2.shape = (m,c), the resulting array would be an array3.shape = ((n*m), c)
Look at the example below:
array1 = np.array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
array2 = np.array([[0, 1, 2],
[3, 4, 5]])
The result would be:
array3 = np.array([[0, 2, 4],
[3, 5, 7]
[3, 5, 7]
[6, 8, 10]
[6, 8, 10]
[9, 11, 13]])
The only way I see I can do this is to repeat each row of one of the arrays the number of rows of the other array. For instance, by doing np.repeat(array1, len(array2), axis=0) and then sum this array with array2. This is not very practical however if the number of rows is too big. The other way would be with a for loop but this is too slow.
Any other better way to do it..?
Thanks in advance.
Extend array1 to 3D so that it becomes broadcastable against 2D array2 and then perform broadcasted addition and a final reshape is needed for desired output -
In [30]: (array1[:,None,:] + array2).reshape(-1,array1.shape[1])
Out[30]:
array([[ 0, 2, 4],
[ 3, 5, 7],
[ 3, 5, 7],
[ 6, 8, 10],
[ 6, 8, 10],
[ 9, 11, 13]])
You could try the following inline code if you haven't already. This is the simplest and probably also the quickest on a single thread.
>>> import numpy as np
>>> array1 = np.array([[0, 1, 2],
... [3, 4, 5],
... [6, 7, 8]])
>>>
>>> array2 = np.array([[0, 1, 2],
... [3, 4, 5]])
>>> array3 = np.array([i+j for i in array1 for j in array2])
>>> array3
array([[ 0, 2, 4],
[ 3, 5, 7],
[ 3, 5, 7],
[ 6, 8, 10],
[ 6, 8, 10],
[ 9, 11, 13]])
>>>
If you are looking for speed up by treading, you could consider using CUDA or multithreading. This suggestion goes a bit out of scope of your question but gives you an idea of what can be done to speed up matrix operations.
I'm having trouble finding a way to clone a 1-D numpy array to create a 3-D array. Say for example, I have
z = np.array([0, 2, 3, 5, 7, 9, 10])
This represents a vertical column in 3-D space (let's say it's in meters, for example). I want to create a horizontal dimension (x, y), so that the final array has dimensions (len(z), len(x), len(y)) , where each column at each x,y point is the same. I'm doing this so that I can match the spatial dimensions to other 3-D data that I have.
So with the array z as input, and given horizontal dimensions of, say, ndimx=3, ndimy=2, I want to find a function that outputs the array
np.array([ [[0, 0], [0, 0], [0, 0]],
[[2, 2], [2, 2], [2, 2]],
[[3, 3], [3, 3], [3, 3]],
[[5, 5], [5, 5], [5, 5]],
[[7, 7], [7, 7], [7, 7]],
[[9, 9], [9, 9], [9, 9]],
[[10, 10], [10, 10], [10, 10]] ])
which has a shape of (7, 3, 2). This seemed trivial to me at first, but after ages of experimenting with np.dstack(), np.astype(), np.repeat(), and transposes with ().T, I can't get the dimensional order right.
That's the critical thing here, that the vertical column is the first dimension. I'm willing to bet the answer is indeed trivial, I just can't find the magic numpy function that will do it.
Any suggestions?
We can simply get a view into input z with np.broadcast_to with no additional memory overhead and as such virtually free on performance -
np.broadcast_to(z[:,None,None],(len(z),ndimx,ndimy))
Sample run -
In [23]: z
Out[23]: array([ 0, 2, 3, 5, 7, 9, 10])
In [24]: ndimx=3; ndimy=2
In [25]: np.broadcast_to(z[:,None,None],(len(z),ndimx,ndimy))
Out[25]:
array([[[ 0, 0],
[ 0, 0],
[ 0, 0]],
[[ 2, 2],
[ 2, 2],
[ 2, 2]],
[[ 3, 3],
[ 3, 3],
[ 3, 3]],
[[ 5, 5],
[ 5, 5],
[ 5, 5]],
[[ 7, 7],
[ 7, 7],
[ 7, 7]],
[[ 9, 9],
[ 9, 9],
[ 9, 9]],
[[10, 10],
[10, 10],
[10, 10]]])
In [360]: z = np.array([0, 2, 3, 5, 7, 9, 10])
In [361]: z1 = np.stack([z,z], axis=1)
In [362]: z2 = np.stack([z1,z1,z1],axis=1)
In [363]: z2
Out[363]:
array([[[ 0, 0],
[ 0, 0],
[ 0, 0]],
[[ 2, 2],
[ 2, 2],
[ 2, 2]],
[[ 3, 3],
[ 3, 3],
[ 3, 3]],
...
[[10, 10],
[10, 10],
[10, 10]]])
I am looking for a way to apply a function n items at the time along an axis. E.g.
array([[ 1, 2],
[ 3, 4],
[ 5, 6],
[ 7, 8]])
If I apply sum across the rows 2 items at a time I get:
array([[ 4, 6],
[ 12, 14]])
Which is the sum of 1st 2 rows and the last 2 rows.
NB: I am dealing with much larger array and I have to apply the function to n items which I can be decided at runtime.
The data extends along different axis. E.g.
array([[... [ 1, 2, ...],
[ 3, 4, ...],
[ 5, 6, ...],
[ 7, 8, ...],
...], ...])
This is a reduction:
numpy.add.reduceat(a, [0,2])
>>> array([[ 4, 6],
[12, 14]], dtype=int32)
As long as by "larger" you mean longer in the "y" axis, you can extend:
a = numpy.array([[ 1, 2],
[ 3, 4],
[ 5, 6],
[ 7, 8],
[ 9, 10],
[11, 12]])
numpy.add.reduceat(a, [0,2,4])
>>> array([[ 4, 6],
[12, 14],
[20, 22]], dtype=int32)
EDIT: actually, this works fine for "larger in both dimensions", too:
a = numpy.arange(24).reshape(6,4)
numpy.add.reduceat(a, [0,2,4])
>>> array([[ 4, 6, 8, 10],
[20, 22, 24, 26],
[36, 38, 40, 42]], dtype=int32)
I will leave it up to you to adapt the indices to your specific case.
Reshape splitting the first axis into two axes, such that the second split axis is of length n to have a 3D array and then sum along that split axis, like so -
a.reshape(a.shape[0]//n,n,a.shape[1]).sum(1)
It should be pretty efficient as reshaping just creates a view into input array.
Sample run -
In [55]: a
Out[55]:
array([[2, 8, 0, 0],
[1, 5, 3, 3],
[6, 1, 4, 7],
[0, 4, 0, 7],
[8, 0, 8, 1],
[8, 3, 3, 8]])
In [56]: n = 2 # Sum every two rows
In [57]: a.reshape(a.shape[0]//n,n,a.shape[1]).sum(1)
Out[57]:
array([[ 3, 13, 3, 3],
[ 6, 5, 4, 14],
[16, 3, 11, 9]])
How about something like this?
n = 2
# calculate the cumsum along axis 0 and take one row from every n rows
cumarr = arr.cumsum(axis = 0)[(n-1)::n]
# calculate the difference of the resulting numpy array along axis 0
np.vstack((cumarr[0][None, :], np.diff(cumarr, axis=0)))
# array([[ 4, 6],
# [12, 14]])