This question already has an answer here:
python: Multiply slice i of a matrix stack by column i of a matrix efficiently
(1 answer)
Closed 5 years ago.
There are really similar questions here, here, here, but I don't really understand how to apply them to my case precisely.
I have an array of matrices and an array of vectors and I need element-wise dot product. Illustration:
In [1]: matrix1 = np.eye(5)
In [2]: matrix2 = np.eye(5) * 5
In [3]: matrices = np.array((matrix1,matrix2))
In [4]: matrices
Out[4]:
array([[[ 1., 0., 0., 0., 0.],
[ 0., 1., 0., 0., 0.],
[ 0., 0., 1., 0., 0.],
[ 0., 0., 0., 1., 0.],
[ 0., 0., 0., 0., 1.]],
[[ 5., 0., 0., 0., 0.],
[ 0., 5., 0., 0., 0.],
[ 0., 0., 5., 0., 0.],
[ 0., 0., 0., 5., 0.],
[ 0., 0., 0., 0., 5.]]])
In [5]: vectors = np.ones((5,2))
In [6]: vectors
Out[6]:
array([[ 1., 1.],
[ 1., 1.],
[ 1., 1.],
[ 1., 1.],
[ 1., 1.]])
In [9]: np.array([m # v for m,v in zip(matrices, vectors.T)]).T
Out[9]:
array([[ 1., 5.],
[ 1., 5.],
[ 1., 5.],
[ 1., 5.],
[ 1., 5.]])
This last line is my desired output. Unfortunately it is very inefficient, for instance doing matrices # vectors that computes unwanted dot products due to broadcasting (if I understand well, it returns the first matrix dot the 2 vectors and the second matrix dot the 2 vectors) is actually faster.
I guess np.einsum or np.tensordot might be helpful here but all my attempts have failed:
In [30]: np.einsum("i,j", matrices, vectors)
ValueError: operand has more dimensions than subscripts given in einstein sum, but no '...' ellipsis provided to broadcast the extra dimensions.
In [34]: np.tensordot(matrices, vectors, axes=(0,1))
Out[34]:
array([[[ 6., 6., 6., 6., 6.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]],
[[ 0., 0., 0., 0., 0.],
[ 6., 6., 6., 6., 6.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]],
[[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 6., 6., 6., 6., 6.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]],
[[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 6., 6., 6., 6., 6.],
[ 0., 0., 0., 0., 0.]],
[[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 6., 6., 6., 6., 6.]]])
NB: my real-case scenario use more complicated matrices than matrix1 and matrix2
With np.einsum, you might use:
np.einsum("ijk,ki->ji", matrices, vectors)
#array([[ 1., 5.],
# [ 1., 5.],
# [ 1., 5.],
# [ 1., 5.],
# [ 1., 5.]])
You can use # as follows
matrices # vectors.T[..., None]
# array([[[ 1.],
# [ 1.],
# [ 1.],
# [ 1.],
# [ 1.]],
# [[ 5.],
# [ 5.],
# [ 5.],
# [ 5.],
# [ 5.]]])
As we can see it computes the right thing but arranges them wrong.
Therefore
(matrices # vectors.T[..., None]).squeeze().T
# array([[ 1., 5.],
# [ 1., 5.],
# [ 1., 5.],
# [ 1., 5.],
# [ 1., 5.]])
Related
I have a multi-level indexed square matrix, that needs to be reordered.
Say I have a two-level indexing system x and y and the square matrix M has the shape (len(x)*len(y), len(x)*len(y)).
M is sorted by the x index and I want to transform it to be sorted by the y index. Here is an example to contruct an arbitary square matrix M:
import numpy as np
nx = 4 # equal to len(x), arbitary
ny = 3 # equal to len(y), arbitary
A=np.ones(ny*ny).reshape(ny,ny) #arbitary
B=np.ones(ny*ny).reshape(ny,ny)*2 #arbitary
C=np.ones(ny*ny).reshape(ny,ny)*3 #arbitary
D=np.ones(ny*ny).reshape(ny,ny)*4 #arbitary
E=np.arange(ny*ny).reshape(ny,ny) #arbitary
M = np.block([[A, np.zeros((ny,ny)), E, np.zeros((ny,ny))],
[np.zeros((ny,ny)), B, np.zeros((ny,ny)),np.zeros((ny,ny))],
[np.zeros((ny,ny)),np.zeros((ny,ny)),C, np.zeros((ny,ny))],
[np.zeros((ny,ny)), np.zeros((ny,ny)), np.zeros((ny,ny)), D]])
and the resulting matrix M may look like this
array([[1., 1., 1., 0., 0., 0., 0., 1., 2., 0., 0., 0.],
[1., 1., 1., 0., 0., 0., 3., 4., 5., 0., 0., 0.],
[1., 1., 1., 0., 0., 0., 6., 7., 8., 0., 0., 0.],
[0., 0., 0., 2., 2., 2., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 2., 2., 2., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 2., 2., 2., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 3., 3., 3., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 3., 3., 3., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 3., 3., 3., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 4., 4., 4.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 4., 4., 4.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 4., 4., 4.]])
Now I want to transform the M into M_transformed that looks like this
array([[1., 0., 0., 0., 1., 0., 1., 0., 1., 0., 2., 0.],
[0., 2., 0., 0., 0., 2., 0., 0., 0., 2., 0., 0.],
[0., 0., 3., 0., 0., 0., 3., 0., 0., 0., 3., 0.],
[0., 0., 0., 4., 0., 0., 0., 4., 0., 0., 0., 4.],
[1., 0., 3., 0., 1., 0., 4., 0., 1., 0., 5., 0.],
[0., 2., 0., 0., 0., 2., 0., 0., 0., 2., 0., 0.],
[0., 0., 3., 0., 0., 0., 3., 0., 0., 0., 3., 0.],
[0., 0., 0., 4., 0., 0., 0., 4., 0., 0., 0., 4.],
[1., 0., 6., 0., 1., 0., 7., 0., 1., 0., 8., 0.],
[0., 2., 0., 0., 0., 2., 0., 0., 0., 2., 0., 0.],
[0., 0., 3., 0., 0., 0., 3., 0., 0., 0., 3., 0.],
[0., 0., 0., 4., 0., 0., 0., 4., 0., 0., 0., 4.]])
I use a very elementary, 4 layers of for loops to solve this problem and I believe there must be a more straight forward way (like a library) to solve this issue, as the matrix M can grow very large depending on the length of x and y (nx and ny)
M_transformed = np.zeros(M.shape)
for i in range(nx):
for j in range(nx):
for k in range(ny):
for l in range(ny):
M_transformed[k * nx + i,l * nx + j] = M[i * ny + k, j * ny + l]
I did it with no calculations, just borrowing some ideas from how to do maxpooling and experimenting a lot with swaps of axes.
I came to solution with this plan:
And this is my solution:
w = (3, 3)
initial_shape = M.shape
M = M.reshape((M.shape[0]//w[0], w[0], M.shape[1]//w[1], w[1]))
M = M.swapaxes(0, 1)
M = M.swapaxes(2, 3)
M = M.reshape(initial_shape)
I am trying to do something similar to One-Hot-Encoding but instead of the selected class being 1 and the rest zero, I want all the classes up to (and including the selected class) to be 1. Say I have a training batch with labels (5 possible class labels; 0, 1, 2, 3, 4)
y = np.array([0,2,1,3,4,1])
I can one-hot-encode with
def one_hot_encode(arr, num_classes):
return np.eye(num_classes)[arr]
which gives
>>> one_hot_encode(y, 5)
array([[ 1., 0., 0., 0., 0.],
[ 0., 0., 1., 0., 0.],
[ 0., 1., 0., 0., 0.],
[ 0., 0., 0., 1., 0.],
[ 0., 0., 0., 0., 1.],
[ 0., 1., 0., 0., 0.]])
I liked to get
array([[ 1., 0., 0., 0., 0.],
[ 1., 1., 1., 0., 0.],
[ 1., 1., 0., 0., 0.],
[ 1., 1., 1., 1., 0.],
[ 1., 1., 1., 1., 1.],
[ 1., 1., 0., 0., 0.]])
Anyone know how to do this?
You could achieve this by using a lower-triangular matrix instead of an identity matrix in your function definition:
def many_hot_encode(arr, num_classes):
return np.tril(np.ones(num_classes))[arr]
many_hot_encode(y,5)
array([[ 1., 0., 0., 0., 0.],
[ 1., 1., 1., 0., 0.],
[ 1., 1., 0., 0., 0.],
[ 1., 1., 1., 1., 0.],
[ 1., 1., 1., 1., 1.],
[ 1., 1., 0., 0., 0.]])
You can also use broadcasting -
out = (y[:,None] >= np.arange(num_classes)).astype(float)
Sample run -
In [71]: y = np.array([0,2,1,3,4,1])
In [72]: num_classes = 5
In [73]: (y[:,None] >= np.arange(num_classes)).astype(float)
Out[73]:
array([[ 1., 0., 0., 0., 0.],
[ 1., 1., 1., 0., 0.],
[ 1., 1., 0., 0., 0.],
[ 1., 1., 1., 1., 0.],
[ 1., 1., 1., 1., 1.],
[ 1., 1., 0., 0., 0.]])
Input array for reference,
u = array([[ 0., 0., 0., 0., 0.],
[ 0., 1., 1., 1., 0.],
[ 0., 1., 1., 1., 0.],
[ 0., 1., 1., 1., 0.],
[ 0., 0., 0., 0., 0.]])
python function using for loop
import numpy as np
u = np.zeros((5,5))
u[1:-1,1:-1]=1
def cds(n):
for i in range(1,4):
for j in range(1,4):
u[i,j] = u[i,j+1] + u[i,j-1] + u[i+1,j] + u[i-1,j]
return u
above function cds(5) provide the following result by using for loop,
u=array([[ 0., 0., 0., 0., 0.],
[ 0., 2., 4., 5., 0.],
[ 0., 4., 10., 16., 0.],
[ 0., 5., 16., 32., 0.],
[ 0., 0., 0., 0., 0.]])
same function using numpy
def cds(n):
u[1:-1,1:-1] = u[1:-1,2:] + u[1:-1,:-2] + u[2:,1:-1] + u[:-2,1:-1]
return u
But for the same input array(u), function cds(5) using NUMPY provide different result.,
u=array([[ 0., 0., 0., 0., 0.],
[ 0., 2., 3., 2., 0.],
[ 0., 3., 4., 3., 0.],
[ 0., 2., 3., 2., 0.],
[ 0., 0., 0., 0., 0.]])
The reason for this problem is, python "for loop" updates every u[i,j] value to the exsisting u array while looping but "numpy" didn't.....
I want same result from numpy as like as from the for loop.
Is there any way to solve this issue in NUMPY? please help me, Thanks in advance...
In theano, once the sharedvarialbe is initialized in one function, it will never be initialized again even if the function is accessed repeatedly, am I right?
def sgd_updates_adadelta(params,cost,rho=0.95,epsilon=1e-6,norm_lim=9,word_vec_name='Words'):
updates = OrderedDict({})
exp_sqr_grads = OrderedDict({})
exp_sqr_ups = OrderedDict({})
gparams = []
for param in params:
empty = np.zeros_like(param.get_value())
exp_sqr_grads[param] = theano.shared(value=as_floatX(empty),name="exp_grad_%s" % param.name)
gp = T.grad(cost, param)
exp_sqr_ups[param] = theano.shared(value=as_floatX(empty), name="exp_grad_%s" % param.name)
gparams.append(gp)
In the code above, the exp_sqr_grads variable and the exp_sqr_ups variable will not be initialized with zeros again when the sgd_updates_adadelta function is called the second time?
Shared variables are not static, if that is what you mean. My understanding of your code:
import theano
import theano.tensor as T
global_list = []
def f():
a = np.zeros((4, 5), dtype=theano.config.floatX)
b = theano.shared(a)
global_list.append(b)
Copy and paste this into an IPython and then try:
f()
f()
print global_list
The list will contain two items. They are not the same object:
In [9]: global_list[0] is global_list[1]
Out[9]: False
And they don't refer to the same memory: Do
global_list[0].set_value(np.arange(20).reshape(4, 5).astype(theano.config.floatX))
Then
In [20]: global_list[0].get_value()
Out[20]:
array([[ 0., 1., 2., 3., 4.],
[ 5., 6., 7., 8., 9.],
[ 10., 11., 12., 13., 14.],
[ 15., 16., 17., 18., 19.]])
In [21]: global_list[1].get_value()
Out[21]:
array([[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])
Having established that initializing shared variables several times leads to different variables, here is how to update a shared variable using a function. We re-use the established shared variables:
s = global_list[1]
x = T.scalar(dtype=theano.config.floatX)
g = theano.function([x], [s], updates=[(s, T.inc_subtensor(s[0, 0], x))])
g now increments the top left value of s by x at every call:
In [7]: s.get_value()
Out[7]:
array([[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])
In [8]: g(1)
Out[8]:
[array([[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])]
In [9]: s.get_value()
Out[9]:
array([[ 1., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])
In [10]: g(10)
Out[10]:
[array([[ 1., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])]
In [11]: s.get_value()
Out[11]:
array([[ 11., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])
I'd like to calculate the "cumulative minimum" array--basically, the minimum value of an array up to each index such as:
import numpy as np
nums = np.array([5.,3.,4.,2.,1.,1.,2.,0.])
cumulative_min = np.zeros(nums.size, dtype=float)
for i,num in enumerate(nums):
cumulative_min[i] = np.min(nums[0:i+1])
This works (it returns the correct array([ 5., 3., 3., 2., 1., 1., 1., 0.])
), but I'd like to avoid the for loop if I can. I thought it might be faster to construct a 2-d array and use the np.amin() function, but I needed a loop for that as well.
For any 2-argument NumPy universal function, its accumulate method is the cumulative version of that function. Thus, numpy.minimum.accumulate is what you're looking for:
>>> numpy.minimum.accumulate([5,4,6,10,3])
array([5, 4, 4, 4, 3])
Create a matrix which lower triangle (np.tril) is filled with values of your array nums and your upper triangle (np.triu, with second parameter 1, so the diagonal stays free) is filled with the maximum of the array. (EDIT: instead of the maximum, the first element of the array is the better way. -> comments)
nums = np.array([5.,3.,4.,2.,1.,1.,2.,0.])
oneSquare = np.ones((nums.size, nums.size))
A = nums * np.tril(oneSquare)
B = np.triu(oneSquare, 1) * nums[0]
A, B
Out:
(array([[ 5., 0., 0., 0., 0., 0., 0., 0.],
[ 5., 3., 0., 0., 0., 0., 0., 0.],
[ 5., 3., 4., 0., 0., 0., 0., 0.],
[ 5., 3., 4., 2., 0., 0., 0., 0.],
[ 5., 3., 4., 2., 1., 0., 0., 0.],
[ 5., 3., 4., 2., 1., 1., 0., 0.],
[ 5., 3., 4., 2., 1., 1., 2., 0.],
[ 5., 3., 4., 2., 1., 1., 2., 0.]]),
array([[ 0., 5., 5., 5., 5., 5., 5., 5.],
[ 0., 0., 5., 5., 5., 5., 5., 5.],
[ 0., 0., 0., 5., 5., 5., 5., 5.],
[ 0., 0., 0., 0., 5., 5., 5., 5.],
[ 0., 0., 0., 0., 0., 5., 5., 5.],
[ 0., 0., 0., 0., 0., 0., 5., 5.],
[ 0., 0., 0., 0., 0., 0., 0., 5.],
[ 0., 0., 0., 0., 0., 0., 0., 0.]]))
Now take the minimum of each row:
(A+B).min(axis=1)
Out:
array([ 5., 3., 3., 2., 1., 1., 1., 0.])