Suppose I have two arrays x and t of length N and I want to create a matrix, where
M_i,j = t_i * t_j * func(x_i, x_j)
In this case func() simply takes in two elements of x and returns a scalar value.
t.shape = (N, )
x.shape = (N, 2)
// expected
M.shape = (N, N)
Question is: Can this be done without looping through the whole matrix in a vectorized fashion using numpy? I know there are ways to populate matrizes via functions, the problem is, that here the function arguments depend on the indices of the matrix which has me stuck.
Not sure the shortest way to do but it's possible
> np.diag(t) # np.fromfunction(lambda i, j: x[i] + x[j], (2, 2), dtype=int) # np.diag(t)
array([[ 0, 15],
[15, 50]])
the function is defined as a simple lambda, substitute with your own.
here I used
> t=np.array([3,5])
> x=np.array([0,1])
Ok, in this particular case I was able to answer my own question, since my function that originally takes two inputs actually just computes their differences and then calculates further values with that result.
With that in mind I was able to create a "Matrix of Differences" and use standard numpy functions in a vectorized manner as follows (by breaking up my function into its components):
Original Function (RBF Kernel):
def CalculateGaussKernel(self, x1minusx2):
return np.exp((-np.linalg.norm(x1minusx2) ** 2) / (2 * self.sigma ** 2))
Code that calculates Formular from OP in vectorized manner:
t_outer = np.outer(t, t)
DiffMat = x.reshape(-1, 1, 2) - x
KernelMat = np.exp(-(np.linalg.norm(DiffMat, axis=2)**2) / (2 * self.sigma**2))
K = np.multiply(t_outer, KernelMat)
As mentioned, this works only due to the nature of my specific function. #karakfa provided another answer for general function, however I couldn't make it work so far. Anyway, maybe his answer is better suitable for my question.
Related
I am new user to Python.
I want to add many exponential functions, and then take (and store in memory) the logarithm of the result. (Side note : I am doing this because the sum of the exponential functions is very large so storing the log value of this result is a workaround). Can anyone help me use this numpy function https://numpy.org/doc/stable/reference/generated/numpy.logaddexp.html
In the below code I have a 2 x 2 matrix M and a 2 dimensional vector v. I want to first add v the columns of M. So in the below code the result should be
[[11, 22], [13, 24]]
Then I want to take the exponential of each value and sum across the rows (ending up with a vector of length 2), and storing the logarithm of the result. However the below code outputs a matrix and I cant work out how to use the "out=None" imput for the logaddexp function.
import numpy as np
M = np.array([[1, 2], [3, 4]])
v = np.array([10, 20])
result = np.logaddexp(M, v[None, :])
The function np.logaddexp() performs an elementwise operation. In your case, you need the addition to be performed along a given axis. Using some basic functions, you can try the following.
import numpy as np
M = np.array([[1, 2], [3, 4]]) # '2 x 2' array
v = np.array([[10, 20]]) # '1 x 2' array
sum_Mv = M + v # '2 x 2' array
result = np.log(np.sum(np.exp(sum_Mv), axis=1))
Change the 'axis' parameter if needed.
If you still want to use np.logaddexp(), you can split the summed matrix into two halves and perform the operation as shown below.
result = np.logaddexp(sum_Mv[:, 0], sum_Mv[:, 1])
TLDR:
import numpy as np
M = np.array([[1, 2], [3, 4]])
v = np.array([10, 20])
result = np.logaddexp.reduce(M + v, axis=___)
Fill in ___ depending on what "sum across the rows" means
Consider the difference between np.add and np.sum.
np.add, much like the + operator, always takes in 2 arguments, x1 and x2, and adds them together. np.add is a numpy ufunc. If x1 or x2 is an array_like, then the arguments are broadcast together.
np.sum always takes in 1 argument, typically an array_like of items, and performs a summation of all of the elements in the array_like. This is essentially equivalent to iteratively taking an element from the array_like and repeatedly calling np.add with that element on a running result variable. The running result variable is initialized with 0.
Similarly, what np.sum is to np.add, np.prod is to np.multiply (with running result initalized as 1).
Every np.ufunc (such as np.add and np.multiply, but also np.logaddexp), comes with a reduce method and an accompanying identity property that is used as initialization for the running result.
np.add.reduce is exactly equivalent to np.sum. np.multiply.reduce is exactly equivalent to np.prod.
What you're looking to do is a log-sum-exp; but numpy only offers np.logaddexp. As such, you can use np.logaddexp.reduce to get the required functionality. Confusion arises from the fact that you're adding M and v as well as adding exponential terms together. You can simply perform the M + v operation first, and pass the resulting array (the intermediate result in your question), to np.logaddexp.reduce. Note that M + v is equivalent to M + v[None, :] in this case due to numpy's broadcasting rules.
I wrote the following function, which takes as inputs three 1D array (namely int_array, x, and y) and a number lim. The output is a number as well.
def integrate_to_lim(int_array, x, y, lim):
if lim >= np.max(x):
res = 0.0
if lim <= np.min(x):
res = int_array[0]
else:
index = np.argmax(x > lim) # To find the first element of x larger than lim
partial = int_array[index]
slope = (y[index-1] - y[index]) / (x[index-1] - x[index])
rest = (x[index] - lim) * (y[index] + (lim - x[index]) * slope / 2.0)
res = partial + rest
return res
Basically, outside form the limit cases lim>=np.max(x) and lim<=np.min(x), the idea is that the function finds the index of the first value of the array x larger than lim and then uses it to make some simple calculations.
In my case, however lim can also be a fairly big 2D array (shape ~2000 times ~1000 elements)
I would like to rewrite it such that it makes the same calculations for the case that lim is a 2D array.
Obviously, the output should also be a 2D array of the same shape of lim.
I am having a real struggle figuring out how to vectorize it.
I would like to stick only to the numpy package.
PS I want to vectorize my function because efficiency is important and as I understand using for loops is not a good choice in this regard.
Edit: my attempt
I was not aware of the function np.take, which made the task way easier.
Here is my brutal attempt that seems to work (suggestions on how to clean up or to make the code faster are more than welcome).
def integrate_to_lim_vect(int_array, x, y, lim_mat):
lim_mat = np.asarray(lim_mat) # Make sure that it is an array
shape_3d = list(lim_mat.shape) + [1]
x_3d = np.ones(shape_3d) * x # 3 dimensional version of x
lim_3d = np.expand_dims(lim_mat, axis=2) * np.ones(x_3d.shape) # also 3d
# I use np.argmax on the 3d matrices (is there a simpler way?)
index_mat = np.argmax(x_3d > lim_3d, axis=2)
# Silly calculations
partial = np.take(int_array, index_mat)
y1_mat = np.take(y, index_mat)
y2_mat = np.take(y, index_mat - 1)
x1_mat = np.take(x, index_mat)
x2_mat = np.take(x, index_mat - 1)
slope = (y1_mat - y2_mat) / (x1_mat - x2_mat)
rest = (x1_mat - lim_mat) * (y1_mat + (lim_mat - x1_mat) * slope / 2.0)
res = partial + rest
# Make the cases with np.select
condlist = [lim_mat >= np.max(x), lim_mat <= np.min(x)]
choicelist = [0.0, int_array[0]] # Shoud these options be a 2d matrix?
output = np.select(condlist, choicelist, default=res)
return output
I am aware that if the limit is larger than the maximum value in the array np.argmax returns the index zero (leading to wrong results). This is why I used np.select to check and correct for these cases.
Is it necessary to define the three dimensional matrices x_3d and lim_3d, or there is a simpler way to find the 2D matrix of the indices index_mat?
Suggestions, especially to improve the way I expanded the dimension of the arrays, are welcome.
I think you can solve this using two tricks. First, a 2d array can be easily flattened to a 1d array, and then your answers can be converted back into a 2d array with reshape.
Next, your use of argmax suggests that your array is sorted. Then you can find your full set of indices using digitize. Thus instead of a single index, you will get a complete array of indices. All the calculations you are doing are intrinsically supported as array operations in numpy, so that should not cause any problems.
You will have to specifically look at the limiting cases. If those are rare enough, then it might be okay to let the answers be derived by the default formula (they will be garbage values), and then replace them with the actual values you desire.
I want to create a 2D numpy array of size (N_r * N_z).
Across columns, the elements for 1 specific column (say j) shall be created based on the value r_thresh[j].
So 1 column (say j) out of the total of N_z columns in the numpy 2D array is created as:
(np.arange(N_r) + 0.5) * r_thresh[j] # this gives an array of size (1, N_r)
Of course, the column j + 1 shall be created as:
(np.arange(N_r) + 0.5) * r_thresh[j+1] # this gives an array of size (1, N_r)
r_thresh is a numpy array of size (1, N_z), already populated with values before I want to create the 2D array.
I want to ask you how do I go further and use this ''rule'' of creating each element of the numpy 2D array and actually create the whole array, in the most efficient way possible (speed-wise).
I initially wrote all the code using 2 nested for loops and plain python lists and the code worked, but took forever to run.
More experienced programmers told me to avoid for loops and use numpy because it's the best.
I now understand how to create 1D arrays using numpy np.arange() instruction, but I lack the knowledge on how to extrapolate this to 2 Dimensions.
Thanks!
The easiest way is to use einsum. In the case of r_thresh with the shape of (N_z,), you can use this code:
res = np.einsum("i,j->ij", np.arange(N_r) + 0.5, r_thresh)
Also, you can reshape np.arange(N_r) + 0.5 to the shape (N_r,1) and r_thresh to the shape (1,N_z). Thus, you can use the dot product (for Python version > 3.5):
res = (np.arange(N_r) + 0.5).reshape(N_r,1) # r_thresh.reshape(1,N_z)
or following to the comment of hpaulj:
res = (np.arange(N_r) + 0.5)[:,None] # r_thresh[None,:]
EDIT1
The comment of hpaulj is also very helpful (I pasted this into my answer to see better):
res = (np.arange(N_r) + 0.5)[:,None] * r_thresh
res = np.outer(np.arange(N_r) + 0.5, r_thresh)
IN ADDITION
You can also use tensordot:
res = np.tensordot((np.arange(N_r) + 0.5)[:,None], r_thresh[:,None], axes=[[-1],[-1]])
I'm trying to implement a machine learning model from a research paper into tensorflow which requires that I create a tensor block diagonal matrix. In the paper, they give the formula for the block which looks something like
[[cos(x * theta_1), sin(x * theta_2)], [-sin(x * theta_3), cos(x * theta_4)]]
I've made a function in tf which takes in x and returns a diagonal matrix but this matrix is going to be utilized hundreds of thousands of times over a training cycle so I'd like to find a way to avoid creating it from scratch every time I need to use it. Unfortunately, because x could be any real number within a range, I can't just create a matrix for every possible value of x and store them in a list for later use.
I'm wondering if there is a way to create the matrix such that it includes the variable x in some of its elements so that I can do something like
"create_tensor_from_schematic(tensor_with_variables, value_of_x)"
and it will
return the tensor evaluated for that value of x, saving me from having to reconstruct the diagonal matrix every time.
This matrix is a key component in a function that sits right in the middle of my model and is utilized by every training and testing sample once every epoch. Here's the code for that matrix:
def build_D(self, n, j):
def theta(k):
return (2 * math.pi * k) / n
def A(k, j):
j_thetak = j * theta(k)
return tf.convert_to_tensor([[math.cos(j_thetak), math.sin(j_thetak)],
[-math.sin(j_thetak), math.cos(j_thetak)]], dtype=tf.float32)
if n % 2 == 1: #n is odd
s = int((n - 1) / 2)
block_list_A = [tf.reshape(tf.convert_to_tensor([1], dtype=tf.float32), [1, 1])] + [A(k, j) for k in range(1, s + 1)]
else: #n is even
s = int((n - 2) / 2)
last_term = (-1) ** j
block_list_A = [tf.reshape(tf.convert_to_tensor([1], dtype=tf.float32), [1, 1])] + [A(k, j) for k in range(1, s + 1)] \
+ [tf.reshape(tf.convert_to_tensor([last_term], dtype=tf.float32), [1, 1])]
return tf.linalg.LinearOperatorBlockDiag(list(
map(lambda x: tf.linalg.LinearOperatorFullMatrix(x), block_list_A))).to_dense()
(this version of the code is the one I'm currently using which only supports integer valued j which allows me to just create the matrix for every j within my range and store them in a list but In the future j will be real valued and I obviously can't create a matrix for every possible j value.)
j is the only variable that changes, so it would be nice if there was a way to run this once and to keep just copy the j's over into the matrix for evaluation when I need the matrix which corresponds to a certain j value.
I wondered if it was possible to create a tensor with lambda expressions as elements but I can't imagine how I could pass an argument to them.
Is there an inbuilt conventional way to create a something like a tensor schematic in tensorflow? What are my options? Any ideas are appreciated.
I want to compute the sum product along one dimension of two multidimensional arrays, using Theano.
I'll describe precisely what I want to do using numpy first. numpy.tensordot and numpy.dot seem to always do a matrix product, whereas I'm in essence looking for a batched equivalent of a vector product. Given x and y, I want to compute z like so:
x = np.random.normal(size=(200, 2, 2, 1000))
y = np.random.normal(size=(200, 2, 2))
# this is how I now approach it:
z = np.sum(y[:,:,:,np.newaxis] * x, axis=1)
# z is of shape (200, 2, 1000)
Now I know that numpy.einsum would probably be able to help me here, but again, I want to do this particular computation in Theano, which does not have an einsum equivalent. I will need to use dot, tensordot, or Theano's specialized einsum subset functions batched_dot or batched_tensordot.
The reason I'm looking to change my approach to this is performance; I suspect that using builtin (CUDA) dot products will be faster than relying on broadcasting, element-wise product, and sum.
In Theano, none of the dimensions of three and four dimensional tensors are broadcastable. You have to explicitly set them. Then the Numpy principles will work just fine. One way to do this is to use T.patternbroadcast. To read more about broadcasting, refer this.
You have three dimensions in one of the tensors. So first you need to append a singleton dimension at the end and then make that dimension broadcastable. These two things can be achieved with a single command - T.shape_padaxis. The entire code is as follows:
import theano
from theano import tensor as T
import numpy as np
X = T.ftensor4('X')
Y = T.ftensor3('Y')
Y_broadcast = T.shape_padaxis(Y, axis=-1) # appending extra dimension and making it
# broadcastable
Z = T.sum((X*Y_broadcast), axis=1) # element-wise multiplication
f = theano.function([X, Y], Z, allow_input_downcast=True)
# Making sure that it works and gives correct results
x = np.random.normal(size=(3, 2, 2, 4))
y = np.random.normal(size=(3, 2, 2))
theano_result = f(x,y)
numpy_result = np.sum(y[:,:,:,np.newaxis] * x, axis=1)
print np.amax(theano_result - numpy_result) # prints 2.7e-7 on my system, close enough!
I hope this helps.