How do I perform the following in a TensorFlow tensor?
In matrix A: if A[i,j] > 1 then A[i,j] = 1
(in numpy I would do this: A[A>1] = 1)
You can use tf.minimum, which does element-wise minimum calculation; By setting y = 1, values in x will be clipped with the maximum of 1:
A = tf.constant([-1, 0, 1, 3, 4])
A_clipped = tf.minimum(A, 1)
sess = tf.InteractiveSession()
A_clipped.eval()
# array([-1, 0, 1, 1, 1], dtype=int32)
Another option is use tf.where to set values:
tf.where(A > 1, tf.constant(1, shape=A.shape), A).eval()
# array([-1, 0, 1, 1, 1], dtype=int32)
If you need to update Variable A:
A = tf.Variable([-1, 0, 1, 3, 4])
tf.global_variables_initializer().run()
tf.assign(A, tf.minimum(A, 1)).eval()
A.eval()
# array([-1, 0, 1, 1, 1], dtype=int32)
Related
this has probably been answered before but I could not come up with the appropriate search query to find the previous answers so apologies if you've answered this before.
Let's say I have a batch size of 4 and a 1D tensor specifying the "unpadded" lengths of my input features input_lengths=[4, 6, 8, 10].
My feature tensor will be of shape (4, 10, C) for C dimensional features at each timestep. I want to create a sample weights matrix of shape (4, 10) which is for each datum in the batch, filled with ones up to the "unpadded" length of that datum. So, for the case above,
sample_weights = [[1, 1, 1, 1, 0, 0, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1, 1, 1, 0, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]
I don't want to have a for-loop and was wondering if there's a more efficient way of creating this sample_weights matrix using torch.Tensor functions.
Thanks
Here's a benchmark between both approaches.
import torch
from torch.nn.utils.rnn import pad_sequence
def build_w_vec(inputs):
rows = len(inputs)
cols = sorted(inputs)[-1]
w = torch.zeros(rows, cols)
# start and end steps are incremented by `1` to avoid null indexing when filling `w`
aranges_list = list(map(lambda l: torch.arange(start=1, end=l + 1), inputs))
aranges_tensor = pad_sequence(aranges_list, batch_first=True)
w[aranges_tensor != 0] = 1
return w
def build_w_iter(inputs):
rows = len(inputs)
cols = sorted(inputs)[-1]
w = torch.zeros(rows, cols)
for i, length in enumerate(inputs):
arange = torch.arange(length)
w[i, arange] = 1
return w
I want to get familiar with the fourier based convolutions. Therefore, I created a small example using numpy.fft and scipy.signal.convolve. However, the results of the two operations are different and
I do not know why.
Does someone has an idea?
I have already tried to use the different modes of scipy.signal.convolve.
The example:
import numpy as np
from scipy.signal import convolve
# Generate example data
data = np.array([1, 1, 1, 1, 1, 1])
kernel = np.array([0, 1, 2, 1, 0, 0])
# Using scipy.signal.convolve
A = convolve(kernel, data, mode='full')
B = convolve(kernel, data, mode='valid')
C = convolve(kernel, data, mode='same')
# Using the convolution theorem
D = np.fft.ifft(np.fft.fft(kernel) * np.fft.fft(data))
The results are:
A = array([0, 1, 3, 4, 4, 4, 4, 3, 1, 0, 0])
B = array([4])
C = array([3, 4, 4, 4, 4, 3])
D = array([4.+0.j, 4.+0.j, 4.+0.j, 4.+0.j, 4.+0.j, 4.+0.j])
You need to pad data and kernel with N-1 zeroes to avoid circular convolution...
import numpy as np
from scipy.signal import convolve
# Generate example data
data = np.array([1, 1, 1, 1, 1, 1])
kernel = np.array([0, 1, 2, 1, 0, 0])
# Using scipy.signal.convolve
A = convolve(kernel, data, mode='full')
# Using the convolution theorem - need to pad with N-1 zeroes
data = np.array([1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0])
kernel = np.array([0, 1, 2, 1, 0, 0, 0, 0, 0, 0, 0])
D = np.fft.ifft(np.fft.fft(kernel) * np.fft.fft(data))
print (A)
print (D)
Result:
[0 1 3 4 4 4 4 3 1 0 0]
[2.4e-16+0.j 1.0e+00+0.j 3.0e+00+0.j 4.0e+00+0.j 4.0e+00+0.j 4.0e+00+0.j
4.0e+00+0.j 3.0e+00+0.j 1.0e+00+0.j 3.2e-16+0.j 1.6e-16+0.j]
I am trying to generate a vector-matrix outer product (tensor) using PyTorch. Assuming the vector v has size p and the matrix M has size qXr, the result of the product should be pXqXr.
Example:
#size: 2
v = [0, 1]
#size: 2X3
M = [[0, 1, 2],
[3, 4, 5]]
#size: 2X2X3
v*M = [[[0, 0, 0],
[0, 0, 0]],
[[0, 1, 2],
[3, 4, 5]]]
For two vectors v1 and v2, I can use torch.bmm(v1.view(1, -1, 1), v2.view(1, 1, -1)). This can be easily extended for a batch of vectors. However, I am not able to find a solution for vector-matrix case. Also, I need to do this operation for batches of vectors and matrices.
You can use torch.einsum operator:
torch.einsum('bp,bqr->bpqr', v, M) # batch-wise operation v.shape=(b,p) M.shape=(b,q,r)
torch.einsum('p,qr->pqr', v, M) # cross-batch operation
I was able to do it with following code.
Single vector and matrix
v = torch.arange(3)
M = torch.arange(8).view(2, 4)
# v: tensor([0, 1, 2])
# M: tensor([[0, 1, 2, 3],
# [4, 5, 6, 7]])
torch.mm(v.unsqueeze(1), M.view(1, 2*4)).view(3,2,4)
tensor([[[ 0, 0, 0, 0],
[ 0, 0, 0, 0]],
[[ 0, 1, 2, 3],
[ 4, 5, 6, 7]],
[[ 0, 2, 4, 6],
[ 8, 10, 12, 14]]])
For a batch of vectors and matrices, it can be easily extended using torch.bmm.
v = torch.arange(batch_size*2).view(batch_size, 2)
M = torch.arange(batch_size*3*4).view(batch_size, 3, 4)
torch.bmm(v.unsqueeze(2), M.view(-1, 1, 3*4)).view(-1, 2, 3, 4)
If [batch_size, z, x, y] is the shape of the target matrix, another solution is building two matrices of this shape with appropriate elements in each position and then apply an elementwise multiplication. It works fine with batch of vectors:
# input matrices
batch_size = 2
x1 = torch.Tensor([0,1])
x2 = torch.Tensor([[0,1,2],
[3,4,5]])
x1 = x1.unsqueeze(0).repeat((batch_size, 1))
x2 = x2.unsqueeze(0).repeat((batch_size, 1, 1))
# dimensions
b = x1.shape[0]
z = x1.shape[1]
x = x2.shape[1]
y = x2.shape[2]
# solution
mat1 = x1.reshape(b, z, 1, 1).repeat(1, 1, x, y)
mat2 = x2.reshape(b,1,x,y).repeat(1, z, 1, 1)
mat1*mat2
I have my data tensor which is of the shape [batch_size,512] and I have a constant matrix with values only of 0 and 1 which has the shape [256,512].
I would like to compute efficiently for each batch the sum of the products of my vector (second dimension of the data tensor) only for the entries which are 1 and not 0.
An explaining example:
let us say I have 1-sized batch: the data tensor has the values [5,4,3,7,8,2] and my constant matrix has the values:
[0,1,1,0,0,0]
[1,0,0,0,0,0]
[1,1,1,0,0,1]
it means that I would like to compute for the first row 4*3, for the second 5 and for the third 5*4*3*2.
and in total for this batch, I get 4*3+5+5*4*3*2 which equals to 137.
Currently, I do it by iterating over all the rows, compute elementwise the product of my data and constant-matrix-row and then sum, which runs pretty slow.
How about something like this:
import tensorflow as tf
# Two-element batch
data = [[5, 4, 3, 7, 8, 2],
[4, 2, 6, 1, 6, 8]]
mask = [[0, 1, 1, 0, 0, 0],
[1, 0, 0, 0, 0, 0],
[1, 1, 1, 0, 0, 1]]
with tf.Graph().as_default(), tf.Session() as sess:
# Data as tensors
d = tf.constant(data, dtype=tf.int32)
m = tf.constant(mask, dtype=tf.int32)
# Tile data as needed
dd = tf.tile(d[:, tf.newaxis], (1, tf.shape(m)[0], 1))
mm = tf.tile(m[tf.newaxis, :], (tf.shape(d)[0], 1, 1))
# Replace values with 1 wherever the mask is 0
w = tf.where(tf.cast(mm, tf.bool), dd, tf.ones_like(dd))
# Multiply row-wise and sum
result = tf.reduce_sum(tf.reduce_prod(w, axis=-1), axis=-1)
print(sess.run(result))
Output:
[137 400]
EDIT:
If you input data is a single vector then you would just have:
import tensorflow as tf
# Two-element batch
data = [5, 4, 3, 7, 8, 2]
mask = [[0, 1, 1, 0, 0, 0],
[1, 0, 0, 0, 0, 0],
[1, 1, 1, 0, 0, 1]]
with tf.Graph().as_default(), tf.Session() as sess:
# Data as tensors
d = tf.constant(data, dtype=tf.int32)
m = tf.constant(mask, dtype=tf.int32)
# Tile data as needed
dd = tf.tile(d[tf.newaxis], (tf.shape(m)[0], 1))
# Replace values with 1 wherever the mask is 0
w = tf.where(tf.cast(m, tf.bool), dd, tf.ones_like(dd))
# Multiply row-wise and sum
result = tf.reduce_sum(tf.reduce_prod(w, axis=-1), axis=-1)
print(sess.run(result))
Output:
137
I have a tensor for example : X = [1, 1, 0, 0, 1, 2, 2, 0, 1, 2].
And what I want is to reduce this tensor X to a tensor such as: Y = [3, 4, 3].
Where Y in position 0 is the count of how many 0s there are in X, and the position 1 how many 1s, so on and so forth.
What I'm doing right now is iterating through this tensor using the tf.where function. But this doesn`t seem elegant, and there must be a better way to do it.
Thanks.
You are looking for tf.unique_with_counts.
import tensorflow as tf
X = tf.constant([1, 1, 0, 0, 1, 2, 2, 0, 1, 2])
op = tf.unique_with_counts(X)
sess = tf.InteractiveSession()
res = sess.run(op)
print(res.count)
# [4 3 3]
Beware that tf.bincount only handle positive integers. If your input tensor is not of integer type, or contains negative values, you must use tf.unique_with_count. Otherwise bincount is fine and to the point.
I think you are looking for Y = tf.bincount(X):
X = tf.constant([1, 1, 0, 0, 1, 2, 2, 0, 1, 2])
Y = tf.bincount(X)
sess = tf.InteractiveSession()
tf.global_variables_initializer().run()
Y.eval()
# output
#[3, 4, 3]
For negative integers you can use:
tf.bincount(X + tf.abs(tf.reduce_min(X)) )