My question is in two connected parts:
How do I calculate the max along a certain axis of a tensor? For example, if I have
x = tf.constant([[1,220,55],[4,3,-1]])
I want something like
x_max = tf.max(x, axis=1)
print sess.run(x_max)
output: [220,4]
I know there is a tf.argmax and a tf.maximum, but neither give the maximum value along an axis of a single tensor. For now I have a workaround:
x_max = tf.slice(x, begin=[0,0], size=[-1,1])
for a in range(1,2):
x_max = tf.maximum(x_max , tf.slice(x, begin=[0,a], size=[-1,1]))
But it looks less than optimal. Is there a better way to do this?
Given the indices of an argmax of a tensor, how do I index into another tensor using those indices? Using the example of x above, how do I do something like the following:
ind_max = tf.argmax(x, dimension=1) #output is [1,0]
y = tf.constant([[1,2,3], [6,5,4])
y_ = y[:, ind_max] #y_ should be [2,6]
I know slicing, like the last line, does not exist in TensorFlow yet (#206).
My question is: what is the best workaround for my specific case (maybe using other methods like gather, select, etc.)?
Additional information: I know x and y are going to be two dimensional tensors only!
The tf.reduce_max() operator provides exactly this functionality. By default it computes the global maximum of the given tensor, but you can specify a list of reduction_indices, which has the same meaning as axis in NumPy. To complete your example:
x = tf.constant([[1, 220, 55], [4, 3, -1]])
x_max = tf.reduce_max(x, reduction_indices=[1])
print sess.run(x_max) # ==> "array([220, 4], dtype=int32)"
If you compute the argmax using tf.argmax(), you could obtain the the values from a different tensor y by flattening y using tf.reshape(), converting the argmax indices into vector indices as follows, and using tf.gather() to extract the appropriate values:
ind_max = tf.argmax(x, dimension=1)
y = tf.constant([[1, 2, 3], [6, 5, 4]])
flat_y = tf.reshape(y, [-1]) # Reshape to a vector.
# N.B. Handles 2-D case only.
flat_ind_max = ind_max + tf.cast(tf.range(tf.shape(y)[0]) * tf.shape(y)[1], tf.int64)
y_ = tf.gather(flat_y, flat_ind_max)
print sess.run(y_) # ==> "array([2, 6], dtype=int32)"
As of TensorFlow 1.10.0-dev20180626, tf.reduce_max accepts axis and keepdims keyword arguments offering the similar functionality of numpy.max.
In [55]: x = tf.constant([[1,220,55],[4,3,-1]])
In [56]: tf.reduce_max(x, axis=1).eval()
Out[56]: array([220, 4], dtype=int32)
To have a resultant tensor of the same dimension as the input tensor, use keepdims=True
In [57]: tf.reduce_max(x, axis=1, keepdims=True).eval()Out[57]:
array([[220],
[ 4]], dtype=int32)
If the axis argument is not explicitly specified then the tensor level maximum element is returned (i.e. all axes are reduced).
In [58]: tf.reduce_max(x).eval()
Out[58]: 220
Related
I am trying to get the dotproduct of two arrays in python using the numpy package. I get as output an array of size (n,). It says that my array has no column while I do see the results when I print it. Why does my array have no column and how do I fix this?
My goal is to calculate y - np.dot(x,b). The issue is that y is (124, 1) while np.dot(x,b) is (124,)
Thanks
It seems that you are trying to subtract two arrays of a different shape. Fortunately, it is off by a single additional axis, so there are two ways of handling it.
(1) You slice the y array to match the shape of the dot(x,b) array:
y = y[:,0]
print(y-np.dot(x,b))
(2) You add an additional axis on the np.dot(x,b) array:
dot = np.dot(x,b)
dot = dot[:,None]
print(y-dot)
Hope this helps
it may depends on the dimension of your array
For example :
a = [1, 0]
b = [[4, 1], [2, 2]]
c = np.dot(a,b)
gives
array([4, 1])
and its shape is (2,)
but if you change a like :
a = [[1, 0],[1,1]]
then result is :
array([[4, 1],
[6, 3]])
and its shape is (2,2)
Arrays of labels of objects and distances to that objects are given. I want to apply knn to find the label of prediction. I want to use np.bincount for that. However, I don't understand how to use this.
See some example
labels = [[1,1,2,0,0,3,3,3,5,1,3],
[1,1,2,0,0,3,3,3,5,1,3]]
weights= [[0,0,0,0,0,0,0,0,1,0,0],
[0,0,0,0,0,0,0,0,1,0,0]]
Imagine 10 nearest neighbors for 2 objects are given and their labels and distances are given above. So I want the output as [5,5], because only neighbours with that label have nonzero weight. I am doing the next thing:
eps = 1e-5
lab_weight = np.array(list(zip(labels, weights)))
predict = np.apply_along_axis(lambda x: np.bincount(x[0], weights=x[1]).argmax(), 2, lab_weight)
I expect that x will correspond to [[1,1,2,0,0,3,3,3,5,1,3], [0,0,0,0,0,0,0,0,1,0,0]], but it won't. Other axis parameters are not working too. How can I achieve the goal? I want to use numpy functions and avoid python loops.
The next solution gives me desired result:
labels = [[1,1,2,0,0,3,3,3,5,1,3],
[1,1,2,0,0,3,3,3,5,1,3]]
weights= [[0,0,0,0,0,0,0,0,1,0,0],
[0,0,0,0,0,0,0,0,1,0,0]]
length = len(labels[0])
lab_weight = np.hstack((labels, weights))
predict = np.apply_along_axis(lambda x: np.bincount(x[:length], weights=x[length:]).argmax(), 1, lab_weight)
The problem with your code is that you attempt to use your
function to 2-D slices of your array, whereas apply_along_axis
applies the given function to 1-D slices.
So your code generates an exception: ValueError: object of too small
depth for desired array.
To apply your function to 2-D slices, use a list comprehension based on
np.rollaxis and then create a Numpy array from it:
result = np.array([ np.bincount(x[0], weights=x[1]).argmax()
for x in np.rollaxis(lab_weight, 2) ])
The result, for your array, is:
array([1, 1, 2, 0, 0, 3, 3, 3, 5, 1, 3], dtype=int64)
To trace, for each interation, the source array, intermediate results
and the final result, run:
i = 0
for x in np.rollaxis(lab_weight, 2):
print(f' i: {i}\n{x}'); i += 1
bc = np.bincount(x[0], weights=x[1])
bcm = bc.argmax()
print(bc, bcm)
Let's say I want to create torch.tensor object of size [2,3] filled with random elements, and I intend to use this matrix in the network and optimize it's values. However, I want to update only some of the the values in the matrix.
I know that it can be done for a tensor by setting up parameter requires_grad To True or False. However, the following code
z = torch.rand([2,3], requires_grad=True)
z[-1][-1].requires_grad=False
does not work as expected
RuntimeError: you can only change requires_grad flags of leaf variables. If you want to use a computed variable in a subgraph that doesn't require differentiation use var_no_grad = var.detach().
How to fix this RuntimeError? How to initialize torch tensor and then define which elements there would have requires_grad =True?
If I write code in a similar manner:
z = torch.rand([2,3], requires_grad=False)
z[-1][-1].requires_grad=True
There will be no error, but no change of the requires_grad as well.
It does not really make much sense to have a single tensor which requires_grad for only part of its entries.
Why not have two separate tensors one that us updated (requires_grad=True) and another one fixed (requires_grad=False)? You can then merge them for computational ease:
fixed = torch.rand([2, 3], require_grad=False)
upd = torch.rand([2, 3], require_grad=True)
mask = torch.tensor([[0, 1, 0], [1, 0, 1]], require_grad=False) # how to combine the two
# combine them using fixed "mask":
z = mask * fixed + (1-mask) * upd
You can obviously have other methods of combining fixed and upd other than using a binary mask.
For example, if upd occupies the first two columns of z and fixed the rest, then:
fixed = torch.rand([2, 1], require_grad=False)
upd = torch.rand([2, 2], require_grad=True)
# combine them using concatination
z = torch.cat((upd, fixed),dim=1)
Or, if you know the indices
fidx = torch.tensor([0, 2], dtype=torch.long)
uidx = torch.tensor([1, 3, 4, 5], dtype=torch.long)
fixed = torch.rand([2,], require_grad=False)
upd = torch.rand([4,], require_grad=True)
z = torch.empty([2, 3])
z[fidx] = fixed
z[uidx] = upd
In the model I want to build, there are two placeholders
x = tf.placeholder('float32', shape=[1000, 10])
tags = tf.placeholder('int32', shape=[1000, 1])
(1000 is just the number of examples)
x holds the inputs to neural networks, tags determines which one of the three neural networks will be used to compute the output.
w1 = tf.get_variable('w1', [10, 1], tf.truncated_normal_initializer())
w2 = tf.get_variable('w2', [10, 1], tf.truncated_normal_initializer())
w3 = tf.get_variable('w3', [10, 1], tf.truncated_normal_initializer())
def nn_1(): return tf.matmul(x, w1)
def nn_2(): return tf.matmul(x, w2)
def nn_3(): return tf.matmul(x, w3)
I want to find an elegant way to implement a TensorFlow graph which can compute the output of an x given its tag.
[x1, x2, x3, ..., xn]
[1, 2, 3, ..., 1]
[nn_1(x), nn_2(x), nn_3(x), ..., nn_1(x)]
If x and tags are not arrays, I can implement it with tf.case, for example,
a = tf.placeholder('int32')
b = tf.placeholder('int32')
result = tf.case(
{
tf.equal(b, 1): a + 1,
tf.equal(b, 2): a + 2
})
But I have no idea how to do when x and tags are arrays.
You can use some math trick to do the job.
let's say you want to implement the code you implemented but with a and b being arrays.
First, you compute an array of condition.
This would be the condition that must be true in order to apply the operation.
Typically conditions use "less", "equal", "greater" operation or a logical composition of those.
You can use tf.bitwise or tf.math.logical* for logical operation and tf.math for the others.
The condition must be a boolean array. 1 if the condition is true, 0 if false.
After that, you initialize the result array with the default value (what is in the "else" statement)
To apply the condition you simply multiply the condition array with the value you want to assign.
The code would be something like this.
//default value
result = tf.zeros(tf.shape(a)[0])
condition = tf.equal(b, index)
condition = tf.cast(condition, tf.float32)
result = tf.multiply(condition, a) + index
If you want to use tag as index of the functions array you need to use a 2d array. Create a 2d array of all possible combination nn X x.
This array will contain nn_j(x[i]) for each i,j couple.
To do this you need to create an array x X nn X 2 array.
First, expand x and create an array with x X nn array
if your x is x=[0,2,1] and len(n) = 2 then you need to have x_nn = [[0,0], [2,2], [1,1]].
nn_x = x
nn_x = tf.expand_dims(nn_x,0)
nn_x = tf.tile(nn_x, [len(nn), 1])
Then you create a 2d array with the same shape having the index of nn.
For the arrey used early index2d = [[0,1],[0,1],[0,1]]
index = tf.linspace(0,len(nn)-1)
index2d = tf.expand_dims(index,0)
index2d = tf.tile(index2d, tf.shape(x)[0])
Then you need to stack these and arrays, move the first dimension at the last place, then flat along axis 0 and 1.
In this way you will have map2d = [[0,0],[0,1],[2,0],[2,1],[1,0],[1,1]]
For each couple the first is the value of x, second is the index of the nn
Then you map this 2d array using the tf.map_fn function. write something like
tf.map_fn(t => [nn[t[1]](t[0]), t[1]], map2d)
Now you have all possible value of nn for each x
At this point, you can reshape back map2d compare map2d[:,:,1] with you tag and select the one that is equal.
#reshape map2d
# ...
# transform tag
tag2d = tag
tag2d = tf.expand_dims(tag2d,0)
tag2d = tf.tile(tag2d, [len(nn), 1])
result = tf.equal(tag, map2d[:,:,1])
the result will have only one non-zero value for each column
result = tf.multiply(result, map2d[:,:,0])
result = tf.reduce_max(result, [1])
I didn't try the code, but the mechanism should work.
Hope this help
TL;DR How to scale part of tensor by 2 (row-indices present in a tf list)
Details:
indices_of_scaling_ids: Stores list of row_ids
Tensor("Squeeze:0", dtype=int64, device=/device:GPU:0)
[1, 4, 5, 6, 12]
emb_inputs = tf.nn.embedding_lookup(embedding, self.all_rows)
#tensor with shape (batch_size=4, all_row_len, emb_size=128)
So, for every self.all_rows, the emb_inputs is evaluated.
Question / Challenge faced: I need to scale the emb_inputs by 2.0 for every row_ids mentioned in indices_of_scaling_ids.
I have tried various splicing things, but can't seem to get to a nice solution. Can someone suggest? Thanks
N.B. Beginner at Tensorflow
Try with something like this:
SCALE = 2
emb_inputs = ...
indices_of_scaling_ids = ...
emb_shape = tf.shape(emb_inputs)
# Select indices in boolean array
r = tf.range(emb_shape[1])
mask = tf.reduce_any(tf.equal(r[:, tf.newaxis], indices_of_scaling_ids), axis=1)
# Tile the mask
mask = tf.tile(mask[tf.newaxis, :, tf.newaxis], (emb_shape[0], 1, emb_shape[2]))
# Choose scaled or not depending on indices
result = tf.where(mask, SCALE * emb_inputs, emb_inputs)