In the model I want to build, there are two placeholders
x = tf.placeholder('float32', shape=[1000, 10])
tags = tf.placeholder('int32', shape=[1000, 1])
(1000 is just the number of examples)
x holds the inputs to neural networks, tags determines which one of the three neural networks will be used to compute the output.
w1 = tf.get_variable('w1', [10, 1], tf.truncated_normal_initializer())
w2 = tf.get_variable('w2', [10, 1], tf.truncated_normal_initializer())
w3 = tf.get_variable('w3', [10, 1], tf.truncated_normal_initializer())
def nn_1(): return tf.matmul(x, w1)
def nn_2(): return tf.matmul(x, w2)
def nn_3(): return tf.matmul(x, w3)
I want to find an elegant way to implement a TensorFlow graph which can compute the output of an x given its tag.
[x1, x2, x3, ..., xn]
[1, 2, 3, ..., 1]
[nn_1(x), nn_2(x), nn_3(x), ..., nn_1(x)]
If x and tags are not arrays, I can implement it with tf.case, for example,
a = tf.placeholder('int32')
b = tf.placeholder('int32')
result = tf.case(
{
tf.equal(b, 1): a + 1,
tf.equal(b, 2): a + 2
})
But I have no idea how to do when x and tags are arrays.
You can use some math trick to do the job.
let's say you want to implement the code you implemented but with a and b being arrays.
First, you compute an array of condition.
This would be the condition that must be true in order to apply the operation.
Typically conditions use "less", "equal", "greater" operation or a logical composition of those.
You can use tf.bitwise or tf.math.logical* for logical operation and tf.math for the others.
The condition must be a boolean array. 1 if the condition is true, 0 if false.
After that, you initialize the result array with the default value (what is in the "else" statement)
To apply the condition you simply multiply the condition array with the value you want to assign.
The code would be something like this.
//default value
result = tf.zeros(tf.shape(a)[0])
condition = tf.equal(b, index)
condition = tf.cast(condition, tf.float32)
result = tf.multiply(condition, a) + index
If you want to use tag as index of the functions array you need to use a 2d array. Create a 2d array of all possible combination nn X x.
This array will contain nn_j(x[i]) for each i,j couple.
To do this you need to create an array x X nn X 2 array.
First, expand x and create an array with x X nn array
if your x is x=[0,2,1] and len(n) = 2 then you need to have x_nn = [[0,0], [2,2], [1,1]].
nn_x = x
nn_x = tf.expand_dims(nn_x,0)
nn_x = tf.tile(nn_x, [len(nn), 1])
Then you create a 2d array with the same shape having the index of nn.
For the arrey used early index2d = [[0,1],[0,1],[0,1]]
index = tf.linspace(0,len(nn)-1)
index2d = tf.expand_dims(index,0)
index2d = tf.tile(index2d, tf.shape(x)[0])
Then you need to stack these and arrays, move the first dimension at the last place, then flat along axis 0 and 1.
In this way you will have map2d = [[0,0],[0,1],[2,0],[2,1],[1,0],[1,1]]
For each couple the first is the value of x, second is the index of the nn
Then you map this 2d array using the tf.map_fn function. write something like
tf.map_fn(t => [nn[t[1]](t[0]), t[1]], map2d)
Now you have all possible value of nn for each x
At this point, you can reshape back map2d compare map2d[:,:,1] with you tag and select the one that is equal.
#reshape map2d
# ...
# transform tag
tag2d = tag
tag2d = tf.expand_dims(tag2d,0)
tag2d = tf.tile(tag2d, [len(nn), 1])
result = tf.equal(tag, map2d[:,:,1])
the result will have only one non-zero value for each column
result = tf.multiply(result, map2d[:,:,0])
result = tf.reduce_max(result, [1])
I didn't try the code, but the mechanism should work.
Hope this help
Related
Arrays of labels of objects and distances to that objects are given. I want to apply knn to find the label of prediction. I want to use np.bincount for that. However, I don't understand how to use this.
See some example
labels = [[1,1,2,0,0,3,3,3,5,1,3],
[1,1,2,0,0,3,3,3,5,1,3]]
weights= [[0,0,0,0,0,0,0,0,1,0,0],
[0,0,0,0,0,0,0,0,1,0,0]]
Imagine 10 nearest neighbors for 2 objects are given and their labels and distances are given above. So I want the output as [5,5], because only neighbours with that label have nonzero weight. I am doing the next thing:
eps = 1e-5
lab_weight = np.array(list(zip(labels, weights)))
predict = np.apply_along_axis(lambda x: np.bincount(x[0], weights=x[1]).argmax(), 2, lab_weight)
I expect that x will correspond to [[1,1,2,0,0,3,3,3,5,1,3], [0,0,0,0,0,0,0,0,1,0,0]], but it won't. Other axis parameters are not working too. How can I achieve the goal? I want to use numpy functions and avoid python loops.
The next solution gives me desired result:
labels = [[1,1,2,0,0,3,3,3,5,1,3],
[1,1,2,0,0,3,3,3,5,1,3]]
weights= [[0,0,0,0,0,0,0,0,1,0,0],
[0,0,0,0,0,0,0,0,1,0,0]]
length = len(labels[0])
lab_weight = np.hstack((labels, weights))
predict = np.apply_along_axis(lambda x: np.bincount(x[:length], weights=x[length:]).argmax(), 1, lab_weight)
The problem with your code is that you attempt to use your
function to 2-D slices of your array, whereas apply_along_axis
applies the given function to 1-D slices.
So your code generates an exception: ValueError: object of too small
depth for desired array.
To apply your function to 2-D slices, use a list comprehension based on
np.rollaxis and then create a Numpy array from it:
result = np.array([ np.bincount(x[0], weights=x[1]).argmax()
for x in np.rollaxis(lab_weight, 2) ])
The result, for your array, is:
array([1, 1, 2, 0, 0, 3, 3, 3, 5, 1, 3], dtype=int64)
To trace, for each interation, the source array, intermediate results
and the final result, run:
i = 0
for x in np.rollaxis(lab_weight, 2):
print(f' i: {i}\n{x}'); i += 1
bc = np.bincount(x[0], weights=x[1])
bcm = bc.argmax()
print(bc, bcm)
I'm trying to create a method for evaluating co-ordinates for a project that's due in about a week.
Assuming that I'm working in a 3D cartesian co-ordinate system - whose values are stored as rows in a numpy array. I am trying to read if 'z' (n[i, 2]) values exist given the corresponding, predetermined 'x' (n[i,0]) and 'y' (n[i,1]) values.
In the case where the values that are assigned are scalars, I am content to think that:
# Given that n is some numpy array
x, y = 2,3
out = []
for i in range(0,n.shape[0]):
if n[i, 0] == x and n[i,1] == y:
out.append(n[i,2])
However, where the sorrow comes in is having to check if the values in another numpy array are in the original numpy array 'n'.
# Given that n is the numpy array that is to be searched
# Given that x contains the 'search elements'
out = []
for i in range(0,n.shape[0]):
for j in range(0, x.shape[0]):
if n[i, 0] == x[j,0] and n[i,1] == x[j,1]:
out.append(n[i,2])
The issue with doing it this way is that the 'n' matrix in my application may well be in excess of 100 000 lines long.
Is there a more efficient way of performing this function?
This might be more efficient than nested loops:
out = []
for row in x:
idx = np.equal(n[:,:2], row).all(1)
out.extend(n[idx,2].tolist())
Note this assumes that x is of shape (?, 2). Otherwise, if it has more than two columns, just change row to row[:2] in the loop body.
Numpythonic solution without loops.
This solution works in case the x and y coordinates are non-negative.
import numpy as np
# Using a for x and b for n, to avoid confusion with x,y coordinates and array names
a = np.array([[1,2],[3,4]])
b = np.array([[1,2,10],[1,2,11],[3,4,12],[5,6,13],[3,4,14]])
# Adjust the shapes by taking the z coordinate as 0 in a and take the dot product with b transposed
a = np.insert(a,2,0,axis=1)
dot_product = np.dot(a,b.T)
# Reshape a**2 to check the dot product values corresponding to exact values in the x, y coordinates
sum_reshaped = np.sum(a**2,axis=1).reshape(a.shape[0],1)
# Match for values for indivisual elements in a. Can be used if you want z coordinates corresponding to some x, y separately
indivisual_indices = ( dot_product == np.tile(sum_reshaped,b.shape[0]) )
# Take OR of column values and take z if atleast one x,y present
indices = np.any(indivisual_indices, axis=0)
print(b[:,2][indices]) # prints [10 11 12 14]
TL;DR How to scale part of tensor by 2 (row-indices present in a tf list)
Details:
indices_of_scaling_ids: Stores list of row_ids
Tensor("Squeeze:0", dtype=int64, device=/device:GPU:0)
[1, 4, 5, 6, 12]
emb_inputs = tf.nn.embedding_lookup(embedding, self.all_rows)
#tensor with shape (batch_size=4, all_row_len, emb_size=128)
So, for every self.all_rows, the emb_inputs is evaluated.
Question / Challenge faced: I need to scale the emb_inputs by 2.0 for every row_ids mentioned in indices_of_scaling_ids.
I have tried various splicing things, but can't seem to get to a nice solution. Can someone suggest? Thanks
N.B. Beginner at Tensorflow
Try with something like this:
SCALE = 2
emb_inputs = ...
indices_of_scaling_ids = ...
emb_shape = tf.shape(emb_inputs)
# Select indices in boolean array
r = tf.range(emb_shape[1])
mask = tf.reduce_any(tf.equal(r[:, tf.newaxis], indices_of_scaling_ids), axis=1)
# Tile the mask
mask = tf.tile(mask[tf.newaxis, :, tf.newaxis], (emb_shape[0], 1, emb_shape[2]))
# Choose scaled or not depending on indices
result = tf.where(mask, SCALE * emb_inputs, emb_inputs)
Good afternoon.
I continue to have issues with updating random elements in tensorflow by index.
I want to randomly choose indices (half of all, for instance), and then set to zero elements correspond to that indices.
Here's the problematic part:
with tf.variable_scope("foo", reuse=True):
temp_var = tf.get_variable("W")
size_2a = tf.get_variable("b")
s1 = tf.shape(temp_var).eval()[0]
s2 = tf.shape(size_2a).eval()[0]
row_indices = tf.random_uniform(dtype=tf.int32, minval=0, maxval = s1 - 1, shape=[s1]).eval()
col_indices = tf.random_uniform(dtype=tf.int32, minval=0, maxval = s2 - 1, shape=[s2]).eval()
ones_mask = tf.ones([s1,s2])
# turn 'ones_mask' into 1d variable since "scatter_update" supports linear indexing only
ones_flat = tf.Variable(tf.reshape(ones_mask, [-1]))
# no automatic promotion, so make updates float32 to match ones_mask
updates = tf.zeros(shape=(s1,), dtype=tf.float32)
# get linear indices
linear_indices = row_indices*s2 + tf.reshape(col_indices,s1*s2)
ones_flat = tf.scatter_update(ones_flat, linear_indices/2, updates)
#I want to set to zero only half of all elements,that's why linear_indices/2
# convert back into original shape
ones_mask = tf.reshape(ones_flat, ones_mask.get_shape())
It gives me ValueError: Cannot reshape a tensor with 10 elements to shape [784,10] (7840 elements) for 'foo_1/Reshape_1' (op: 'Reshape') with input shapes: [10], [2]., but I don't know how to be here without reshaping (I tried to reshape to both s1 and s2, no use)
I have already read these topics:Update values of a matrix variable in tensorflow, advanced indexing (feed_dict doesn't seem to work in my case), python numpy ValueError: operands could not be broadcast together with shapes and practically everything on the subject on stackoverflow =(
My question is in two connected parts:
How do I calculate the max along a certain axis of a tensor? For example, if I have
x = tf.constant([[1,220,55],[4,3,-1]])
I want something like
x_max = tf.max(x, axis=1)
print sess.run(x_max)
output: [220,4]
I know there is a tf.argmax and a tf.maximum, but neither give the maximum value along an axis of a single tensor. For now I have a workaround:
x_max = tf.slice(x, begin=[0,0], size=[-1,1])
for a in range(1,2):
x_max = tf.maximum(x_max , tf.slice(x, begin=[0,a], size=[-1,1]))
But it looks less than optimal. Is there a better way to do this?
Given the indices of an argmax of a tensor, how do I index into another tensor using those indices? Using the example of x above, how do I do something like the following:
ind_max = tf.argmax(x, dimension=1) #output is [1,0]
y = tf.constant([[1,2,3], [6,5,4])
y_ = y[:, ind_max] #y_ should be [2,6]
I know slicing, like the last line, does not exist in TensorFlow yet (#206).
My question is: what is the best workaround for my specific case (maybe using other methods like gather, select, etc.)?
Additional information: I know x and y are going to be two dimensional tensors only!
The tf.reduce_max() operator provides exactly this functionality. By default it computes the global maximum of the given tensor, but you can specify a list of reduction_indices, which has the same meaning as axis in NumPy. To complete your example:
x = tf.constant([[1, 220, 55], [4, 3, -1]])
x_max = tf.reduce_max(x, reduction_indices=[1])
print sess.run(x_max) # ==> "array([220, 4], dtype=int32)"
If you compute the argmax using tf.argmax(), you could obtain the the values from a different tensor y by flattening y using tf.reshape(), converting the argmax indices into vector indices as follows, and using tf.gather() to extract the appropriate values:
ind_max = tf.argmax(x, dimension=1)
y = tf.constant([[1, 2, 3], [6, 5, 4]])
flat_y = tf.reshape(y, [-1]) # Reshape to a vector.
# N.B. Handles 2-D case only.
flat_ind_max = ind_max + tf.cast(tf.range(tf.shape(y)[0]) * tf.shape(y)[1], tf.int64)
y_ = tf.gather(flat_y, flat_ind_max)
print sess.run(y_) # ==> "array([2, 6], dtype=int32)"
As of TensorFlow 1.10.0-dev20180626, tf.reduce_max accepts axis and keepdims keyword arguments offering the similar functionality of numpy.max.
In [55]: x = tf.constant([[1,220,55],[4,3,-1]])
In [56]: tf.reduce_max(x, axis=1).eval()
Out[56]: array([220, 4], dtype=int32)
To have a resultant tensor of the same dimension as the input tensor, use keepdims=True
In [57]: tf.reduce_max(x, axis=1, keepdims=True).eval()Out[57]:
array([[220],
[ 4]], dtype=int32)
If the axis argument is not explicitly specified then the tensor level maximum element is returned (i.e. all axes are reduced).
In [58]: tf.reduce_max(x).eval()
Out[58]: 220