I'm using add_metric trying to create a custom metric that computes top 3 accuracy for a classifier. Here's as far as I got:
def custom_metrics(labels, predictions):
# labels => Tensor("fifo_queue_DequeueUpTo:422", shape=(?,), dtype=int64)
# predictions => {
# 'logits': <tf.Tensor 'dnn/logits/BiasAdd:0' shape=(?, 26) dtype=float32>,
# 'probabilities': <tf.Tensor 'dnn/head/predictions/probabilities:0' shape=(?, 26) dtype=float32>,
# 'class_ids': <tf.Tensor 'dnn/head/predictions/ExpandDims:0' shape=(?, 1) dtype=int64>,
# 'classes': <tf.Tensor 'dnn/head/predictions/str_classes:0' shape=(?, 1) dtype=string>
# }
Looking at the implementation of existing tf.metrics, everything is implemented using tf ops. How could I implement top 3 accuracy?
If you want to implement it yourself tf.nn.in_top_k is very useful - it returns a boolean array which indicates if target is within the top k predictions. You just have to take the mean of the result:
def custom_metrics(labels, predictions):
return tf.metrics.mean(tf.nn.in_top_k(predictions=predictions, targets=labels, k=3))
You can also import it:
from tf.keras.metrics import top_k_categorical_accuracy
Related
I have two tensors of size
A <tf.Tensor 'sequential_12/my_layer_56/add:0' shape=(?, 300, 2) dtype=float32>
and
B <tf.Tensor 'input_82:0' shape=(?, 2, 2) dtype=float32>
Now, I would like to multiply them in the sense of the usual matrix row-column product to obtain
A * B of size (?, 300, 2), so I would be doing the matrix product only over the second and third dimension. How can I achieve this?
I tried to use tf.tensordot with different axes specifications, but it did not work so far. For example I tried
tf.tensordot(A,B,axes=[[2], [0]])
but this produces a tensor of the following form
<tf.Tensor 'Tensordot_10:0' shape=(?, 300, 2, 2) dtype=float32>
Maybe try tf.matmul:
import tensorflow as tf
samples = 1
A = tf.random.normal((samples, 300, 2))
B = tf.random.normal((samples, 2, 2))
print(tf.matmul(A, B).shape)
# (1, 300, 2)
I am using DeepCTR (version 0.7.5) keras library to predict ctr (using DeepFM)
https://deepctr-doc.readthedocs.io/en/latest/deepctr.models.deepfm.html.
Here is a small example of the code to fit the model:
#Imports, then feature preperation...
model = DeepFM(linear_feature_columns, dnn_feature_columns, task='binary')
model.compile(optimizer, loss)
train_model_input = [train_df[name] for name in feature_names]
model.fit(x=train_model_input, y=train_df[TARGET].values, validation_split=0.3)
But when I try the following:
e = shap.DeepExplainer(model, test_df.head(50))
I get the following error:
ValueError: Cannot feed value of shape (50, 17) for Tensor 'x:0', which has shape '(?, 1)'
I looked all over google and tried playing alot with the inputs shape and the SHAP API but nothing worked for me.
Additional info:
The model inputs format (17 values) is:
[<tf.Tensor 'x:0' shape=(?, 1) dtype=int32>, <tf.Tensor 'x1:0' shape=(?, 1) dtype=int32>...
And outputs is:
<tf.Tensor 'prediction_layer/Reshape:0' shape=(?, 1) dtype=float32>
I have seen similar problems described on stackoverflow, but anything I found fit/solve my problem.
I have some reinforcement learning task, when I want to use two neural networks to steers two degrees of freedom.
I have a code like this with two neural network:
def reset_graph(seed=42):
tf.reset_default_graph()
tf.set_random_seed(seed)
np.random.seed(seed)
reset_graph()
n_inputs = 10
n_hidden = 8
n_outputs = 3
learning_rate = 0.0025
initializer = tf.contrib.layers.variance_scaling_initializer()
X1 = tf.placeholder(tf.float32, shape=[None, n_inputs],name='X1')
hidden = tf.layers.dense(X1, 10, activation=tf.nn.tanh,name = 'hidden1', kernel_initializer=initializer)
logits1 = tf.layers.dense(hidden, n_outputs,name='logit1')
outputs1 = tf.nn.softmax(logits1,name='out1')
action1 = tf.multinomial(logits1, num_samples=1,name='action1')
cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels= action1[0], logits=logits1,name='cross_e1')
optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate,name='opt1')
grads_and_vars = optimizer.compute_gradients(cross_entropy)
gradients = [grad for grad, variable in grads_and_vars]
gradient_placeholders = []
grads_and_vars_feed = []
for grad, variable in grads_and_vars:
gradient_placeholder = tf.placeholder(tf.float32)
gradient_placeholders.append(gradient_placeholder)
grads_and_vars_feed.append((gradient_placeholder, variable))
training_op = optimizer.apply_gradients(grads_and_vars_feed)
X2 = tf.placeholder(tf.float32, shape=[None, n_inputs],name='X2')
initializer2 = tf.contrib.layers.variance_scaling_initializer()
hidden2 = tf.layers.dense(X2, 10, activation=tf.nn.tanh,name='hidden2', kernel_initializer=initializer2)
logits2 = tf.layers.dense(hidden2, 3,name='logit2')
outputs2 = tf.nn.softmax(logits2,name='out2')
action2 = tf.multinomial(logits2, num_samples=1,name='action2')
cross_entropy2 = tf.nn.sparse_softmax_cross_entropy_with_logits(labels= action2[0], logits=logits2,name='cross_e2')
optimizer2 = tf.train.GradientDescentOptimizer(learning_rate=0.002,name = 'opt2')
grads_and_vars2 = optimizer2.compute_gradients(cross_entropy2)
gradients2 = [grad2 for grad2, variable2 in grads_and_vars2]
gradient_placeholders2 = []
grads_and_vars_feed2 = []
for grad2, variable2 in grads_and_vars2:
gradient_placeholder2 = tf.placeholder(tf.float32)
gradient_placeholders2.append(gradient_placeholder2)
grads_and_vars_feed2.append((gradient_placeholder2, variable2))
training_op2 = optimizer2.apply_gradients(grads_and_vars_feed2)
init = tf.global_variables_initializer()
saver = tf.train.Saver()
and when I run it by :
action_val,action_val2,gradients_val,gradients_val2 = sess.run([action,action2, gradients,gradients2], feed_dict={X1: obs.reshape(1, n_inputs),X2: obs.reshape(1, n_inputs)})
I have an error:
TypeError Traceback (most recent call last)
<ipython-input-70-fb66a94fa4dc> in <module>
50 reward, done, obs = agent.step(rotor_speeds)
51
---> 52 action_val,gradients_val,action_val2,gradients_val2 = sess.run([action, gradients,action2, gradients2], feed_dict={X1: obs.reshape(1, n_inputs),X2: obs.reshape(1, n_inputs)})
...
TypeError: Fetch argument None has invalid type <class 'NoneType'>
The problem is with gradients2. When I calculate other parts of graph it work fine. For example:
action_val,action_val2,gradients_val = sess.run(([action1,action2,gradients]),feed_dict={X1: obs.reshape(1, n_inputs),X2: obs.reshape(1, n_inputs)})
it works without problems.
Also, I wonder why on graph generated from above code, the hidden layer(hidden1) and logits (logit1) of first neural network are connected to second optimizer(opt2) , because I don't see this unwanted connections in the code. Maybe it is the reason of problem, but also I don't know how to change it.
Let's look at gradients2, since it seems to cause the error:
>>> gradients2
[None,
None,
None,
None,
<tf.Tensor 'gradients_1/hidden2/MatMul_grad/tuple/control_dependency_1:0' shape=(10, 10) dtype=float32>,
<tf.Tensor 'gradients_1/hidden2/BiasAdd_grad/tuple/control_dependency_1:0' shape=(10,) dtype=float32>,
<tf.Tensor 'gradients_1/logit2/MatMul_grad/tuple/control_dependency_1:0' shape=(10, 3) dtype=float32>,
<tf.Tensor 'gradients_1/logit2/BiasAdd_grad/tuple/control_dependency_1:0' shape=(3,) dtype=float32>]
The first 4 elements are None, which explains why sess.run() fails (you cannot evaluate None).
So why does gradients2 contain None values? These values come from grads_and_vars2 so let's look at it:
>>> grads_and_vars2
[(None, <tf.Variable 'hidden1/kernel:0' shape=(10, 10) dtype=float32_ref>),
(None, <tf.Variable 'hidden1/bias:0' shape=(10,) dtype=float32_ref>),
(None, <tf.Variable 'logit1/kernel:0' shape=(10, 3) dtype=float32_ref>),
(None, <tf.Variable 'logit1/bias:0' shape=(3,) dtype=float32_ref>),
(<tf.Tensor 'gradients_1/hidden2/MatMul_grad/tuple/control_dependency_1:0' shape=(10, 10) dtype=float32>,
<tf.Variable 'hidden2/kernel:0' shape=(10, 10) dtype=float32_ref>),
(<tf.Tensor 'gradients_1/hidden2/BiasAdd_grad/tuple/control_dependency_1:0' shape=(10,) dtype=float32>,
<tf.Variable 'hidden2/bias:0' shape=(10,) dtype=float32_ref>),
(<tf.Tensor 'gradients_1/logit2/MatMul_grad/tuple/control_dependency_1:0' shape=(10, 3) dtype=float32>,
<tf.Variable 'logit2/kernel:0' shape=(10, 3) dtype=float32_ref>),
(<tf.Tensor 'gradients_1/logit2/BiasAdd_grad/tuple/control_dependency_1:0' shape=(3,) dtype=float32>,
<tf.Variable 'logit2/bias:0' shape=(3,) dtype=float32_ref>)]
The None values correspond to the variables hidden1/kernel, hidden1/bias, logit1/kernel and logit1/bias. These are the parameters of the hidden1 and logit1 dense layers. What does this mean? Consider how grads_and_vars2 was computed:
grads_and_vars2 = optimizer2.compute_gradients(cross_entropy2)
So we are asking TensorFlow to compute the gradients of cross_entropy2 with regards to all the variables in the TensorFlow graph. This includes variables which cross_entropy2 does not depend on at all, such as the parameters of the hidden1 and logit1 dense layers. This is why there are some None values.
And since sess.run() does not allow you to evaluate None values, you get an error.
To get rid of this error, you can simply filter out the None values:
gradients = [grad for grad, variable in grads_and_vars if grad is not None]
...
gradients2 = [grad2 for grad2, variable2 in grads_and_vars2 if grad2 is not None]
The error should disappear.
As a side-note: I encourage you to check out TensorFlow 2 (with tf.keras): it's much easier to use.
Given data = tf.placeholder(tf.float32, [2, None, 3]) (batch_size * time_step * feature_size), Ideally I want do tf.unstack(data, axis = 1) to get a number of tensors each of which has the shape of [2,3] so later feed them to a rnn with a for loop like
for rnn_input in rnn_inputs:
state = rnn_cell(rnn_input, state)
Using high-level API like tf.nn.dynamic_rnn is off the table so I create a work around like
import tensorflow as tf
data = tf.placeholder(tf.float32, [2, None, 3])
step_number = tf.placeholder(tf.int32, None)
loop_counter_inital = tf.constant(0)
initi_state = tf.zeros([2,3], tf.float32)
def while_condition(loop_counter, rnn_states):
return loop_counter < step_number
def while_body(loop_counter, rnn_states):
loop_counter_current = loop_counter
current_states = tf.gather_nd(data, tf.stack([tf.range(0, 2), tf.zeros([2], tf.int32)+loop_counter_current], axis=1))
cell = tf.nn.rnn_cell.BasicRNNCell(3)
rnn_states = cell(current_states, rnn_states)
return [loop_counter_current, rnn_states]
_, _states = tf.while_loop(while_condition, while_body,
loop_vars=[loop_counter_inital, initi_state],
shape_invariants=[loop_counter_inital.shape, tf.TensorShape([2, 3])])
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print (sess.run(_states, feed_dict={data:[[[3,1,6],[4,1,2]],[[5,8,1],[0,5,2]]], step_number:2 }))
The idea is to loop through each row in each of the 2D tensor of data to get the features for each time step. I got a error
First structure (2 elements): [<tf.Tensor 'while/Identity:0' shape=() dtype=int32>, <tf.Tensor 'while/Identity_1:0' shape=(2, 3) dtype=float32>]
Second structure (3 elements): [<tf.Tensor 'while/Identity:0' shape=() dtype=int32>, (<tf.Tensor 'while/basic_rnn_cell/Tanh:0' shape=(2, 3) dtype=float32>, <tf.Tensor 'while/basic_rnn_cell/Tanh:0' shape=(2, 3) dtype=float32>)]
There seems to be some related posts. None actually worked. Can anyone help?
You need to know every BasicRNNCell will implement call() with the signature (output, next_state) = call(input, state). This means that your result is a list of shapes ((?,unit),(?,unit)). So you need to do as follows.
rnn_states = cell(current_states, rnn_states)[1]
You also made a mistake here. You forgot to add 1 to loop_counter_current.
return [loop_counter_current+1, rnn_states]
Add
First structure represents the initial value of parameter loop_vars you passed in, which contains the initial values of loop_counter_inital and initi_state. So its structure corresponds to the following.
[
<tf.Tensor 'while/Identity:0' shape=() dtype=int32> #---> loop_counter_inital
, <tf.Tensor 'while/Identity_1:0' shape=(2, 3) dtype=float32> #---> initi_state
]
Second structure represents the parameter loop_vars after a loop. Its results correspond to the following based on previous errors.
[
<tf.Tensor 'while/Identity:0' shape=() dtype=int32> #---> loop_counter_inital
, (<tf.Tensor 'while/basic_rnn_cell/Tanh:0' shape=(2, 3) dtype=float32> #---> output
, <tf.Tensor 'while/basic_rnn_cell/Tanh:0' shape=(2, 3) dtype=float32>) #---> initi_state
]
I have the list of all tensors in my model which had the float32 type:
import os
import os.path
import tensorflow as tf
from tensorflow.python.platform import gfile
import numpy
numpy.set_printoptions(threshold=numpy.nan)
with tf.Session() as sess:
model_filename = 'MY_pb_file.pb'
with gfile.FastGFile(model_filename, 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
_= tf.import_graph_def(graph_def,name='')
from pprint import pprint
pprint([out for op in tf.get_default_graph().get_operations() if op.type != 'Placeholder' for out in op.values() if out.dtype == tf.float32])
which gives me all the list:
<tf.Tensor 'MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/add:0' shape=(16,) dtype=float32>, <tf.Tensor 'MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/Rsqrt:0' shape=(16,) dtype=float32>,
<tf.Tensor 'MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/mul:0' shape=(16,) dtype=float32>,
<tf.Tensor 'MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/mul_1:0' shape=(?, 64, 64, 16) dtype=float32>,
<tf.Tensor 'MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/mul_2:0' shape=(16,) dtype=float32>,
<tf.Tensor 'MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/sub:0' shape=(16,) dtype=float32>, <tf.Tensor 'MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/add_1:0' shape=(?, 64, 64, 16) dtype=float32>,
...
At this point I can use sess.run('NAME') to see its value:
>>> sess.run('MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/mul:0')
array([ 0.51656026, 29.6620369 , 0.48722425, 7.73186255,
-9.51173401, 0.60846734, 0.21111809, 0.23865609,
23.85105324, 1.04134226, 28.59620476, 35.79195023,
0.34110394, 0.5557093 , 10.39805031, 10.99952412], dtype=float32)
However, I would like print all tf.Tensor values in a loop. How can I do that?
Apparently, some needs a dictionary to be defined:
sess.run('MobilenetV1/MobilenetV1/Conv2d_1_pointwise/Relu6:0')
For example:
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input' with dtype float and shape [?,128,128,3]
[[Node: input = Placeholder[dtype=DT_FLOAT, shape=[?,128,128,3], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
When you request the value of a tensor, Tensorflow computes that value in the graph as tensor values are not usually retained across different calls to sess.run (that's what variables are for). It seems like the operations necessary to compute the value of the tensor you requested need input from one of your input placeholders (named input in the error statement), thus you must feed a value for that placeholder through the feed dictionary in sess.run.
Following your comment, consider this example:
import tensorflow as tf
a = tf.constant(4)
b = tf.constant(3)
c = tf.placeholder(tf.int32, [], 'c')
d = a + b
e = a + c
Requesting tensor d works fine:
with tf.Session() as sess:
print(sess.run(d)) # prints 7
However, requesting e throws the same error you report:
with tf.Session() as sess:
print(sess.run(e))
which prints
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'c' with dtype int32
[[Node: c = Placeholder[dtype=DT_INT32, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]()]]
This happens because in order to evaluate e we must compute a + c and if we don't feed a value to c, this is not possible. So, e.g., this works:
with tf.Session() as sess:
print(sess.run(e, feed_dict={c: 1})) # prints 5
Evaluating d worked fine because the computation path necessary to evaluate d does not involve a placeholder. So, to fix your problem, you should feed a value for the placeholder called 'input' in your call to sess.run('MobilenetV1/MobilenetV1/Conv2d_1_pointwise/Relu6:0').