I have two scalars resulting from the following operations:
a = tf.reduce_sum(tensor1), b = tf.matmul(tf.transpose(tensor2), tensor3) this is a dot product since tensor2 and tensor3 have the same dimensions (1-D vectors). Since these tensors have shape [None, dim1] it becomes difficult to deal with the shapes.
I want to build a tensor that has shape (2,1) using a and b.
I tried tf.Tensor([a,b], dtype=tf.float64, value_index=0) but raises the error
TypeError: op needs to be an Operation: [<tf.Tensor 'Sum_5:0' shape=() dtype=float32>, <tf.Tensor 'MatMul_67:0' shape=(?, ?) dtype=float32>]
Any easier way to build that tensor/vector?
This would do probably. Change axis based on what you need
a = tf.constant(1)
b = tf.constant(2)
c = tf.stack([a,b],axis=0)
Output:
array([[1],
[2]], dtype=int32)
You can use concat or stack to achieve this:
import tensorflow as tf
t1 = tf.constant([1])
t2 = tf.constant([2])
c = tf.reshape(tf.concat([t1, t2], 0), (2, 1))
with tf.Session() as sess:
print sess.run(c)
In a similar way you can achieve it with tf.stack.
Related
I have two tensors of size
A <tf.Tensor 'sequential_12/my_layer_56/add:0' shape=(?, 300, 2) dtype=float32>
and
B <tf.Tensor 'input_82:0' shape=(?, 2, 2) dtype=float32>
Now, I would like to multiply them in the sense of the usual matrix row-column product to obtain
A * B of size (?, 300, 2), so I would be doing the matrix product only over the second and third dimension. How can I achieve this?
I tried to use tf.tensordot with different axes specifications, but it did not work so far. For example I tried
tf.tensordot(A,B,axes=[[2], [0]])
but this produces a tensor of the following form
<tf.Tensor 'Tensordot_10:0' shape=(?, 300, 2, 2) dtype=float32>
Maybe try tf.matmul:
import tensorflow as tf
samples = 1
A = tf.random.normal((samples, 300, 2))
B = tf.random.normal((samples, 2, 2))
print(tf.matmul(A, B).shape)
# (1, 300, 2)
I am working in Keras with a tensor of the form
A = <tf.Tensor 'lambda_87/strided_slice:0' shape=(?, 40, 2) dtype=float32>
Now, I would like to add, for each of the 40 "rows" the index 0 row of the a Tensor with dimensions
B = <tf.Tensor 'lambda_92/mul:0' shape=(?, 2, 2) dtype=float32>
For short, for the second tensor I only need for the present step B[:,0,:]. So, excluding the first dimension, this would be the first "row" of the matrix B.
The Add() layer seems to work only with equally-sized tensors. Any suggestion on how I could specify a Lambda function that does the job?
Thanks for reading!
Maybe try something like this:
import tensorflow as tf
samples = 1
A = tf.random.normal((samples, 40, 2))
B = tf.random.normal((samples, 2, 2))
B = tf.expand_dims(B[:, 0, :], axis=1) # or just B = B[:, 0, :]
C = A + B
print(C.shape)
# (1, 40, 2)
Or with a Lambda layer:
import tensorflow as tf
samples = 1
A = tf.random.normal((samples, 40, 2))
B = tf.random.normal((samples, 2, 2))
lambda_layer = tf.keras.layers.Lambda(lambda x: x[0] + x[1][:, 0, :])
print(lambda_layer([A, B]))
I am trying to import a pretrained Model from tensorflow to PyTorch. It takes a single input and maps it onto a single output.
Confusion comes up, when I try to import the LSTM weights
I read the weights and their variables from the file with the following function:
def load_tf_model_weights():
modelpath = 'models/model1.ckpt.meta'
with tf.Session() as sess:
tf.train.import_meta_graph(modelpath)
init = tf.global_variables_initializer()
sess.run(init)
vars = tf.trainable_variables()
W = sess.run(vars)
return W,vars
W,V = load_tf_model_weights()
Then I am inspecting the shapes of the weights
In [33]: [w.shape for w in W]
Out[33]: [(51, 200), (200,), (100, 200), (200,), (50, 1), (1,)]
furthermore the variables are defined as
In [34]: V
Out[34]:
[<tf.Variable 'rnn/multi_rnn_cell/cell_0/lstm_cell/kernel:0' shape=(51, 200) dtype=float32_ref>,
<tf.Variable 'rnn/multi_rnn_cell/cell_0/lstm_cell/bias:0' shape=(200,) dtype=float32_ref>,
<tf.Variable 'rnn/multi_rnn_cell/cell_1/lstm_cell/kernel:0' shape=(100, 200) dtype=float32_ref>,
<tf.Variable 'rnn/multi_rnn_cell/cell_1/lstm_cell/bias:0' shape=(200,) dtype=float32_ref>,
<tf.Variable 'weight:0' shape=(50, 1) dtype=float32_ref>,
<tf.Variable 'FCLayer/Variable:0' shape=(1,) dtype=float32_ref>]
So I can say that the first element of W defines the Kernel of an LSTM and the second element define its bias. According to this post, the shape for the Kernel is defined as
[input_depth + h_depth, 4 * self._num_units]
and the bias as [4 * self._num_units]. We already know that input_depth is 1. So we get, that h_depth and _num_units both have the value 50.
In pytorch my LSTMCell, to which I want to assign the weights, looks like this:
In [38]: cell = nn.LSTMCell(1,50)
In [39]: [p.shape for p in cell.parameters()]
Out[39]:
[torch.Size([200, 1]),
torch.Size([200, 50]),
torch.Size([200]),
torch.Size([200])]
The first two entries can be covered by the first value of W which has the shape (51,200). But the LSTMCell from Tensorflow yields only one bias of shape (200) while pytorch wants two of them
And by leaving the bias out I have weights left over:
cell2 = nn.LSTMCell(1,50,bias=False)
[p.shape for p in cell2.parameters()]
Out[43]: [torch.Size([200, 1]), torch.Size([200, 50])]
Thanks!
pytorch uses CuDNN's LSTM underlayer(even when you don't have CUDA, it still uses something compatible) thus it has one extra bias term.
So you can pick two numbers with their sum equal to 1(0 and 1, 1/2 and 1/2 or anything else) and set your pytorch biases as those numbers times TF's bias.
pytorch_bias_1 = torch.from_numpy(alpha * tf_bias_data)
pytorch_bias_2 = torch.from_numpy((1.0-alpha) * tf_bias_data)
I'm so news to Tensorflow . I already search for same questions,but i can't understand. there is the code .Hope you can help me.
Code:
import tensorflow as tf
w1 = tf.Variable(tf.random_normal([2,3],stddev=1,seed=1))
w2 = tf.Variable(tf.random_normal([3,3],stddev=1,seed=1))
x = tf.constant([0.7,0.9])
a = tf.matmul(x, w1)
y = tf.matmul(a, w2)
sess = tf.Session()
sess.run(w1.initializer)
sess.run(w2.initializer)
print(sess.run(y))
sess.close()
The shape of constant x is (2,), i.e. a one-dimensional array, and you are trying to multiply it with a two-dimensional array w1 of shape (2, 3), which is not possible for matrix multiplication, as number of columns of first parameter must be equal to number of rows in second parameter. Also, I think tf.matmul only works if both arrays are two-dimensional.
One of the many ways you can change your declaration of x as
x = tf.constant([[0.7], [0.9]])
This will create a two-dimensional constant tensor of shape (2, 1). And, then multiply it as,
a = tf.matmul(tf.transpose(x), w1)
tf.transpose() is used to create transpose of array x with shape (2, 1) to shape (1, 2).
Hope this helps.
In your case, the rank of variable x is 1. Hence the issue.
Following is the reason you are having this issue.
Please refer the tensorflow API https://www.tensorflow.org/api_docs/python/tf/matmul
tf.matmul(a, b, transpose_a=False, transpose_b=False, adjoint_a=False, adjoint_b=False,
a_is_sparse=False, b_is_sparse=False, name=None)
Args:
a: Tensor of type float16, float32, float64, int32, complex64, complex128 and rank > 1.
b: Tensor with same type and rank as a.
The shape of x is (2,) does not match the shape (2,3) of w1.
You should change
x = tf.constant([0.7,0.9])
to
x = tf.constant([[0.7,0.9]])
now the shape of x is (1,2) and works fine.
I have the list of all tensors in my model which had the float32 type:
import os
import os.path
import tensorflow as tf
from tensorflow.python.platform import gfile
import numpy
numpy.set_printoptions(threshold=numpy.nan)
with tf.Session() as sess:
model_filename = 'MY_pb_file.pb'
with gfile.FastGFile(model_filename, 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
_= tf.import_graph_def(graph_def,name='')
from pprint import pprint
pprint([out for op in tf.get_default_graph().get_operations() if op.type != 'Placeholder' for out in op.values() if out.dtype == tf.float32])
which gives me all the list:
<tf.Tensor 'MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/add:0' shape=(16,) dtype=float32>, <tf.Tensor 'MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/Rsqrt:0' shape=(16,) dtype=float32>,
<tf.Tensor 'MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/mul:0' shape=(16,) dtype=float32>,
<tf.Tensor 'MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/mul_1:0' shape=(?, 64, 64, 16) dtype=float32>,
<tf.Tensor 'MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/mul_2:0' shape=(16,) dtype=float32>,
<tf.Tensor 'MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/sub:0' shape=(16,) dtype=float32>, <tf.Tensor 'MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/add_1:0' shape=(?, 64, 64, 16) dtype=float32>,
...
At this point I can use sess.run('NAME') to see its value:
>>> sess.run('MobilenetV1/MobilenetV1/Conv2d_1_pointwise/BatchNorm/batchnorm/mul:0')
array([ 0.51656026, 29.6620369 , 0.48722425, 7.73186255,
-9.51173401, 0.60846734, 0.21111809, 0.23865609,
23.85105324, 1.04134226, 28.59620476, 35.79195023,
0.34110394, 0.5557093 , 10.39805031, 10.99952412], dtype=float32)
However, I would like print all tf.Tensor values in a loop. How can I do that?
Apparently, some needs a dictionary to be defined:
sess.run('MobilenetV1/MobilenetV1/Conv2d_1_pointwise/Relu6:0')
For example:
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input' with dtype float and shape [?,128,128,3]
[[Node: input = Placeholder[dtype=DT_FLOAT, shape=[?,128,128,3], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
When you request the value of a tensor, Tensorflow computes that value in the graph as tensor values are not usually retained across different calls to sess.run (that's what variables are for). It seems like the operations necessary to compute the value of the tensor you requested need input from one of your input placeholders (named input in the error statement), thus you must feed a value for that placeholder through the feed dictionary in sess.run.
Following your comment, consider this example:
import tensorflow as tf
a = tf.constant(4)
b = tf.constant(3)
c = tf.placeholder(tf.int32, [], 'c')
d = a + b
e = a + c
Requesting tensor d works fine:
with tf.Session() as sess:
print(sess.run(d)) # prints 7
However, requesting e throws the same error you report:
with tf.Session() as sess:
print(sess.run(e))
which prints
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'c' with dtype int32
[[Node: c = Placeholder[dtype=DT_INT32, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]()]]
This happens because in order to evaluate e we must compute a + c and if we don't feed a value to c, this is not possible. So, e.g., this works:
with tf.Session() as sess:
print(sess.run(e, feed_dict={c: 1})) # prints 5
Evaluating d worked fine because the computation path necessary to evaluate d does not involve a placeholder. So, to fix your problem, you should feed a value for the placeholder called 'input' in your call to sess.run('MobilenetV1/MobilenetV1/Conv2d_1_pointwise/Relu6:0').