I'm trying to set up a sequential RNN in Tensorflow with seq2seq.rnn_decoder(). The input that rnn_decoder() wants is a list of tensors, so to generate this I've passed in a rank-3 tensor and used tf.unpack() to make it into a list. The problem arises when the float32 array that I pass in in turned into a float64 tensor by tf.unpack(), making it incompatible with the rest of the model. Here's the code I put together to convince me that the culprit is tf.unpack():
inputDat = loader.getSequential(BATCH_SIZE)
print(inputDat.shape)
output (BATCH_SIZE is five, sequence length is ten):
(10, 5, 3)
Then I can load this data in a Tensorflow session:
sess = tf.InteractiveSession()
input_tensor = tf.constant(inputDat.astype('float32'), dtype=tf.float32)
print "Input tensor type: " + str(type(input_tensor.eval()[0,0,0]))
input_tensor = tf.unpack(inputDat)
print "Input tensor shape: " + str(len(input_tensor)) + "x" + str(input_tensor[0].eval().shape)
print "Input tensor type: " + str(type(input_tensor[0].eval()[0,0]))
Output:
Input tensor type: <type 'numpy.float32'>
Input tensor shape: 10x(5, 3)
Input tensor type: <type 'numpy.float64'>
What's going on here? Using a FOR loop to iterate through each of the sequential entries and re-cast it seems like the wrong way to do this, and I can't find a method inside Tensorflow to cast every member of a list.
You don't need a for-loop: you can use tf.cast().
Example:
input_tensor = tf.unpack(inputDat) # result is 64-bit
input_tensor = tf.cast(input_tensor, tf.float32) # now it's 32-bit
Related
I have two EagerTensor of type :
<class 'tensorflow.python.framework.ops.EagerTensor'>
tensor1 of size(64, 100) and tensor2 of size(64, 10).
I want output to be in the dimension (64, 110).
I tried:
tf.concat(axis=1, values = [tensor1, tensor2])
Getting an error :
InvalidArgumentError: cannot compute ConcatV2 as input #1(zero-based) was expected to be a uint8 tensor but is a float tensor [Op:ConcatV2] name: concat
But not working, Please help.
You need to ensure that both tensors have the same type when concatenating. Cast the uint8 one to float (more likely what you want), or vice-versa (though casting float to uint8 will likely not give you the results you expect)
import tensorflow as tf
import numpy as np
tf.compat.v1.enable_eager_execution()
tensor1 = tf.ones([64, 10], dtype=tf.uint8)
tensor2 = tf.ones([64, 100], dtype=tf.float32)
# print(type(x)) # <type 'EagerTensor'>
tf.concat(axis=1, values = [tf.cast(tensor1, tf.float32), tensor2]) # <<<< note the cast
I want to generate a constant tensor in Tensorflow, which will be initialized with a specified mechanism, eg, random_uniform, random_normal.
I know that I can generate a random numpy array according to these mechanisms, say random_uniform, random_normal, etc; Then we feed the resulted numpy array as value argument in tf.constant.
However, the question is that we must give a shape when using numpy version of random mechanism. However, I don't want to pre-specify the shape, and I hope the shape is resilient, just like we write Tensorflow code shape = tf.shape(some_previous_tensor)
Way1 I tried: There is not a must to pre-specify the concrete shape of the constant in the graph construction phase. However, the generated tensor is random rather than static. That is not I expected.
var = tf.random.normal(
[2,2], mean=0.0, stddev=0.5, dtype=tf.float32,
)
with tf.Session() as sess:
print('var:', sess.run(var))
print('var:', sess.run(var))
Output:
var: [[ 0.21260215 0.13721827]
[ 0.7704196 -0.48304045]]
var: [[-0.63397115 -0.0956466 ]
[ 0.0761982 0.54037064]]
Way2 I tried: I can get static constant, but it is necessary to give a size in np.random.normal, which is not I expected.
var_np = np.random.normal(0,0.5, size=(2,2))
var = tf.constant(value=var_np)
with tf.Session() as sess:
print('var:', sess.run(var))
print('var:', sess.run(var))
Output:
var: [[-0.73357953 -0.10277695]
[ 0.57406473 0.32157612]]
var: [[-0.73357953 -0.10277695]
[ 0.57406473 0.32157612]]
You can use tf.Variable / tf.get_variable with trainable=False and validate_shape=False. You can use a value depending on a placeholder for the shape as initial value. Then, when you initialize the variable (either using the initializer attribute or something more common like tf.global_variables_initializer), you just have to give the shape for initialization. After the initialization, the value of the variable will be kept the same for the whole session, as long as it is not initialized again or assigned a different value.
import tensorflow as tf
shape = tf.placeholder(tf.int32, [None])
var_init = tf.random.normal(
shape, mean=0.0, stddev=0.5, dtype=tf.float32,
)
var = tf.Variable(var_init, validate_shape=False, trainable=False, name='Var')
with tf.Session() as sess:
tf.random.set_random_seed(0)
sess.run(var.initializer, feed_dict={shape: [2, 3]})
print('var:', sess.run(var), sep='\n')
print('var:', sess.run(var), sep='\n')
Output:
var:
[[-0.4055751 0.7597851 -0.04810145]
[ 0.92776746 -0.3747548 -0.03715562]]
var:
[[-0.4055751 0.7597851 -0.04810145]
[ 0.92776746 -0.3747548 -0.03715562]]
Just run tf.shape(t) for a tensor t in whose shape you want your static random tensor to be. Feed the output value as size argument to np.random.normal and you're all set.
I am trying to make a speech recognizer using tensorflow and am getting this error that
ValueError: Tensor conversion requested dtype int64 for Tensor with
dtype float32: 'Tensor("loss_2/lambda_loss/ExpandDims:0", shape=(?,
1), dtype=float32)'
and
TypeError: Input 'y' of 'Mul' Op has type float32 that does not match
type int64 of argument 'x'.
Minimum Running Code where tensorflow is imported as tf
train_input_val = tf.keras.layers.Input(name='the_input', shape=[None,num_features], dtype='float32')
seq_len = tf.keras.layers.Input(name='input_length', shape=[1], dtype='int64')
y_pred = tf.keras.layers.Lambda(ctc_decode_func ,name = 'lambda')([train_input_val, seq_len])
model2 = tf.keras.Model(inputs=[train_input_val,seq_len],outputs=y_pred)
model2.compile(loss={'lambda': lambda y_true, y_pred: y_true}, optimizer='adam')
and ctc_decode_func is defined as
def ctc_decode_func(args):
y_pred, seq_len = args
y_pred,log_prob = tf.keras.backend.ctc_decode(y_pred, tf.squeeze(seq_len))
return y_pred
I tried casting everything to int64 , but still the error persisted. I dont Even know which part of lambda layer is throwing the error.
Please Help
I read in a *.csv file using tf.data.TextLineDataset and apply map on it:
dataset = tf.data.TextLineDataset(os.path.join(data_dir, subset, 'label.txt'))
dataset = dataset.map(lambda value: parse_record_fn(value, is_training),
num_parallel_calls=num_parallel_calls)
Parse function parse_record_fn looks like this:
def parse_record(raw_record, is_training):
default_record = ["./", -1]
filename, label = tf.decode_csv([raw_record], default_record)
# do something
return image, label
But there raise an ValueError at tf.decode_csv in parse function:
ValueError: Shape must be rank 1 but is rank 0 for 'DecodeCSV' (op: 'DecodeCSV') with input shapes: [1], [], [].
My *.csv file example:
/data/1.png, 5
/data/2.png, 7
Question:
Where goes wrong?
What does shapes: [1], [], [] mean?
Reproduce
This error can be reproduced in this code:
import tensorflow as tf
import os
def parse_record(raw_record, is_training):
default_record = ["./", -1]
filename, label = tf.decode_csv([raw_record], default_record)
# do something
return image, label
with tf.Session() as sess:
csv_path = './labels.txt'
dataset = tf.data.TextLineDataset(csv_path)
dataset = dataset.map(lambda value: parse_record(value, True))
sess.run(dataset)
Looking at the documentation of tf.decode_csv, it says about the default records:
record_defaults: A list of Tensor objects with specific types.
Acceptable types are float32, float64, int32, int64, string. One
tensor per column of the input record, with either a scalar default
value for that column or empty if the column is required.
I believe the error you are getting originates from how you define the tensor default_record. Your default_record certainly is a list of tensor objects (or objects convertible to tensors), but I think the error message is telling that they should be rank-1 tensors, not rank-0 tensors as in your case.
You can fix the issue by making the default records rank 1 tensors. See the following toy example:
import tensorflow as tf
my_line = 'filename.png, 10'
default_record_1 = [['./'], [-1]] # do this!
default_record_2 = ['./', -1] # this is what you do now
decoded_1 = tf.decode_csv(my_line, default_record_1)
with tf.Session() as sess:
d = sess.run(decoded_1)
print(d)
# This will cause an error
decoded_2 = tf.decode_csv(my_line, default_record_2)
The error produced on the last line is familiar:
ValueError: Shape must be rank 1 but is rank 0 for 'DecodeCSV_1' (op:
'DecodeCSV') with input shapes: [], [], [].
In the message, the input shapes, the three brackets [], refer to the shapes of the input arguments records, record_defaults, and field_delim of tf.decode_csv. In your case the first of these shapes is [1] since you input [raw_record]. I agree that the message for this case is not very informative...
Each of my training examples is a list with different length.
I am trying to find a way to feed those examples into the graph.
Below is my attempt to do so by creating a list whose elements are placeholders with unknown dimensions.
graph2 = tf.Graph()
with graph2.as_default():
A = list ()
for i in np.arange(3):
A.append(tf.placeholder(tf.float32 ,shape = [None,None]))
A_size = tf.shape(A)
with tf.Session(graph=graph2) as session:
tf.initialize_all_variables().run()
feed_dict = {A[0]:np.zeros((3,7)) ,A[1] : np.zeros((3,2)) , A[2] : np.zeros((3,2)) }
print ( type(feed_dict))
B = session.run(A_size ,feed_dict=feed_dict)
print type(B)
However I got the following error:
InvalidArgumentError: Shapes of all inputs must match: values[0].shape = [3,7] != values[1].shape = [3,2]
Any idea on how to solve it?
From the documentation of tf.placeholder:
shape: The shape of the tensor to be fed (optional). If the shape is not specified, you can feed a tensor of any shape.
You need to write shape=None instead of shape=[None, None]. With your code, Tensorflow doesn't know you are dealing with variable size input.