Develop Reference

Invalid Argument Error: Graph Execution Error on Tensorflow NN

I'm trying to fit the following neural network:
def make_model():
input = tf.keras.Input(shape=train_df.shape[1:])
x = tf.keras.layers.Flatten()(input)
x = tf.keras.layers.Dense(128, activation='relu')(x)
x = tf.keras.layers.Dense(64, activation='relu')(x)
x = tf.keras.layers.Dense(32, activation='relu')(x)
output = tf.keras.layers.Dense(8, activation='softmax')(x)
model = tf.keras.models.Model(input,output)
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=1e-3),
loss= tf.keras.losses.CategoricalCrossentropy(),
metrics=[tf.keras.metrics.CategoricalAccuracy(),
tf.keras.metrics.FalseNegatives(),
tf.keras.metrics.AUC(name='prc', curve='PR')])
return model
model = make_model()
model.fit(x=train_features, y=train_labels, epochs=2)
where:
model.summary()
print(train_features.shape, train_labels.shape)
outputs the following:
Model: "model_8"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_9 (InputLayer) [(None, 17)] 0
flatten_8 (Flatten) (None, 17) 0
dense_32 (Dense) (None, 128) 2304
dense_33 (Dense) (None, 64) 8256
dense_34 (Dense) (None, 32) 2080
dense_35 (Dense) (None, 8) 264
=================================================================
Total params: 12,904
Trainable params: 12,904
Non-trainable params: 0
_________________________________________________________________
Train Features Shape: (64140, 17)
Train Labels Shape: (64140, 8)
However, it keeps getting this error mid-epoch:
Epoch 1/2
444/2005 [=====>........................] - ETA: 1s - loss: 1.1139 - categorical_accuracy: 0.4904 - false_negatives_6: 10143.0000 - prc: 0.5232
Output exceeds the size limit. Open the full output data in a text editor
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
c:\Users\nrtc\OneDrive\Documentos\AI Summer School\Competition\Cópia_de_imbalanced_data.ipynb Cell 34' in <module>
----> 1 model.fit(x=train_features, y=train_labels, epochs=2)
File c:\Python39\lib\site-packages\keras\utils\traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb
File ~\AppData\Roaming\Python\Python39\site-packages\tensorflow\python\eager\execute.py:54, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
52 try:
53 ctx.ensure_initialized()
---> 54 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
55 inputs, attrs, num_outputs)
56 except core._NotOkStatusException as e:
57 if name is not None:
InvalidArgumentError: Graph execution error:
Detected at node 'assert_greater_equal/Assert/AssertGuard/Assert' defined at (most recent call last):
File "c:\Python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
...
File "c:\Python39\lib\site-packages\keras\utils\metrics_utils.py", line 602, in update_confusion_matrix_variables
tf.debugging.assert_greater_equal(
Node: 'assert_greater_equal/Assert/AssertGuard/Assert'
assertion failed: [predictions must be >= 0] [Condition x >= y did not hold element-wise:] [x (model_6/dense_27/Softmax:0) = ] [[0.101199746 0.358387947 0.118633337...]...] [y (Cast_4/x:0) = ] [0]
[[{{node assert_greater_equal/Assert/AssertGuard/Assert}}]] [Op:__inference_train_function_10341]
Any idea what might be the error? I've seen other threads for the same error (*), but I do think that the last layer is correctly set up for 8 output labels.
*Other stack overflow threads for a similar problem
https://stackoverflow.com/questions/62606345/tensorflow-2-2-0-error-predictions-must-be-0-condition-x-y-did-not-hold
https://stackoverflow.com/questions/71153492/invalid-argument-error-graph-execution-error

Some labels on the training set are NaN values, so plotting the labels does not make the error clear.
train_df.dropna(inplace=True)
does the trick.

'Functional' object has no attribute 'predict_classes' error when we use cross_val_predict

I want to use Bert through Tensorflow.
My plan is to use Bert with 5-fold cross validation, batch_size = 64, epoch = 100, and early stopping.
I have 5 classes in my data.
Therefore, I wrote the following code (This code is an example to define my problem here):
import tensorflow_hub as hub
import tensorflow_text as text
...
bert_preprocess = hub.KerasLayer("https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3")
bert_encoder = hub.KerasLayer("https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/4")
def create_model():
model = Sequential()
text_input = tf.keras.layers.Input(shape=(), dtype=tf.string, name='text')
preprocessed_text = bert_preprocess(text_input)
outputs = bert_encoder(preprocessed_text)
l = tf.keras.layers.Dropout(0.1, name="dropout")(outputs['pooled_output'])
l = tf.keras.layers.Dense(5, activation='softmax', name="output")(l)
model = tf.keras.Model(inputs=[text_input], outputs = [l])
model.summary()
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['categorical_accuracy'])
return model
estimator = KerasClassifier(build_fn = create_model,
epochs=1,
batch_size=64,
verbose=1
)
kf = KFold(n_splits=5)
pred = cross_val_predict(estimator, X_train, Y_train,
cv=kf,
verbose=1)
But I got the following error:
Layer (type) Output Shape Param # Connected to
==================================================================================================
text (InputLayer) [(None,)] 0
__________________________________________________________________________________________________
keras_layer (KerasLayer) {'input_type_ids': ( 0 text[0][0]
__________________________________________________________________________________________________
keras_layer_1 (KerasLayer) {'encoder_outputs': 109482241 keras_layer[12][0]
keras_layer[12][1]
keras_layer[12][2]
__________________________________________________________________________________________________
dropout (Dropout) (None, 768) 0 keras_layer_1[12][13]
__________________________________________________________________________________________________
output (Dense) (None, 5) 3845 dropout[0][0]
==================================================================================================
Total params: 109,486,086
Trainable params: 3,845
Non-trainable params: 109,482,241
__________________________________________________________________________________________________
38/38 [==============================] - 226s 6s/step - loss: 1.6111 - categorical_accuracy: 0.2434
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-26-55c3162a6b6d> in <module>()
16
17
---> 18 verbose=1)
19
20
10 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/wrappers/scikit_learn.py in predict(self, x, **kwargs)
239 """
240 kwargs = self.filter_sk_params(Sequential.predict_classes, kwargs)
--> 241 classes = self.model.predict_classes(x, **kwargs)
242 return self.classes_[classes]
243
AttributeError: 'Functional' object has no attribute 'predict_classes'
I also removed model = Sequential() but I still have the same problem.
How can I fix it? or
Is there any way to use all my plan?

image captioning with deep learning

can anyone help me ..where is the error here?? can anyone help me please. i am trying to train a model for image captioning in persian language.i try to build my model .. below is the code and the model summary.....................
`embeddings_dim = 256
input_image_dim = X_train_image.shape[1]
keras.backend.clear_session()
# image model
input_image = keras.layers.Input(shape=(input_image_dim,), name='input_image')
#input_image_dropout = keras.layers.Dropout(0.4)(input_image)
image_embeddings = keras.layers.Dense(embeddings_dim, activation='tanh', name='image_embeddings') (input_image)
# text model
# Set up the decoder, using `image_embeddings` as initial state.
decoder_inputs = keras.layers.Input(shape=(max_len,))
embeddings = keras.layers.Embedding(len(word2index), embeddings_dim, mask_zero=True)(decoder_inputs)
#embeddings_dropout = keras.layers.Dropout(0.3)(embeddings)
gru = keras.layers.GRU(embeddings_dim)(embeddings, initial_state=image_embeddings) # , return_sequences=True
#flat = keras.layers.Flatten()(gru)
dense = keras.layers.Dense(embeddings_dim, activation='relu')(gru)
#dense_dropout = keras.layers.Dropout(0.3)(dense)
decoder_outputs = keras.layers.Dense(len(word2index), activation='softmax')(dense)
seq2seq = keras.Model([input_image, decoder_inputs], decoder_outputs)
seq2seq.summary()
# prepare callback
#histories = My_Callback()
model_checkpoint_path = 'models/model.{epoch:02d}-{val_loss:.3f}--{b1:.3f}.hdf5'
checkpoint_callback = keras.callbacks.ModelCheckpoint(model_checkpoint_path, monitor='val_loss', verbose=1, save_best_only=False, save_weights_only=False, mode='auto', period=1)
callbacks = [TQDMNotebookCallback(), Bleu_Callback(), checkpoint_callback] #[checkpoint_callback, TQDMNotebookCallback(), My_Callback()]
seq2seq.compile(optimizer=keras.optimizers.Adam(), loss='categorical_crossentropy')
seq2seq.fit([X_train_image, X_train_text], y_train_text,
validation_data=([X_test_image, X_test_text], y_test_text),
batch_size=1024,
epochs=10,
verbose=2,
callbacks=callbacks) # add My_Callback() to callbacks to calculate & display BLEU score after each epoch`
this is the error after running my code .. gives me KeyError: 'metrics'... thanks
` Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 20)] 0 []
input_image (InputLayer) [(None, 1000)] 0 []
embedding (Embedding) (None, 20, 256) 730112 ['input_1[0][0]']
image_embeddings (Dense) (None, 256) 256256 ['input_image[0][0]']
gru (GRU) (None, 256) 394752 ['embedding[0][0]',
'image_embeddings[0][0]']
dense (Dense) (None, 256) 65792 ['gru[0][0]']
dense_1 (Dense) (None, 2852) 732964 ['dense[0][0]']
==================================================================================================
Total params: 2,179,876
Trainable params: 2,179,876
Non-trainable params: 0
__________________________________________________________________________________________________
WARNING:tensorflow:`period` argument is deprecated. Please use `save_freq` to specify the frequency in number of batches seen.
Training: 0%
0/20 [00:00<?, ?it/s]
Epoch 0: 0%
0/237 [00:00<?, ?it/s]
Epoch 1/20
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-32-0fc16141d8f6> in <module>
32 seq2seq.compile(optimizer=keras.optimizers.Adam(), loss='categorical_crossentropy')
33
---> 34 seq2seq.fit([X_train_image, X_train_text], y_train_text,
35 validation_data=([X_test_image, X_test_text], y_test_text),
36 batch_size=1024,
~\anaconda3\lib\site-packages\keras\utils\traceback_utils.py in error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb
~\anaconda3\lib\site-packages\keras_tqdm\tqdm_callback.py in append_logs(self, logs)
134
135 def append_logs(self, logs):
--> 136 metrics = self.params['metrics']
137 for metric, value in six.iteritems(logs):
138 if metric in metrics:
KeyError: 'metrics'
`

Why is the sequential layer expecting 3 dimensions in Keras?

Upon trying to train my LSTM network, I am given the error: ValueError: Input 0 of layer sequential_2 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, None].
My code is as follows:
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
numFeatures = 26; #amino acids
numClasses = 64; #codons
x_test = tf.ragged.constant(XTest)
y_train = tf.ragged.constant(YTrain)
y_test = tf.ragged.constant(YTest)
x_train = tf.ragged.constant(XTrain)
model = keras.Sequential(
[
keras.Input(shape=(26, 10), ragged=True),
layers.LSTM(128,use_bias=False),
layers.Dense(10, activation="softmax")
]
)
model.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer="adam",
metrics=["accuracy"]
)
model.fit(
x_train, y_train, validation_data=(x_test, y_test), batch_size=5, epochs=1
)
model.summary() shows me this:
Model: "sequential_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
lstm_2 (LSTM) (None, 128) 70656
_________________________________________________________________
dense_2 (Dense) (None, 10) 1290
=================================================================
Total params: 71,946
Trainable params: 71,946
Non-trainable params: 0
_________________________________________________________________
I get the following error:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-11-ae3b9eb7daa0> in <module>()
15 )
16 model.fit(
---> 17 x_train, y_train, validation_data=(x_test, y_test), batch_size=5, epochs=1
18 )
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
966 except Exception as e: # pylint:disable=broad-except
967 if hasattr(e, "ag_error_metadata"):
--> 968 raise e.ag_error_metadata.to_exception(e)
969 else:
970 raise
ValueError: in user code:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function *
outputs = self.distribute_strategy.run(
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:951 run **
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2290 call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2649 _call_for_each_replica
return fn(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:531 train_step **
y_pred = self(x, training=True)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py:886 __call__
self.name)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/input_spec.py:180 assert_input_compatibility
str(x.shape.as_list()))
ValueError: Input 0 of layer sequential_2 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, None]

TesorFlow Keras GRU layer invalid argument error - 'expected to be a double tensor but is a float tensor'

I am currently working on an RNN in TensorFlow (and Keras) to generate moving object data. My RNN model is defined as follows:
if tf.test.is_gpu_available():
my_gru = tf.keras.layers.CuDNNGRU
else:
import functools
my_gru = functools.partial(
tf.keras.layers.GRU, recurrent_activation='sigmoid')
def build_model(internal_units, batch_size):
model = tf.keras.Sequential([
my_gru(internal_units, return_sequences=True, recurrent_initializer='glorot_uniform', stateful=True, batch_input_shape=[batch_size, None, 3]),
tf.keras.layers.Dense(3)
])
return model
INTERNAL_UNITS = 1024
model = build_model(internal_units=INTERNAL_UNITS, batch_size=BATCH_SIZE)
I get the following summary for the model, which is all well and good:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
cu_dnngru_2 (CuDNNGRU) (64, None, 1024) 3161088
_________________________________________________________________
dense_4 (Dense) (64, None, 3) 3075
=================================================================
Total params: 3,164,163
Trainable params: 3,164,163
Non-trainable params: 0
_________________________________________________________________
When I try to run the model for a single prediction before training, just to see if it generally works and the output has the correct shape, I get an error.
This is how I run the model ...
for input_example_batch, target_example_batch in dataset.take(1):
example_batch_predictions = model(input_example_batch)
print(example_batch_predictions.shape, "# (batch_size, sequence_length, n_features)")
... and this is the error I get ...
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
<ipython-input-52-dd1ea240c0d1> in <module>()
1 for input_example_batch, target_example_batch in dataset.take(1):
----> 2 example_batch_predictions = model(input_example_batch)
3 print(example_batch_predictions.shape, "# (batch_size, sequence_length, n_features)")
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, inputs, *args, **kwargs)
755 if not in_deferred_mode:
756 self._in_call = True
--> 757 outputs = self.call(inputs, *args, **kwargs)
758 self._in_call = False
759 if outputs is None:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/sequential.py in call(self, inputs, training, mask)
227 def call(self, inputs, training=None, mask=None):
228 if self._is_graph_network:
--> 229 return super(Sequential, self).call(inputs, training=training, mask=mask)
230
231 outputs, _ = self._call_and_compute_mask(
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/network.py in call(self, inputs, training, mask)
843 outputs, _ = self._run_internal_graph(inputs,
844 training=training,
--> 845 mask=masks)
846 return outputs
847
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/network.py in _run_internal_graph(self, inputs, training, mask)
1029 computed_tensor, **kwargs)
1030 else:
-> 1031 output_tensors = layer.call(computed_tensor, **kwargs)
1032 if hasattr(layer, 'compute_mask'):
1033 output_masks = layer.compute_mask(computed_tensor,
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/layers/cudnn_recurrent.py in call(self, inputs, mask, training, initial_state)
107 # Reverse time axis.
108 inputs = K.reverse(inputs, 1)
--> 109 output, states = self._process_batch(inputs, initial_state)
110
111 if self.stateful:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/layers/cudnn_recurrent.py in _process_batch(self, inputs, initial_state)
297 params=params,
298 is_training=True,
--> 299 rnn_mode='gru')
300
301 if self.stateful or self.return_state:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_cudnn_rnn_ops.py in cudnn_rnn(input, input_h, input_c, params, rnn_mode, input_mode, direction, dropout, seed, seed2, is_training, name)
142 input_mode=input_mode, direction=direction, dropout=dropout,
143 seed=seed, seed2=seed2, is_training=is_training, name=name,
--> 144 ctx=_ctx)
145 except _core._NotOkStatusException as e:
146 if name is not None:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_cudnn_rnn_ops.py in cudnn_rnn_eager_fallback(input, input_h, input_c, params, rnn_mode, input_mode, direction, dropout, seed, seed2, is_training, name, ctx)
184 "is_training", is_training)
185 _result = _execute.execute(b"CudnnRNN", 4, inputs=_inputs_flat,
--> 186 attrs=_attrs, ctx=_ctx, name=name)
187 _execute.record_gradient(
188 "CudnnRNN", _inputs_flat, _attrs, _result, name)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
64 else:
65 message = e.message
---> 66 six.raise_from(core._status_to_exception(e.code, message), None)
67 # pylint: enable=protected-access
68 return tensors
/usr/local/lib/python3.6/dist-packages/six.py in raise_from(value, from_value)
InvalidArgumentError: cannot compute CudnnRNN as input #1(zero-based) was expected to be a double tensor but is a float tensor [Op:CudnnRNN]
The input I use (input_example_batch) looks as follows:
tf.Tensor(
[[[ 1.04234461 4.66466794 -4.32528214]
[ 1.04244826 4.49530966 -4.42294239]
[ 1.04256889 4.47099585 -4.51911731]
...
[ 1.05549699 -0.64915764 0.67451403]
[ 1.05559893 -0.7788313 0.66355975]
[ 1.05570257 -0.88502956 0.65223413]]
[[-0.83849063 -0.47476892 -0.22299478]
[-0.8383632 -0.41943403 -0.22782209]
[-0.83830883 -0.41635987 -0.24527468]
...
[-0.82015615 0.05482504 0.681198 ]
[-0.8200627 -0.04187127 0.69233796]
[-0.81995906 -0.13493448 0.66615907]]
[[-1.27264128 0.12441285 -0.48032767]
[-1.27256143 0.29544794 -0.714081 ]
[-1.27247477 0.46871879 -0.97197089]
...
[-1.25435437 3.47832158 0.72724314]
[-1.25427451 3.90674772 0.63255355]
[-1.25418956 4.28934093 0.49219015]]
...
[[ 1.01251773 1.16012535 -2.9933152 ]
[ 1.01263666 1.16906836 -2.99554319]
[ 1.0127505 0.99943062 -2.95135471]
...
[ 1.02650943 -0.42362607 0.46805359]
[ 1.02661308 -0.49377282 0.42702143]
[ 1.02666575 -0.49544963 0.42739276]]
[[-1.34049978 -0.01029126 0.5430626 ]
[-1.34038934 -0.04690171 0.52078269]
[-1.34029419 0.07857991 0.52895199]
...
[-1.3289751 1.94152097 1.03489148]
[-1.32888165 1.51113855 0.94372954]
[-1.32878311 1.16208164 0.76511898]]
[[ 0.57716585 -1.38779448 0.0469634 ]
[ 0.57728308 -1.44536512 -0.01542033]
[ 0.57734255 -1.49846425 -0.02581762]
...
[ 0.58594146 -0.63183055 -0.03621491]
[ 0.58603491 -0.60136842 -0.04419854]
[ 0.58613855 -0.63015374 -0.07204842]]], shape=(64, 100, 3), dtype=float64)
All of this is running in a Google Colab environment.
I have already tried forcing it to use the normal GRU layer, which leads to pretty much the same error. A notable difference, though, with the normal GRU layer is that it claims to expect a float tensor instead of a double tensor (exactly the other way around as the error I copied into this post).
My entire code basically the same as an example given on the TensorFlow website. I modified the model slightly and changed the input data to have 3 features instead of having as many features as the alphabet is long. The original TensorFlow example works just fine for me.
Please excuse the long post and ask for more information, if you need any. I tried to be thorough, but you never know. I appreciate every help.
EDIT:
Here you can find a short example Notebook on Google Colab that you can use to reproduce the error yourself.

So I found a solution that seems to work for me. When I call model.input I get the following output that suggests that the GRU layer requires its input to be a float tensor.
<DeferredTensor 'gru_input' shape=(64, ?, 3) dtype=float32>
Using the line
x = tf.cast(x, dtype=tf.float32)
(TensorFlow documentation) I can cast my input data to be a float tensor. Using this float tensor, the model works just fine.
I hope this helps someone with the same problem. According to this post float64 compatibility is a known issue with TensorFlow.