I have a simple network:
input_layer = Input(1)
inner_layer = Dense(4, activation='relu')(input_layer)
output_layer = Dense(1, activation='linear')(inner_layer)
model = Model(input_layer, output_layer)
optimizer = Adam(learning_rate=0.01)
model.compile(optimizer=optimizer, loss='mse')
Intuitively, inference for input 0 would simply be model.predict(0). However this generates this error: expected input_2 to have 2 dimensions, but got array with shape ()
I understand it expects the input (which is a single number) to be two dimensional, but I don't understand what Tensorflow accepts as valid input. I tried many different combinations of inputs, some work and some don't, it seems quite inconsistent and the warnings/error are usually not useful:
When calling model.predict():
model.predict(0) - Throws
model.predict([0]) - Works
model.predict([[0]]) - Works
When calling model() (I saw here that's needed to get the gradients):
model(0) - Throws
model([0]) - Throws
model([[0]]) - Throws
When using np.reshape:
model(np.reshape(0,[1,1])) - Works
model(np.reshape([0],[1,1])) - Works
model(np.reshape([[0]],[1,1])) - Works
What seems to be working consistently is using numpy's reshape function. it always works both for model.predict() and for model() on all the inputs as long as they're reshaped to a [1,1] shape.
My questions:
What are the guidelines to feeding inputs into tensorflow models in regards to inputs shapes/types?
What does "shape ()" mean?
What does "(None, 1)" mean?
Why does reshape work but [[0]] does not? Both create a 2-dimensional collection.
Why when calling model(0)/model([0])/model([[0]]) does this warning show: WARNING:tensorflow:Model was constructed with shape Tensor("input_1:0", shape=(None, 1), dtype=float32) for input (None, 1), but it was re-called on a Tensor with incompatible shape ()?
The shape of the tensor inputs = tf.keras.layers.Input(1) is (None, 1) (run inputs.get_shape().as_list()). The None means any size that is determined dynamically (batch size). The 1 is the shape of your data point. For example, this is a tensor of shape (3, 1):
[[1], [2], [1]]
This is a tensor of shape (3,)
[1, 2, 1]
If you define a tensor of shape (None, 1) you must feed the data of same shape.
The [[0]] has correct shape (1, 1) and won't throw any error or warning if you pass it as numpy array of expected data type:
import tensorflow as tf
import numpy as np
input_layer = tf.keras.layers.Input(1)
inner_layer = tf.keras.layers.Dense(4, activation='relu')(input_layer)
output_layer = tf.keras.layers.Dense(1, activation='linear')(inner_layer)
model = tf.keras.models.Model(input_layer, output_layer)
optimizer = tf.keras.optimizers.Adam(learning_rate=0.01)
model.compile(optimizer=optimizer, loss='mse')
print(model(np.array([[0.]], dtype=np.float32)).numpy()) # [[0.]]
print(model.predict(np.array([[0.], [1]], dtype=np.float32))) # [[0. ]
# [0.08964952]]
np.reshape() works because it automatically converts your list to numpy array. For more about np.reshape refer to the official documentation.
The model.predict() also expects the same shape as model.__call__(), but can perform automatic reshaping (expands dimension on the left, i.e [1] -- > [[1]]).
Related
i was trying to implement a simple RNN on a 1 dimensional toy dataset (just one x and one target variable y) to learn but I am having issue with the input shape.
X_train has input shape equal to (848,)
y_train the same
my model is very simple
from keras.layers import Input, SimpleRNN, Dense
def create_rnn_model(train_size):
inputs = Input(shape=( 1,))
rnn = SimpleRNN(units=32, input_shape=( 1,))(inputs)
dense1 = Dense(units=64, activation='relu')(rnn)
dense2 = Dense(units=64, activation='relu')(dense1)
modello = Model(inputs=inputs, outputs=dense2)
return modello
optimizer = tf.optimizers.SGD(learning_rate=0.0001,momentum=0.9)
rnn_model = create_rnn_model(train_size=train_size)
rnn_model.compile(optimizer=optimizer,
loss="mse" , jit_compile=True ]
)
but whenever i try to fit it i get this input shape that doesn't allow me to go further
ValueError: Input 0 of layer "simple_rnn_5" is incompatible with the
layer: expected ndim=3, found ndim=2. Full shape received: (None, 1)
i know it depends on the format that the rnn want but i don't understand how to organize the data correctly.
i was trying without any windows size but i would really like to understand how to set a windows size of 10 for example.
I am using tf 1.14, and following the official guide to pass my own sparse tensor that has a dense shape of 2000 x 2000 as input to a model, like so:
input = layers.Input((2000,), sparse=True)
print(input.get_shape().as_list())
output = some_layers(input)
model = models.Model(inputs=input, outputs=output)
However when I print the input shape, it returns [None, None], and I get the error:
ValueError: Error when checking input: expected input_10 to have 2 dimensions, but got array with shape (None, None, None)
If I then specify a batch input = layers.Input((2000,), batch_size=1, sparse=True), the size is (1, 2000) and it runs without error.
And if I specific input is 2d like so: input = layers.Input((2000,2000), batch_size=1, sparse=True) then i return a size (1,2000,2000), and again it runs without error
Which approach is correct? Ultimately, I want to use the sparse tensor as an adjacency matrix for a GCN layer. Therefore I want it without the batch dimension.
I have sample data in the form: Data[n][31][31][5][2] with:
"[n]" being the sample
"[31][31]" being the array of data points
"[5]" being the number of bits within that data point
and "[2]" being one-hot encoding of the bits (eg a bit of 1 would be [1, 0] and a zero [0, 1])
The output is intended to either be a [5][2] or a [10] array of values which is validated against another [5][2] or [10] array.
When trying to build the model, I get the following error:
"ValueError: Shapes (None, 5, 2) and (None, 10) are incompatible"
The model code looks like this:
(with train_m[n][31][31][5][2], tr_m[5][2], check_m[n][31][31][5][2], cr_m[5][2] being training data and expected output followed by validation data and expected output.)
model = Sequential([
Conv2D(num_filters, filter_size, input_shape=(31, 31, 5, 2)),
Flatten(),
Dense(10, activation='relu'),
])
model.compile(
'adam',
loss='categorical_crossentropy',
metrics=['accuracy'],
)
model.summary()
model.fit(
train_m,
tr_m,
epochs=(100),
validation_data=(check_m, cr_m),
verbose=0
)
As the [5][2] outputs are one-hotted, I'm uncertain if they can be made to a [10] matrix while still being interpreted correctly. Further, would there be any way to make the dense layer to a [5][2]?
The full error can be seen here. I felt it would be awfully long to include in rawtext here.
If there's anything more that's needed, please let me know - I'm still very new to working with TensorFlow.
Your label shapes are (5,2) but network output is (10,) so this is confusing. Both output shape and label shape should be the same.
use:
tf.keras.layers.Reshape((5,2))
after the Dense layer. you'll be fine
I am getting an error relating to shapes whilst defining a very simple network using Tensorflow 2.
My code is:
import tensorflow as tf
import pandas as pd
data = pd.read_csv('data.csv')
target = data.pop('result')
target = tf.keras.utils.to_categorical(target.values, num_classes=3)
data_set = tf.data.Dataset.from_tensor_slices((data.values, target))
model = tf.keras.Sequential([
tf.keras.layers.Input(shape=data.shape[1:]),
tf.keras.layers.Dense(12, activation='relu'),
tf.keras.layers.Dense(3, activation='softmax')
])
model.compile(optimizer='adam', loss='categorical_crossentropy')
model.fit(data_set, epochs=5)
The call to fit() throws the following error:
ValueError: Input 0 of layer sequential is incompatible with the layer: expected axis -1 of input shape to have value 12 but received input with shape [12, 1]
Walking through the code:
The input CSV file has thirteen columns - with the last being the label
This is converted to a 3 bit one-hot encoding
The Dataset is constructed of two Tensors - one of shape (12,) and the other of shape (3,)
The network Input layer defines it's expected shape as be the value data shape ignoring the first axis which is the batch size
I am stumped about why there is mismatch between the shape of the data and the expected data shape for the network - especially as the latter is defined by reference to the former.
Add .batch() at the end of the dataset:
data_set = tf.data.Dataset.from_tensor_slices((data.values, target)).batch(8)
I want to create a shallow network that would take a vector and pass it through a network.
I have a vector that is of size 6. vec = [0,1,4,5,1,4,5]
My network:
vec_a = Input(shape=(6,))
x_1 = Convolution1D(nb_filter=10, filter_length=1, input_shape=(1, 6), activation='relu')(vec_a)
x_1 = Dense(16, activation='relu')(x_1)
But I keep getting:
ValueError: Input 0 is incompatible with layer conv1d_1: expected
ndim=3, found ndim=2
The shape of the training data to the fit function is:
(36400, 6)
You have to reshape the input data to have the correct input dimension, e.g.:
your_input_array.reshape(-1, 6, 1)
In addition your input layer should look like:
vec_a = Input(shape=(6,1))
The reason is that the 1D in Conv1D relates to the use of a sequence. But this sequence can have a vector of multiple values at each position. In your case it is the same, but you have "only" a vector of length 1 in the last dimension.