I have a rnn and want to feed in a sentence with a length of 50, and have the output be the same length. (For a chatbot). Does anyone know why this error:
ValueError: Input 0 of layer lstm is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, 5000]
keeps appearing? Here is the code:
def model():
model = Sequential()
model.add(Embedding(vocab_size, 100, input_length=l))
model.add(Flatten())
model.add(LSTM(100, return_sequences=True))
model.add(LSTM(100))
model.add(Dense(100, activation='relu'))
model.add(Dense(vocab_size, activation='softmax'))
model.summary()
return model
model = model()
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(padded_x, padded_y, batch_size=128, epochs=100)
The shape of both arrays are 5000, 50....5000 sentences with 50 words each. They are already encoded. I first though it was because I was flattening...but this is the error before flattening:
ValueError: A target array with shape (5000, 50) was passed for an output of shape (None, 12097) while using as loss `categorical_crossentropy`. This loss expects targets to have the same shape as the output.
##BTW the vocab_size is 12097##
Don't flatten. The size of the output you're expecting is 50, so you need 50 neurons in your last dense layer.
def model():
model = Sequential()
model.add(Embedding(vocab_size, 100, input_length=l))
model.add(LSTM(100, return_sequences=True))
model.add(LSTM(100))
model.add(Dense(100, activation='relu'))
model.add(Dense(50, activation='softmax')) # 50 here, not 12,097
model.summary()
return model
Related
I am trying to run a simple RNN with some data extracted from a csv file. I have already preprocessed my data and split them into train set and validation set, but I get the error above.
This is my network structure and what I tryied so far. My shapes are (33714,12) for x_train, (33714,) for y_train, (3745,12) for x_val and (3745,) for y_val.
model = Sequential()
# LSTM LAYER IS ADDED TO MODEL WITH 128 CELLS IN IT
model.add(LSTM(128, input_shape=x_train.shape, activation='tanh', return_sequences=True))
model.add(Dropout(0.2)) # 20% DROPOUT ADDED FOR REGULARIZATION
model.add(BatchNormalization())
model.add(LSTM(128, input_shape=x_train.shape, activation='tanh', return_sequences=True)) # ADD ANOTHER LAYER
model.add(Dropout(0.1))
model.add(BatchNormalization())
model.add(LSTM(128, input_shape=x_train.shape, activation='tanh', return_sequences=True))
model.add(Dropout(0.2))
model.add(BatchNormalization())
model.add(Dense(32, activation='relu')) # ADD A DENSE LAYER
model.add(Dropout(0.2))
model.add(Dense(2, activation='softmax')) # FINAL CLASSIFICATION LAYER WITH 2 CLASSES AND SOFTMAX
# ---------------------------------------------------------------------------------------------------
# OPTIMIZER SETTINGS
opt = tf.keras.optimizers.Adam(learning_rate=LEARNING_RATE, decay=DECAY)
# MODEL COMPILE
model.compile(loss='sparse_categorical_crossentropy', optimizer=opt, metrics=['accuracy'])
# CALLBACKS
tensorboard = TensorBoard(log_dir=f"logs/{NAME}")
filepath = "RNN_Final-{epoch:02d}-{val_acc:.3f}"
checkpoint = ModelCheckpoint("models/{}.model".format(filepath, monitor='val_acc', verbose=1,
save_best_only=True, mode='max')) # save only the best ones
# RUN THE MODEL
history = model.fit(x_train, y_train, epochs=EPOCHS, batch_size=BATCH_SIZE,
validation_data=(x_val, y_val), callbacks=[tensorboard, checkpoint])
Though it will give you a large value, what may be best to do would be to flatten the one with the larger dimension.
A tensorflow.keras.layers.Flatten() will basically make your output shape the values multiplied, i.e. input: (None, 5, 5) -> Flatten() -> (None, 25)
For your example, this will give you:
(None, 33714,12) -> (None, 404568).
I'm not entirely sure if this will work when you change the shape sizes, but that is how I overcame my issue with incompatible shapes: expected: (None, x), got: (None, y, x).
I am building a Conv1D model with train data (28659, 257) and test data (5053, 257), but I am facing a value error that says: expected min_ndim=3, found ndim=2. Full shape received: [None, 256]
size of datasets
print(train_data.shape)
print(test_data.shape)
model
model = Sequential()
model.add(Conv1D(filters=64, kernel_size=5, activation='relu', input_shape=(256,1)))
model.add(Conv1D(filters=64, kernel_size=3, activation='relu'))
model.add(Dropout(0.5))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(8, activation='relu'))
model.add(Dense(2, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='Adam', metrics=['accuracy'])
model.summary()
opt = keras.optimizers.Adam(learning_rate=0.01)
model.compile(loss='CategoricalCrossentropy', optimizer=opt, metrics=['accuracy'])
history = model.fit(train_data.values[:, 0:256], to_categorical(train_data.values[:, 256]), epochs=180, batch_size=500)
y_pred = model.predict(test_data.values[:, 0:256])
testing accuracy
y_pred = model.predict(test_data.values[:,0:256])
y_pred = (y_pred > 0.5)
accuracy = metrics.accuracy_score(to_categorical(test_data.values[:,256]),y_pred)
print(f'Testing accuracy of the model is {accuracy*100:.4f}%')
The error is from the fit(), but I cannot figure my mistake with the calculation! Any help is appreciated!
try to reshape your train and test data :
X_train=np.reshape(train_data,(train_data.shape[0], train_data.shape[1],1))
X_test=np.reshape(test_data,(test_data.shape[0], test_data.shape[1],1))
You can't feed values like : [1,2,3,4], you have to feed your values like [[1],[2],[3],[4]]
I am trying to create a multi-channel 1D CNN for analyzing ECG signals. I have 258 12 lead ECGs with length 300 samples, so my input dimension is (258, 300, 12).
model = Sequential()
model.add(Conv1D(filters=64, kernel_size=10, activation='relu', input_shape=(n_timesteps,n_features), padding='same'))
model.add(MaxPooling1D(pool_size=2))
model.add(Conv1D(filters=64, kernel_size=10, activation='relu', padding='same'))
model.add(MaxPooling1D(pool_size=2))
model.add(Conv1D(filters=64, kernel_size=10, activation='relu', padding='same'))
model.add(MaxPooling1D(pool_size=2))
model.add(Conv1D(filters=64, kernel_size=10, activation='relu', padding='same'))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(100, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))
model.summary()
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
history = model.fit(X_train, y_train, epochs=10, batch_size=20, verbose=1, validation_split = 0.2)
I'm running the code above, and getting the following error
ValueError: Error when checking target: expected dense_8 to have shape (2,) but got array with shape (1,)
Thanks for any help!
You are trying to train the model like this,
model.fit(X_train, y_train, epochs=10, batch_size=20, verbose=1, validation_split = 0.2)
The shape of y_train is something like (n, 1), where n is the number of samples used to train.
Now, you are building a model with last layer like this,
model.add(Dense(num_classes, activation='softmax'))
From the error message, it can be deduced that you are setting num_classes=2. So, the last layer will have 2 nodes. Such a model expects y_train to be of shape (n,2). But you are using y_train of shape (n,1).
In order to fix the error, you can change the last layer as,
num_classes = 1
model.add(Dense(num_classes, activation='sigmoid'))
Note that, the activation function should be changed to sigmoid.
So, you are solving a binary classification problem.
The error message indicates our model expects label with shape (2,) and i assume you are using num_classes=2. However, Your label is either 1 or 0 as the provided shape of label is (1,). To solve this error, you have yo change the output dense layer of your model, and the layer should have one neuron with sigmoid activation function.
model.add(Dense(num_classes, activation='sigmoid')) # num_classes=1
I am trying to build a simple Convolutional NN with:
340 samples,
260 rows per sample
16 features per row.
I thought the order of the shape is (batch_size, steps, input_dim), which would mean (340, 16, 260) I believe.
Current code:
model = Sequential()
model.add(Conv1D(64, kernel_size=3, activation='relu', input_shape=(340, 16, 260)))
# model.add(Conv2D(64, 2, activation='relu'))
model.add(MaxPooling1D())
# model.add(Conv2D(128, 2, activation='relu'))
model.add(Conv1D(64, kernel_size=3, activation='relu'))
model.add(GlobalAveragePooling1D())
model.add(Dense(1, activation='linear'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
model.summary()
model.fit(xTrain, yTrain, batch_size=16, epochs=1000)
I am getting an error:
ValueError: Input 0 is incompatible with layer conv1d_1: expected ndim=3, found ndim=4
I am very lost and believe that my shapes are off. Could someone help me?
Thank you!
As mentioned in this answer, layers in Keras, accept two arguments: input_shape and batch_input_shape. The difference is that input_shape does not contain the batch size, while batch_input_shape is the full input shape, including the batch size.
Based on this, I think the specification input_shape=(340, 16, 260) tells keras to expect a 4-dimensional input, which is not what you want. The correct argument would be batch_input_shape=(340, 16, 260).
I have a working cnn with text
my X_train's shape is (39971, 10000) , y_train: (39971, 4)
max_words = 10000
model = Sequential()
model.add(Dense(512, input_shape=(max_words,), activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(256, activation='sigmoid'))
model.add(Dropout(0.5))
model.add(Dense(4, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
this works so far.
And this his is how I'm trying to make the RNN:
model = Sequential()
model.add(Embedding(max_words, 128))
model.add(LSTM(64, return_sequences=True, dropout=0.5, recurrent_dropout=0.5))
model.add(Dense(4, activation='softmax'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
I can compile the model but when I try to run model.fit with my training data i get an error:
ValueError: Error when checking target: expected dense_42 to have 3 dimensions, but got array with shape (39971, 4)
What does this mean? How could I fix this?
As this suggest I might need to add input_shape but I'm not sure with what values.
The LSTM layer is currently returning the full sequence, so the output of the network has shape (batch_size, nb_timesteps, 4). I presume you want to use the last LSTM output only. If that's the case, set return_sequences=False.