Develop Reference

How many layers should I stack in a sequential model?

I am trying to train a sequential model using the LSTM layer.
The size of sequence data for learning is as follows:
x = np.array(sequences)
y = to_categorical(labels).astype(int)
x.shape => (1800, 34, 48)
y.shape => (1800, 20)
After that, I make a sequential model and try to stack the LSTM layer and the dense layer, but I don't know how much to do that.
First, I did something like this:
model = Sequential()
model.add(LSTM(64, return_sequences=True, activation='relu', input_shape=x_train.shape[1:3]))
model.add(LSTM(128, return_sequences=True, activation='relu'))
model.add(LSTM(64, return_sequences=False, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(actions.shape[0], activation='softmax'))
model.compile(optimizer='Adam', loss='categorical_crossentropy', metrics=['acc'])
model.summary()
However, this doesn't seem to fit my case as I followed someone else's code.
How many layers should I stack in a sequential model?

expected dense_8 to have shape (2,) but got array with shape (1,)

I am trying to create a multi-channel 1D CNN for analyzing ECG signals. I have 258 12 lead ECGs with length 300 samples, so my input dimension is (258, 300, 12).
model = Sequential()
model.add(Conv1D(filters=64, kernel_size=10, activation='relu', input_shape=(n_timesteps,n_features), padding='same'))
model.add(MaxPooling1D(pool_size=2))
model.add(Conv1D(filters=64, kernel_size=10, activation='relu', padding='same'))
model.add(MaxPooling1D(pool_size=2))
model.add(Conv1D(filters=64, kernel_size=10, activation='relu', padding='same'))
model.add(MaxPooling1D(pool_size=2))
model.add(Conv1D(filters=64, kernel_size=10, activation='relu', padding='same'))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(100, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))
model.summary()
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
history = model.fit(X_train, y_train, epochs=10, batch_size=20, verbose=1, validation_split = 0.2)
I'm running the code above, and getting the following error
ValueError: Error when checking target: expected dense_8 to have shape (2,) but got array with shape (1,)
Thanks for any help!

You are trying to train the model like this,
model.fit(X_train, y_train, epochs=10, batch_size=20, verbose=1, validation_split = 0.2)
The shape of y_train is something like (n, 1), where n is the number of samples used to train.
Now, you are building a model with last layer like this,
model.add(Dense(num_classes, activation='softmax'))
From the error message, it can be deduced that you are setting num_classes=2. So, the last layer will have 2 nodes. Such a model expects y_train to be of shape (n,2). But you are using y_train of shape (n,1).
In order to fix the error, you can change the last layer as,
num_classes = 1
model.add(Dense(num_classes, activation='sigmoid'))
Note that, the activation function should be changed to sigmoid.

So, you are solving a binary classification problem.
The error message indicates our model expects label with shape (2,) and i assume you are using num_classes=2. However, Your label is either 1 or 0 as the provided shape of label is (1,). To solve this error, you have yo change the output dense layer of your model, and the layer should have one neuron with sigmoid activation function.
model.add(Dense(num_classes, activation='sigmoid')) # num_classes=1

What should my input to a keras conv1D layer be and what should the input_shape be?

Note: First time posting. I've tried to be thorough in my description
I've been trying to set up what I thought would be a very simple CNN by following this tutorial:
https://machinelearningmastery.com/cnn-models-for-human-activity-recognition-time-series-classification/
My Xtrain dataset is a time series as a numpy array with 34396 rows (samples) and 600 columns (time steps). My Ytrain dataset is just an array containing labels 0,1, or 2 (as ints). I'm just trying to use the CNN to perform multi-classification.
I'm running into an issue getting errors like
Input 0 is incompatible with layer conv1d_39: expected ndim=3, found
ndim=4
when input_shape=(n_timesteps,n_features,n_outputs)
or
Error when checking input: expected conv1d_40_input to have 3
dimensions, but got array with shape (34396, 600)
when input_shape=(n_timesteps,n_features)
I've been searching online for hours now but I can't seem to find a solution to my problem. I think its a simple problem with my data format and the input_shape values but I haven't been able to fix it.
I've tried setting input_shape to
(None, 600, 1)
(34396,600, 1)
(34396,600)
(None,600)
among various other combinations.
train_df = pd.read_csv('training.csv')
test_df = pd.read_csv('test.csv')
x_train=train_df.iloc[:,2:].values
y_train=train_df.iloc[:,1].values
x_test=train_df.iloc[:,2:].values
y_test=train_df.iloc[:,1].values
n_rows=len(x_train)
n_cols=len(x_train[0])
def evaluate_model(trainX, trainy, testX, testy):
verbose, epochs, batch_size = 0, 10, 32
n_timesteps, n_features, n_outputs = trainX.shape[0], trainX.shape[1], 3
print(n_timesteps, n_features, n_outputs)
model = Sequential()
model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(n_timesteps,n_features,n_outputs)))
model.add(Conv1D(filters=64, kernel_size=3, activation='relu'))
model.add(Dropout(0.5))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(100, activation='relu'))
model.add(Dense(n_outputs, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit network
model.fit(trainX, trainy, epochs=epochs, batch_size=batch_size, verbose=verbose)
# evaluate model
_, accuracy = model.evaluate(testX, testy, batch_size=batch_size, verbose=0)
return accuracy
evaluate_model(x_train,y_train,x_test,y_test)

As given in the keras doc, for Conv1D, for example input_shape=(10, 128) for time series sequences of 10 time steps with 128 features per step.
So for your case since you have 600 timesteps each of 1 feature it should be input_shape=(600,1).
Also you have to feed your labels y's as one-hot-encoded.
Working code
from keras.utils import to_categorical
model = Sequential()
model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(600,1)))
model.add(Conv1D(filters=64, kernel_size=3, activation='relu'))
model.add(Dropout(0.5))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(100, activation='relu'))
model.add(Dense(10, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='adam', metrics=['accuracy'])
x = np.random.randn(100,600)
y = np.random.randint(0,10, size=(100))
# Reshape to no:of sample, time_steps, 1 and convert y to one hot encoding
model.fit(x.reshape(100,600,1), to_categorical(y))
# Same as model.fit(np.expand_dims(x, 2), to_categorical(y))
Output:
Epoch 1/1
100/100 [===========================] - 0s 382us/step - loss: 2.3245 - acc: 0.0800

Creating an RNN with Keras Python

I'm new to machine learning and Keras. I made an Neural Network with Keras for regression looking like this:
model = Sequential()
model.add(Dense(57, input_dim=44, kernel_initializer='normal',
activation='relu'))
model.add(Dense(45, activation='relu'))
model.add(Dense(35, activation='relu'))
model.add(Dense(20, activation='relu'))
model.add(Dense(18, activation='relu'))
model.add(Dense(15, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(5, activation='relu'))
model.add(Dense(5, activation='relu'))
model.add(Dense(1, activation='linear'))
My data has 44 dimensions, so could you please give me an example how could i make an RNN. I'm trying like this:
model = Sequential()
model.add(LSTM(44, input_shape=(6900, 44), ))
model.add(Dense(1))
model.compile(loss='mape', optimizer='adam', metrics=['mse', 'mae', 'mape'])
model.fit(X_train, y_train, epochs=100, batch_size=10, verbose=1)
But i get this error:
Error when checking input: expected lstm_13_input to have 3 dimensions, but got array with shape (6900, 44)

As far as I understood you, your data is 44 dimensional and not a time series. An RNN is computing operations on a sequence of data, i.e. a 2D and not a 1D tensor. But you can still use a RNN for 1D vectors, by interpreting them not as one n-dimensional vector but as a time series of n steps, each containing a 1D vector.
model = Sequential()
model.add(Reshape((-1, 1)
model.add(LSTM(44, input_shape=(6900, 44), ))
model.add(Dense(1))
model.compile(loss='mape', optimizer='adam', metrics=['mse', 'mae', 'mape'])
model.fit(X_train, y_train, epochs=100, batch_size=10, verbose=1)

How to make RNN with same input as CNN?

I have a working cnn with text
my X_train's shape is (39971, 10000) , y_train: (39971, 4)
max_words = 10000
model = Sequential()
model.add(Dense(512, input_shape=(max_words,), activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(256, activation='sigmoid'))
model.add(Dropout(0.5))
model.add(Dense(4, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
this works so far.
And this his is how I'm trying to make the RNN:
model = Sequential()
model.add(Embedding(max_words, 128))
model.add(LSTM(64, return_sequences=True, dropout=0.5, recurrent_dropout=0.5))
model.add(Dense(4, activation='softmax'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
I can compile the model but when I try to run model.fit with my training data i get an error:
ValueError: Error when checking target: expected dense_42 to have 3 dimensions, but got array with shape (39971, 4)
What does this mean? How could I fix this?
As this suggest I might need to add input_shape but I'm not sure with what values.

The LSTM layer is currently returning the full sequence, so the output of the network has shape (batch_size, nb_timesteps, 4). I presume you want to use the last LSTM output only. If that's the case, set return_sequences=False.