How to give a sequece of numpy array as input to CNN - python

How can I reshape a sequence of arrays of shape (90,30,1662)? Meaning 90 arrays with 30 frames each and 1662 keypoints for each frames.And 90 array meaning 30 videos of numpy arrays for a word with 30 frames per video.
x_train, x_test, y_train, y_test=train_test_split(x, y, test_size=0.05)
x_train.shape ---->(85, 30, 1662)
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense
from tensorflow.keras.callbacks import TensorBoard
model = Sequential()
model.add(LSTM(64, return_sequences=True, activation='relu', input_shape=(30,1662)))
model.add(LSTM(128, return_sequences=True, activation='relu'))
model.add(LSTM(64, return_sequences=False, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(actions.shape[0], activation='softmax'))
How can I add CNN before the LSTM?

Reference: https://machinelearningmastery.com/cnn-long-short-term-memory-networks/
You can add CNN model first(with input shape=(30,90,1622)), and use LTSM model to encapsulate CNN model.
It will look like this:
cnn = Sequential()
cnn.add(Conv2D(your output size, (your filter size,your filter size),
activation='relu', padding='same', input_shape=(30,90,1622)))
cnn.add(MaxPooling2D(pool_size=(2, 2)))
cnn.add(Flatten())
model = Sequential()
model.add(TimeDistributed(cnn, ...)) # convert to LTSM type
model.add(LSTM(..))
model.add(Dense(...))

Related

How many layers should I stack in a sequential model?

I am trying to train a sequential model using the LSTM layer.
The size of sequence data for learning is as follows:
x = np.array(sequences)
y = to_categorical(labels).astype(int)
x.shape => (1800, 34, 48)
y.shape => (1800, 20)
After that, I make a sequential model and try to stack the LSTM layer and the dense layer, but I don't know how much to do that.
First, I did something like this:
model = Sequential()
model.add(LSTM(64, return_sequences=True, activation='relu', input_shape=x_train.shape[1:3]))
model.add(LSTM(128, return_sequences=True, activation='relu'))
model.add(LSTM(64, return_sequences=False, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(actions.shape[0], activation='softmax'))
model.compile(optimizer='Adam', loss='categorical_crossentropy', metrics=['acc'])
model.summary()
However, this doesn't seem to fit my case as I followed someone else's code.
How many layers should I stack in a sequential model?

ValueError: Shapes (None, 1) and (None, 64) are incompatible Keras

I'm trying to build a sequential model . I have 32 features as the input dimension and it's a classification problem.
this is the result of the summary :
and this is my model:
#Create an ANN using Keras and Tensorflow backend
from keras.wrappers.scikit_learn import KerasClassifier
from keras.models import Sequential
from keras.layers import Dense, Dropout,Activation
from keras.optimizers import Adam,SGD
nb_epoch = 200
batch_size = 64
input_d = X_train.shape[1]
model = Sequential()
model.add(Dense(512, activation='relu', input_dim=input_d))
model.add(Dropout(0.5))
model.add(Dense(128, activation='relu', input_dim=input_d))
model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.3))
model.add(Activation('softmax'))
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
rms = 'rmsprop'
model.compile(loss='categorical_crossentropy',
optimizer=sgd,
metrics=['accuracy'])
the test and train shape are both 32. I get the ValueError: Shapes (None, 1) and (None, 64) are incompatible error whnever I want to fit the model but I have no idea why.
Much thanks.
The loss function is expecting a tensor of shape (None, 1) but you give it (None, 64). You need to add a Dense layer at the end with a single neuron which will get the final results of the calculation:
model = Sequential()
model.add(Dense(512, activation='relu', input_dim=input_d))
model.add(Dropout(0.5))
model.add(Dense(128, activation='relu', input_dim=input_d))
model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.3))
model.add(Dense(1, activation='softmax'))

expected dense_8 to have shape (2,) but got array with shape (1,)

I am trying to create a multi-channel 1D CNN for analyzing ECG signals. I have 258 12 lead ECGs with length 300 samples, so my input dimension is (258, 300, 12).
model = Sequential()
model.add(Conv1D(filters=64, kernel_size=10, activation='relu', input_shape=(n_timesteps,n_features), padding='same'))
model.add(MaxPooling1D(pool_size=2))
model.add(Conv1D(filters=64, kernel_size=10, activation='relu', padding='same'))
model.add(MaxPooling1D(pool_size=2))
model.add(Conv1D(filters=64, kernel_size=10, activation='relu', padding='same'))
model.add(MaxPooling1D(pool_size=2))
model.add(Conv1D(filters=64, kernel_size=10, activation='relu', padding='same'))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(100, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))
model.summary()
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
history = model.fit(X_train, y_train, epochs=10, batch_size=20, verbose=1, validation_split = 0.2)
I'm running the code above, and getting the following error
ValueError: Error when checking target: expected dense_8 to have shape (2,) but got array with shape (1,)
Thanks for any help!
You are trying to train the model like this,
model.fit(X_train, y_train, epochs=10, batch_size=20, verbose=1, validation_split = 0.2)
The shape of y_train is something like (n, 1), where n is the number of samples used to train.
Now, you are building a model with last layer like this,
model.add(Dense(num_classes, activation='softmax'))
From the error message, it can be deduced that you are setting num_classes=2. So, the last layer will have 2 nodes. Such a model expects y_train to be of shape (n,2). But you are using y_train of shape (n,1).
In order to fix the error, you can change the last layer as,
num_classes = 1
model.add(Dense(num_classes, activation='sigmoid'))
Note that, the activation function should be changed to sigmoid.
So, you are solving a binary classification problem.
The error message indicates our model expects label with shape (2,) and i assume you are using num_classes=2. However, Your label is either 1 or 0 as the provided shape of label is (1,). To solve this error, you have yo change the output dense layer of your model, and the layer should have one neuron with sigmoid activation function.
model.add(Dense(num_classes, activation='sigmoid')) # num_classes=1

Saving the specific layer from within a sequential Keras model

I am building an auto-encoder and training the model so the targeted output is the same as the input.
I am using a sequential Keras model. When I use model.predict I would like it to export the array from a specific layer (Dense256) not the output.
This is my current model:
model = Sequential()
model.add(Dense(4096, input_dim = x.shape[1], activation = 'relu'))
model.add(Dense(2048, activation='relu'))
model.add(Dense(1024, activation='relu'))
model.add(Dense(512, activation='relu'))
model.add(Dense(256, activation='relu'))
model.add(Dense(512, activation='relu'))
model.add(Dense(1024, activation='relu'))
model.add(Dense(2048, activation='relu'))
model.add(Dense(4096, activation='relu'))
model.add(Dense(x.shape[1], activation ='sigmoid'))
model.compile(loss = 'mean_squared_error', optimizer = 'adam')
history = model.fit(data_train,data_train,
verbose=1,
epochs=10,
batch_size=256,
shuffle=True,
validation_data=(data_test, data_test))
After training, create a new model (model2) from your trained model (model) ending in your desired layer.
You can do so either with layer name:
(In model.summary(), your dense's layer 'name' with 256 neurons is dense_5)
from keras.models import Model
model2= Model(model.input,model.get_layer('dense_5').output)
Or with layer order:
(your dense layer with 256 neurons is fifth in model.summary())
from keras.models import Model
model2= Model(model.input,model.layers[4].output)
Then you can use predict
preds=model2.predict(x)
layer.get_weights() returns the weights of a layer as a numpy array which can then be saved, for example with np.save.
To set the weights from a numpy array, layer.set_weights(weights) can be used.
You can access your layer either by name (model.get_layer(LAYER_NAME) or by its number (model.layers[LAYER_INDEX]).

Creating an RNN with Keras Python

I'm new to machine learning and Keras. I made an Neural Network with Keras for regression looking like this:
model = Sequential()
model.add(Dense(57, input_dim=44, kernel_initializer='normal',
activation='relu'))
model.add(Dense(45, activation='relu'))
model.add(Dense(35, activation='relu'))
model.add(Dense(20, activation='relu'))
model.add(Dense(18, activation='relu'))
model.add(Dense(15, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(5, activation='relu'))
model.add(Dense(5, activation='relu'))
model.add(Dense(1, activation='linear'))
My data has 44 dimensions, so could you please give me an example how could i make an RNN. I'm trying like this:
model = Sequential()
model.add(LSTM(44, input_shape=(6900, 44), ))
model.add(Dense(1))
model.compile(loss='mape', optimizer='adam', metrics=['mse', 'mae', 'mape'])
model.fit(X_train, y_train, epochs=100, batch_size=10, verbose=1)
But i get this error:
Error when checking input: expected lstm_13_input to have 3 dimensions, but got array with shape (6900, 44)
As far as I understood you, your data is 44 dimensional and not a time series. An RNN is computing operations on a sequence of data, i.e. a 2D and not a 1D tensor. But you can still use a RNN for 1D vectors, by interpreting them not as one n-dimensional vector but as a time series of n steps, each containing a 1D vector.
model = Sequential()
model.add(Reshape((-1, 1)
model.add(LSTM(44, input_shape=(6900, 44), ))
model.add(Dense(1))
model.compile(loss='mape', optimizer='adam', metrics=['mse', 'mae', 'mape'])
model.fit(X_train, y_train, epochs=100, batch_size=10, verbose=1)

Categories

Resources