I would like to set a start point before training cnn. How to set a starting point for a model? Here is my code. Also I wonder if starting point changes each time I retrain the model? Any help is highly appreciated.
model = Sequential()
model.add(layers.Embedding(vocab_size, embedding_dim, input_length=maxlen))
model.add(layers.Conv1D(16, 5, activation='tanh'))
model.add(layers.GlobalMaxPooling1D())
model.add(layers.Dense(3, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
model_path= "sentiment labelled sentences/imdb models/model{epoch:02d}.hdf5"
check=ModelCheckpoint(model_path, monitor='val_loss', verbose=0, save_best_only=False, save_weights_only=False, mode='auto',save_freq='epoch') #modeli her epoch sonunda kaydet
model.compile(optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy'])
history = model.fit(X_train, y_train,
epochs=15,
validation_data=(X_test, y_test),
batch_size=10, callbacks=[check])
As you typed out your neural network model is Sequential(), meaning the first layer is where your model starts (i.e. model.add(layers.Embedding(vocab_size, embedding_dim, input_length=maxlen)))
If you want to have a different input layer, you can just rearrange the lines of code, or code another one.
If you're looking to debug or just start execution midway thru your neural network, I don't think that's possible. This is an issue neural networks have. They are not very interpretable/explainable, meaning they're like a blackbox. You cannot look inside and check out how, for a certain set of datapoints, a NN reaches a prediction. I found this website article expanding on this issue: https://www.altacognita.com/explainability-in-deep-neural-networks-2/
Related
i have a classification problem and i need to find important features.
my code as follow:
model = Sequential()
model.add(Dropout(0.99, input_dim=len(X_train.columns)))
model.add(Dense(100, activation='relu',name='layer1'))
model.add(Dense(1, activation='sigmoid',name='layer'))
print(model.summary())
# compile the keras model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit the keras model on the dataset
model=model.fit(X_train, y_train,validation_data = (X_val, y_val), epochs=100, batch_size=1,)
how can i find important features? as you can see in the code it is not a regression problem.
I am trying to use keras to train a simple feedforward network. I tried two different methods of what I think is the same network, but one is performing significantly better. The first one and the better performing one is the following:
inputs = keras.Input(shape=(384,))
dense = layers.Dense(64, activation="relu")
x = dense(inputs)
x = layers.Dense(64, activation="relu")(x)
outputs = layers.Dense(384)(x)
model = keras.Model(inputs=inputs, outputs=outputs, name="simple_model")
model.compile(loss='mse',optimizer='Adam')
history = model.fit(X_train,
y_train_tf,
epochs=20,
validation_data=(X_test, y_test),
steps_per_epoch=100,
validation_steps=50)
and it settles on a validation loss of about 0.2. The second model performs much worse:
model = keras.models.Sequential()
model.add(Dense(64, input_shape=(384,), activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(384, activation='relu'))
optimizer = tf.keras.optimizers.Adam()
model.compile(loss='mse', optimizer=optimizer)
history = model.fit(X_train,
y_train_tf,
epochs=20,
validation_data=(X_test, y_test),
steps_per_epoch=100,
validation_steps=50)
and this has validation loss of around 5. But when I do model.summary, they look virtually the same. Is there something wrong with the second model?
I am not sure that they are the same since second model has relu activation after last layer (384 units) and first doesn't. This might be the issue since default activation of the Keras dense layer is None.
I'm trying to solve the Titanic competition on Kaggle. But the modelaccuracy isn't going beyond 80%.
I tried to change a number of hidden nodes, a number of epochs, also tried to apply batch normalization, dropout, changing the weights initializations, but there's the same 80%. What am I doing wrong?
This is my code below:
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(10, input_shape=(5,), kernel_initializer='he_normal', activation='relu'))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.Dense(20, kernel_initializer='he_normal', activation='relu'))
model.add(tf.keras.layers.Dropout(0.3))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.Dense(2, kernel_initializer=tf.keras.initializers.GlorotNormal(), activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
train_scores = model.fit(train_features, train_labels, epochs=200, batch_size=64, verbose=2)
And here's on the picture accuracy in some last epochs:model accuracy
How can I improve it?
You can try normalising the data, Generally while implementing Neural Networks we don't need to normalise our data (if the network is deep) but since here we are only working with 3 layers only I guess normalising the data might help.
I would suggest to split your training data again into training and validation set and use K-fold cross validation ( I am not sure about this one!! I too am new in this field).
But in general I have seen if the accuracy is constant then the best approach is to alter the training data ( I mean normalise it or try imputing NaN values with the mean (rather than setting the to 0)).
I am using a very simple MLP with just 1 hidden layer to estimate option prices.
In addition to the actual output of the neural network I would also like to know the partial derivative of the output value (of each line of the data sample) with regard to one of the 6 input parameters such that the resulting value can be interpreted as the percentage change of the output with regard to a change in the input parameter.
As I am pretty new to Keras and Neural Networks in general I was not able to come up with a solution for the problem myself.
# Create Model
model = Sequential()
model.add(Dense(6, input_dim=6)) #input layer
model.add(Dense(10, activation=relu)) #hidden layer
model.add(Dense(1, activation=linear)) #output layer
# Compile Model
model.compile(loss='mse', optimizer='adam', metrics=['mae'])
# Train model
model.fit(X_train, Y_train, epochs=50, batch_size=10 verbose=2, validation_split=0.2)
# Predict Values
Y_pred = model.predict(X_test, batch_size=10)
I have the following Keras model, although it could be generalised to a normal RNN using GRUs.
model = Sequential()
model.add(GRU(40, batch_input_shape=(batch_size, look_back, 1), stateful=True, return_sequences=True))
model.add(GRU(10, batch_input_shape=(batch_size, look_back, features), stateful=True))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
# Train model
iter = 10000
for i in range(iter):
model.fit(trainX, trainY, nb_epoch=1, batch_size=batch_size, verbose=0, shuffle=False)
if (i<(iter-1)):
model.reset_states()
testPred = model.predict(testX,batch_size=batch_size)
print(mean_squared_error(testY,testPred))
If I don't have the if statement with regards to resetting the state, the mean squared error has always been higher. Considering that the test set is right after the train set wouldn't it make sense that you would want to preserve the state of the last memory block?
This tutorial seems to suggest otherwise: http://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/ (i.e. he simply doesn't have that if statement, doesn't explicitly mention anything about keeping the last state).
So just wondering if I am correct about this.