Keras MLP extracting predictions from each cross validation - python

I have built a sequential Keras MLP and would like to extract the predicted labels (Not Accuracy) at each cross validation (CV=5). I also need to extract the cross validation X_test data that it tests on at each iteration.
Is this possible?
model = Sequential()
model.add(Dense(units = 56, input_dim = 11, activation = "relu",
kernel_initializer= initializer))
model.add(Dropout(0.2))
model.add(Dense(units = 28, activation = "relu"))
model.add(Dropout(0.2))
model.add(Dense(units = 1, activation = "sigmoid"))
model.compile(loss='binary_crossentropy', optimizer=GradDesc12,
metrics=['accuracy'])
model.fit(X_train,y_train, epochs= 20, batch_size =
128,verbose=0)
kFold = StratifiedKFold(n_splits=5)
for train, test in kFold.split(X_data, Y_labels):
Thank you

Related

How do you define input shape without using your input?

I am training to figure out how to define the input shape of my training data without using X_train in the model creation process. I know that LSTM network accepts only 3D arrays as input and my data is prepared with this for loop
X_train = []
y_train = []
for i in range(10, 400):
X_train.append(training_set_scaled[i-10:i, 0])
y_train.append(training_set_scaled[i, 0])
X_train, y_train = np.array(X_train), np.array(y_train)
X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))
Below is the model that I am using.
model = Sequential()
model.add(LSTM(units = 100, return_sequences = True, input_shape = (X_train.shape[1], 1)))
model.add(Dropout(0.2))
model.add(LSTM(units=1000 , return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(units=1000 , return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(units=100))
model.add(Dropout(0.2))
model.add(Dense(units=1 ))
model.compile(optimizer=opt, loss='mean_squared_error' , metrics=[tf.keras.metrics.RootMeanSquaredError()])
As you can see the first LSTM layer has X_train included in the input shape is there any way to define the input shape without using X_train, it knows that it needs to use X_train for training because we have model.fit that takes X_train as one of its arguments
If I do the following my model does not train in the same way
model = Sequential()
model.add(LSTM(units = 100, return_sequences = True))
model.add(Dropout(0.2))
model.add(LSTM(units=1000 , return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(units=1000 , return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(units=100))
model.add(Dropout(0.2))
model.add(Dense(units=1 ))
model.compile(optimizer=opt, loss='mean_squared_error' , metrics=[tf.keras.metrics.RootMeanSquaredError()])
Did you try using None instead of the X_train? Like this?
model.add(LSTM(units = 100, return_sequences = True, input_shape = (None , 1)))

How can I extract Flatten Layer Output for each epoch?

model = Sequential()
model.add(Conv2D(50, (5,5), activation='relu', input_shape =(5,5,1), kernel_initializer='he_normal'))
model.add(Flatten())
model.add(Dense(1, activation='sigmoid'))
model.summary()
# compile the model
model.compile(loss='binary_crossentropy', optimizer= 'adam', metrics=['accuracy'])
model_checkpoint=ModelCheckpoint(r'C:\Users\globo\Desktop\Test_CNN\Results\Kernel5x5\Weights'+'\\'+test+'\model_test{epoch:02d}.h5',save_freq=1,save_weights_only=True)
# fit the model
history = model.fit(X_train, Y_train, epochs=10, batch_size=32, verbose=1, callbacks=[model_checkpoint], shuffle=True, validation_split=0.5)
I'm already extracting weights for each epoch with "ModelCheckpoint", but how can I extract flatten layer output for each epoch and save them?
doing this with sequential models is not feasible at all.
you should use functional API
inp = Input((5,5,1))
x = Conv2D(50, (5,5), activation='relu', kernel_initializer='he_normal')(inp)
xflatten = Flatten()(x)
out = Dense(1, activation='sigmoid')(xflatten)
main_model = Model(inp, out) # this works same as your model
flatten_model = Model(inp, xflatten) # and this only outputs the flatten layer and is not necessary to compile it because we won't train it, it just shows the output of a layer
main_model.compile(loss='binary_crossentropy', optimizer= 'adam', metrics=['accuracy'])
history = main_model.fit(X_train, Y_train, epochs=10, batch_size=32, verbose=1, callbacks=[model_checkpoint], shuffle=True, validation_split=0.5)
to see the flatten layers's output:
flatten_model.predict(X)

Can we change the input_length in a trained model?

I trained the following model
model = Sequential()
model.add(Embedding(10000, 100, input_length = 10, weights=[embedding_matrix], trainable = False))
model.add(Bidirectional(LSTM(64, return_sequences = True)))
model.add(Dense(512, activation='relu'))
model.add(Dense(2 activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam')
model.fit(x, y, epochs=10)
But, I got a input of length 100, while predicting.
So, i would like to know if I can change the value of input_length, according to length of the input while predicting?
If yes then how will that effect the model, or should I use encoders and decoders model?
This is what I have found
model._layers[0].batch_input_shape = (None,500)
new_model = model_from_json(model.to_json())
new_model.summary()

How to fine tune the network automatically in Keras?

How to tune the network automatically instead of adjusting the number of hidden layers and epochs everytime manually? (Using Keras)
from keras.models import Sequential
from keras.layers import Dense
import numpy
seed = 9
numpy.random.seed(seed)
from pandas import read_csv
filename = 'BBCN.csv'
dataframe = read_csv(filename)
array = dataframe.values
x = array[:,0 : 11]
y = array[:, 11]
model = Sequential()
model.add(Dense(11, input_dim=11, kernel_initializer = 'uniform', z = 'relu'))
model.add(Dense(8, kernel_initializer = 'uniform', activation = 'relu'))
model.add(Dense(8, kernel_initializer = 'uniform', activation = 'relu'))
model.add(Dense(1, kernel_initializer = 'uniform', activation = 'sigmoid'))
model.compile(loss='binary_crossentropy', optimizer ='adam', metrics = ['accuracy'])
model.fit(x, y,nb_epoch = 50, batch_size = 10 )
scores = model.evaluate(x,y)
print("%s, %.2f%%" % (model.metrics_names[1], scores[1]*100))
The result I need is to show the process and the percentage of the accuracy.
Thanks a lot!
You could start with a simple loop over some hyperparameters and train with these for some epochs and then compare the results.
You can also look into grid search which is a more systematic approach. Basically you setup a function that creates a model and use it with a set of hyperparameters that you want to try out and an array of values. For more details and boilerplate code I recommend this tutorial.

concatenate flatten output with and other datasets keras python

have 2 datasets, for the first data set i want to apply convolution and keep the result of flatten layyer then concatenate it with an other data set and a do a simple feed forward it is possible with keras ?
def build_model(x_train,y_train):
np.random.seed(7)
left = Sequential()
left.add(Conv1D(nb_filter= 6, filter_length=3, input_shape= (48,1),activation = 'relu', kernel_initializer='glorot_uniform'))
left.add(Conv1D(nb_filter= 6, filter_length=3, activation= 'relu'))
#model.add(MaxPooling1D())
print model
#model.add(Dropout(0.2))
# flatten layer
#https://www.quora.com/What-is-the-meaning-of-flattening-step-in-a-convolutional-neural-network
left.add(Flatten())
left.add(Reshape((48,1)))
right = Sequential()
#model.add(Reshape((48,1)))
# Compile model
model.add(Merge([left, right], mode='sum'))
model.add(Dense(10, 10))
epochs = 100
lrate = 0.01
decay = lrate/epochs
sgd = SGD(lr=lrate, momentum=0.9, decay=decay, nesterov=False)
#clipvalue=0.5)
model.compile(loss='mean_squared_error', optimizer='Adam')
model.fit(x_train,y_train, nb_epoch =epochs, batch_size=10, verbose=1)
#model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'] , )
return model
You need to look at the functional API. The sequential model you are using is not designed to take multiple network inputs.
Follow the "Multi-input and multi-output models" example and you will have it working in no time!

Categories

Resources