I trained a binary classifier then saved the model as a .h5 file.
Originally I had been using fit_generator to train it.
model.fit_generator(
train_generator,
class_weight=class_weights,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size,
callbacks=my_callbacks
)
When I call load_model and load it again to continue training where I left off, do I use the exact same code for fit_generator, referring to the old train_generator and validation_generator?
The Keras documentation is very sparse on this and there are virtually no examples of using load_model with fit_generator together online.
Yes. Why wouldn't you? The load will load the weights. The generator will generate new training samples. Of course it may not start exactly where it finished but as long as shuffled will be fine.
Related
I have a model that I want to train for 10 epochs for a certain hyperparameters setting. After training, I will use the history.history object and find the epoch where the validation loss was at a minimum. Once I have this best scoring epoch, I would like to retrieve this model and use it to predict the test data. Now, imagine that my best scoring epoch was not the last one. Is there any option within this Keras history object (such as history.model) to retrieve past values of weights? I imagine that, if there is not, I would have to create a dictionary and temporarily store each model per epoch until finishing training and finding the best one. But, when using model.fit, there is no option to store each model per epoch right. How would you do this?
Keras offers the option of evaluate your model on validation data after each epoch
After divide your data into trainning, test and validation data you can train you model like this:
model=modelmlp(np.shape(x_trai)[0],hidden,4)
model.compile(loss='categorical_crossentropy', optimizer = 'adam', metrics='accuracy'])
hist=model.fit(x_train,y_train,epochs=epochs,batch_size=batch,
validation_data(x_valid,y_valid),verbose=verbose[1],
callbacks=[ModelCheckpoint(filepath='bestweigths.hdf5',
monitor='val_loss',verbose=verbose[2],save_best_only=True,mode='min')])
model=load_model('bestweigths.hdf5')
This code will train your model and, after each epoch, your model will be evaluated at the validation data. Every time the result on the validation data is improved the model will be saved on a file
After the trainning process end, you just need to load the model from the file
you can use the callback class of keras for this matter.
You can save the model weights based on the metric you need. Let's say for example you need to save the model with minimum loss. You'll have to define a modelcheckpoint.
first before training the model define the checkpoint in the below given format
callbacks = [ModelCheckpoint(filepath='resNet_centering2.h5', monitor='val_loss', mode='min', save_best_only=True)]
now since you have defined callback, you'll have to use use this callbacks in the model.fit call
history = model.fit(
x=X_train,
y=Y_train,
callbacks=callbacks,
batch_size=4,
epochs=100,
verbose=1,
validation_data=(X_test, Y_test))
this will save the best weights of your model at defined filepath and you can fetch those weights using the below given call.
model=load_model('bestweigths.hdf5')
I hope it solves your problem.
In keras, is doing fit() over many single datepoints the same as doing fit() over a dataset? For example, is doing a single
model.fit(train_X,
train_y,
batch_size=1,
epochs=1)
The same as doing
for i in range(len(train_X)):
model.fit([train_X[i]],
[train_y[i]],
batch_size=1,
epochs=1)
Or is it different?
The difference is that model.fit() for a whole dataset will shuffle the samples each epoch, which can help with the learning process. If you do it the looped-way, you it will be updating the weights to the same progression of samples.
model.fit will update the weights after every batch, in your case after every sample. So aside of the shuffling, the two methods you proposed are the same.
I am writing a neural network to train incrementally (not online). Here is a snippet of the code
output = create_model()
model = Model(inputs=values, outputs=output)
if start_epoch > 1:
weights_list = load_model_from_pickle()
model.set_weights(weights_list)
model.compile(loss='binary_crossentropy', optimizer='adam')
model.fit(data , label, epochs=1, verbose=1, batch_size=1024, shuffle=False)
In essence, I want to load previously trained weights and train for a few more epochs. I read some SO reply that calling compile changes the weights? Is there any other way to do it? Does it make sense to set weight after calling compile? Will the answer change if I run my model in multi gpu setting?
You need to compile the model ones and after training when you reload the model, you dont' require to compile it again. Read more here.
Compile function defines the optimizer, loss functions and metrics you want. It does not change any weights. For more detailed information, read here.
When I use EarlyStopping callback does Keras save best model in terms of val_loss or it save model on save_epoch = [best epoch in terms of val_loss] + YEARLY_STOPPING_PATIENCE_EPOCHS ?
If it's second option, how to just save best model?
Here is code snippet:
early_stopping = EarlyStopping(monitor='val_loss', patience=YEARLY_STOPPING_PATIENCE_EPOCHS)
history = model.fit_generator(
train_generator,
steps_per_epoch=100, # 1 epoch = BATCH_SIZE * steps_per_epoch samples
epochs=N_EPOCHS,
validation_data=test_generator,
validation_steps=20,
callbacks=[early_stopping])
#Save train log to .csv
pd.DataFrame(history.history).to_csv('vgg16_binary_crossentropy_train_log.csv', index=False)
model.save('vgg16_binary_crossentropy.h5')
In v2.2.4+ of Keras, EarlyStopping has a restore_best_weights parameter which, when set to True, will set the model to the state of best CV performance. For example:
EarlyStopping(restore_best_weights=True)
From my experience using the 'earlystopping' callback, the model will not be saved automatically...it will just stop training and when you save it manually, it will be the second option you present.
To have your model save each time val_loss decreases, see the following documentation page:
https://keras.io/callbacks/ and look at the "Example: model checkpoints" section which will tell you exactly what to do.
note that if you wish to re-use your saved model, I have had better luck using 'save_weights' in combo with saving the architecture in json. YMMV.
I'm training my data in batches using train_on_batch, but it seems train_on_batch doesn't have an option to use callbacks, which seems to be a requirement to use checkpoints.
I can't use model.fit as that seems to require I load all of my data into memory.
model.fit_generator is giving me strange problems (like hanging at end of an epoch).
Here is the example from Keras API docs showing the use of ModelCheckpoint:
from keras.callbacks import ModelCheckpoint
model = Sequential()
model.add(Dense(10, input_dim=784, kernel_initializer='uniform'))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy', optimizer='rmsprop')
checkpointer = ModelCheckpoint(filepath='/tmp/weights.hdf5', verbose=1,
save_best_only=True)
model.fit(x_train, y_train, batch_size=128, epochs=20,
verbose=0, validation_data=(X_test, Y_test), callbacks=[checkpointer])
If you train on each batch manually, you can do whatever you want at any #epoch(#batch). No need to use callback, just call model.save or model.save_weights.