I'm training my data in batches using train_on_batch, but it seems train_on_batch doesn't have an option to use callbacks, which seems to be a requirement to use checkpoints.
I can't use model.fit as that seems to require I load all of my data into memory.
model.fit_generator is giving me strange problems (like hanging at end of an epoch).
Here is the example from Keras API docs showing the use of ModelCheckpoint:
from keras.callbacks import ModelCheckpoint
model = Sequential()
model.add(Dense(10, input_dim=784, kernel_initializer='uniform'))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy', optimizer='rmsprop')
checkpointer = ModelCheckpoint(filepath='/tmp/weights.hdf5', verbose=1,
save_best_only=True)
model.fit(x_train, y_train, batch_size=128, epochs=20,
verbose=0, validation_data=(X_test, Y_test), callbacks=[checkpointer])
If you train on each batch manually, you can do whatever you want at any #epoch(#batch). No need to use callback, just call model.save or model.save_weights.
Related
I have an LSTM model that was trained on the multi-feature daily dataset and predicts the target feature's value one day in the future.
How should I retrain the model each day as the new data becomes available? Should I rerun the model.fit with the full dataset (which gets updated each day) like in the example below?
model.fit(x_train, y_train, epochs=50, batch_size=20,
validation_data=(x_test, y_test), verbose=2, shuffle=False)
Or I can call model.fit only with the newly available data?
# run at the beggining once
model.fit(x_train, y_train, epochs=50, batch_size=20,
validation_data=(x_test, y_test), verbose=2, shuffle=False)
# run every day as the new data gets available.
model.fit(x_yesterday, x_yesterday)
Assuming you are using Keras, I would use train_on_batch See this previous question for the answer: What is the use of train_on_batch() in keras?
I created a neural network in python that is predicting my time-series very well.
My issue is I want to be able to create a neural network that can predict multiple time series at the same time.
Is this possible and how would I go about it?
This is the code to build the NN for a single time series
nn_model = Sequential()
nn_model.add(Dense(12, input_dim=1, activation='relu'))
nn_model.add(Dense(1))
nn_model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mse', 'mae'])
early_stop = EarlyStopping(monitor='loss', patience=2, verbose=1)
history = nn_model.fit(X_train, y_train, epochs=100, batch_size=1, verbose=1, callbacks=[early_stop], shuffle=False)
Any ideas about how to convert this to run for multiple time series?
I trained a binary classifier then saved the model as a .h5 file.
Originally I had been using fit_generator to train it.
model.fit_generator(
train_generator,
class_weight=class_weights,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size,
callbacks=my_callbacks
)
When I call load_model and load it again to continue training where I left off, do I use the exact same code for fit_generator, referring to the old train_generator and validation_generator?
The Keras documentation is very sparse on this and there are virtually no examples of using load_model with fit_generator together online.
Yes. Why wouldn't you? The load will load the weights. The generator will generate new training samples. Of course it may not start exactly where it finished but as long as shuffled will be fine.
I have an algorithm written in python, it is timeseries analysis using LSTM. My professor asked me to show the details of the model that is created in the code. How do I inspect the "model" here? Does it have some visualization of the model in the background?
model = Sequential()
model.add(LSTM(50, input_shape=(trainX.shape[1], trainX.shape[2])))
model.add(Dropout(0.2))
model.add(Dense(1))
model.compile(loss='mae', optimizer='adam')
history = model.fit(trainX, trainY, epochs=50, batch_size=72, validation_data=(testX, testY), verbose=0, shuffle=False)
There is a visualization tool in Keras called plot_model. You can use it to save your model as an image where you can see the structure of your model including input and output dimensions.
from keras.utils import plot_model
plot_model(model, to_file='model.png')
You can read more about it here: Keras Visualization
I am currently trying train a regression network using keras. To ensure I proper training I've want to train using crossvalidation.
The Problem is that it seems that keras don't have any functions supporting crossvalidation or do they?
The only solution I seemed to have found is to use scikit test_train_split and run a model.fit for for each k fold manually. Isn't there a already an integrated solutions for this, rather than manually doing it ?
Nope... That seem to be the solution. (Of what I know of.)
There is a scikit learn wrapper for Keras that will help you do this easily: https://keras.io/scikit-learn-api/
I recommend reading Dr. Jason Brownlee's example: https://machinelearningmastery.com/regression-tutorial-keras-deep-learning-library-python/
def baseline_model():
# create model
model = Sequential()
model.add(Dense(13, input_dim=13, kernel_initializer='normal', activation='relu'))
model.add(Dense(1, kernel_initializer='normal'))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam')
return model
estimator = KerasRegressor(build_fn=wider_model, nb_epoch=100, batch_size=5, verbose=0)
kfold = KFold(n_splits=10, random_state=seed)
results = cross_val_score(pipeline, X, Y, cv=kfold)