I am trying to use EnsembleVoteClassifier from mlxtend library, where my classifiers are ANN, SVM, Logistic Regression. I am pre-fitting the models and calling EnsembleVoteClassifier just for prediction:
ensemble=EnsembleVoteClassifier(clfs=[model_nn, model_logreg],voting='hard',refit=False)
ensemble.fit(X_train,y_train)
y_pred_ensemble = ensemble.predict(X_test)
The problem is with Keras. My code is below:
model_nn = Sequential()
model_nn.add(Dense(20, input_shape=(X_train.shape[1],),
kernel_initializer=RandomNormal(mean=0.0, stddev=0.05, seed=42),
bias_initializer=RandomNormal(mean=0.0, stddev=0.05, seed=42)))
model_nn.add(Activation('relu'))
model_nn.add(BatchNormalization())
model_nn.add(Dropout(0.5))
model_nn.add(Dense(2, activation='softmax'))
model_nn.compile (loss = 'sparse_categorical_crossentropy', optimizer=k.optimizers.Adam(lr=1e-4))
early_stopping_monitor = EarlyStopping(monitor='val_loss', mode='min', patience=20)
lr_reduce= ReduceLROnPlateau(monitor='val_loss', verbose=1, mode='min', patience=20)
history = model_nn.fit(X_train, y_train, epochs=1000,
class_weight=class_weights,
batch_size=32,
validation_data=(X_val, y_val), verbose = 1,
callbacks=[early_stopping_monitor, lr_reduce])
y_pred_nn = model_nn.predict(X_test)
y_pred_nn = y_pred_nn.argmax(axis=1)
The problem is that the shape of predicted class is (n_samples,2) which produces an error in EnsembleVoteClassifier:
raise ValueError("bad input shape {0}".format(shape))
ValueError: bad input shape (, 2)
Is there any way to pass the pipeline that will take care of the shape problem and output the keras prediction with the same shape as sklearn does?
Thank you.
Related
i want to use a pandas dataset as an input to a neural net.
my neural net model is:
def build_model():
model = Sequential()
model.add(Dense(128, activation = "relu"))
model.add(Dropout(0.2))
model.add(Dense(64, activation = "relu"))
model.add(Dropout(0.1))
model.add(Dense(32, activation = "softmax"))
model.compile(
optimizer='adam',
loss=['binary_crossentropy'],
metrics=['accuracy']
)
return model
tensorboard = TensorBoard(log_dir=f"logs/{time.time()}", histogram_freq=1)
model = build_model()
history = model.fit(
x_train,
y_train,
epochs=5,
batch_size=32,
validation_data=(
x_val,
y_val
),
callbacks=[
tensorboard
]
)
and i pass my dataframe as input as such:
y_val, x_val, y_train, x_train = test_data.drop(['gender',
'comorbidities_count', 'comorbidities_significant_count',
'medication_count'],axis=1),test_data.drop(['fried'],axis=1),training_data.drop([ 'gender', 'comorbidities_count', 'comorbidities_significant_count',
'medication_count'],axis=1),training_data.drop(['fried'],axis=1)
but i get this error:
ValueError: Please provide as model inputs either a single array or a list of arrays.
Does anyone know hot to turn this dataframe into an array so i can feed it? Or is there some other issue i am not in knowledge of?
Use
y_val, x_val, y_train, x_train = test_data.drop(['gender',
'comorbidities_count', 'comorbidities_significant_count',
'medication_count'],axis=1).to_numpy().astype(np.float32) ,test_data.drop(['fried'],axis=1).to_numpy().astype(np.float32) ,training_data.drop([ 'gender', 'comorbidities_count', 'comorbidities_significant_count',
'medication_count'],axis=1).to_numpy().astype(np.float32) ,training_data.drop(['fried'],axis=1).to_numpy().astype(np.float32)
The .to_numpy() function of a pd dataframe turns it into a numpy array.
Here is a snapshot of my dataset, including its shape:
Now, here is the code I am using to build the NN:
# define the architecture of the network
model = Sequential()
model.add(Dense(50, input_dim=X_train.shape[1], init="uniform", activation="relu"))
model.add(Dense(38, activation="relu", kernel_initializer="uniform"))
model.add(Dense(1, activation = 'sigmoid'))
print("[INFO] compiling model...")
adam = Adam(lr=0.01)
model.compile(loss="binary_crossentropy", optimizer=adam,
metrics=["accuracy"])
model.fit(X_train, Y_train, epochs=50, batch_size=128,
verbose=1)
When I do this, I get the following error:
KeyError: '[233946 164308 296688 166151 276165 88219 117980 163503 182033 164328\n 188083 30380 37984 245771 308534 6215 181186 307488 172375 60446\n 29397 166681 5587 243263 103579 262182 107823 234790 258973 116433\n 199283 86118 172148 257334 286452 248407 81280 ...] not in index'
I haven't been able to find a solution to this. Any help would be much appreciated.
I believe that the input is not a numpy array as described in this github issue on the keras page
Try fitting the model using this:
model.fit(np.array(X_train), np.array(Y_train), epochs=50, batch_size=128,
verbose=1)
Which will cast the arrays as numpy arrays when fitting the data.
I encountered some problems when I continued training my model and visualized the progress on tensorboard.
My question is how do I resume training from the same step without specifying any epoch manually? If possible, simply by loading the saved model, it somehow could read the global_step from the optimizer saved and continue training from there.
I have provided some codes below to reproduce similar errors.
import tensorflow as tf
from tensorflow.keras.callbacks import TensorBoard
from tensorflow.keras.models import load_model
mnist = tf.keras.datasets.mnist
(x_train, y_train),(x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(512, activation=tf.nn.relu),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=10, callbacks=[Tensorboard()])
model.save('./final_model.h5', include_optimizer=True)
del model
model = load_model('./final_model.h5')
model.fit(x_train, y_train, epochs=10, callbacks=[Tensorboard()])
You can run the tensorboard by using the command:
tensorboard --logdir ./logs
You can set the parameter initial_epoch in the function model.fit() to the number of the epoch you want your training to start from. Take into account that the model trains until the epoch of index epochs is reached (and not a number of iterations given by epochs).
In your example, if you want to train for 10 epochs more, it should be:
model.fit(x_train, y_train, initial_epoch=9, epochs=19, callbacks=[Tensorboard()])
It will allow you to visualise your plots on Tensorboard in a correct manner.
More extensive information about these parameters can be found in the docs.
Here is sample code in case someone needs it. It implements the idea proposed by Abhinav Anand:
mca = ModelCheckpoint(join(dir, 'model_{epoch:03d}.h5'),
monitor = 'loss',
save_best_only = False)
tb = TensorBoard(log_dir = join(dir, 'logs'),
write_graph = True,
write_images = True)
files = sorted(glob(join(fold_dir, 'model_???.h5')))
if files:
model_file = files[-1]
initial_epoch = int(model_file[-6:-3])
print('Resuming using saved model %s.' % model_file)
model = load_model(model_file)
else:
model = nn.model()
initial_epoch = 0
model.fit(x_train,
y_train,
epochs = 100,
initial_epoch = initial_epoch,
callbacks = [mca, tb])
Replace nn.model() with your own function for defining the model.
It's very simple. Create checkpoints while training the model and then use those checkpoints to resume training from where you left of.
import tensorflow as tf
from tensorflow.keras.callbacks import TensorBoard
from tensorflow.keras.callbacks import ModelCheckpoint
from tensorflow.keras.models import load_model
mnist = tf.keras.datasets.mnist
(x_train, y_train),(x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(512, activation=tf.nn.relu),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=10, callbacks=[Tensorboard()])
model.save('./final_model.h5', include_optimizer=True)
model = load_model('./final_model.h5')
callbacks = list()
tensorboard = Tensorboard()
callbacks.append(tensorboard)
file_path = "model-{epoch:02d}-{loss:.4f}.hdf5"
# now here you can create checkpoints and save according to your need
# here period is the no of epochs after which to save the model every time during training
# another option is save_weights_only, for your case it should be false
checkpoints = ModelCheckpoint(file_path, monitor='loss', verbose=1, period=1, save_weights_only=False)
callbacks.append(checkpoints)
model.fit(x_train, y_train, epochs=10, callbacks=callbacks)
After this just load the checkpoint from where you want to resume training again
model = load_model(checkpoint_of_choice)
model.fit(x_train, y_train, epochs=10, callbacks=callbacks)
And you are done.
Let me know if you have more questions about this.
What I did:
I implement the following model using of Keras:
train_X, test_X, train_Y, test_Y = train_test_split(X, Y, test_size=0.2, random_state=np.random.seed(7), shuffle=True)
train_X = np.reshape(train_X, (train_X.shape[0], 1, train_X.shape[1]))
test_X = np.reshape(test_X, (test_X.shape[0], 1, test_X.shape[1]))
inp = Input((train_X.shape[1], train_X.shape[2]))
lstm = LSTM(1, return_sequences=False)(inp)
output = Dense(train_Y.shape[1], activation='softmax')(lstm)
model = Model(inputs=inp, outputs=output)
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])
model.fit(train_X, train_Y, validation_split=.20, epochs=2, batch_size=50)
What I want:
I want to add SVM to the last layer in my model but i dont know how? Any idea?
This should work for adding svm as last layer.
inp = Input((train_X.shape[1], train_X.shape[2]))
lstm = LSTM(1, return_sequences=False)(inp)
output = Dense(train_Y.shape[1], activation='softmax', W_regularizer=l2(0.01)))(lstm)
model = Model(inputs=inp, outputs=output)
model.compile(loss='hinge', optimizer='adam', metrics=['accuracy'])
model.fit(train_X, train_Y, validation_split=.20, epochs=2, batch_size=50)
Here I have used hinge as loss considering binary categorised target. But if it is more than that, then you can consider using categorical_hinge
Change softmax to linear and add kernel_regularizer=l2(1e-4) instead W_regularizer=l2(0.01) using keras 2.2.4. Use loss = categorical_hinge.
I am trying to implement a stateful Bidirectional LSTM but I continue to get the following error:
" Cannot feed value of shape (72, 24, 8) for Tensor 'bidirectional_18_input:0', which has shape '(96, 24, 8) "
This is my code:
mdl = Sequential()
# create and fit the LSTM network
mdl.add(Bidirectional(LSTM(12 ,return_sequences=True, stateful=True),batch_input_shape=(96, lags, n_features )))
mdl.add(Bidirectional(LSTM(7,stateful=True)))
mdl.add(Dense(12))
mdl.add(Dense(12))
mdl.add(Dense(1))
mdl.compile(loss='mean_squared_error', optimizer='adam')
# early stopping implementation
filepath="weights.best.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=True, mode='min')
early_stop = EarlyStopping(monitor='val_loss', patience=200, mode='min')
callbacks_list = [checkpoint, early_stop]
# fit network
print(mdl.summary())
for i in range(100):
mdl.fit(X_train, y_train, epochs=1, batch_size=96, validation_data=(X_test, y_test), verbose=2, shuffle=False,
callbacks=callbacks_list)
mdl.reset_states()
I have the following architecture:
I can't understand what is the why it says that I am feeding a value of shape (72,24,8) even if I am using a batch size of 96