I'm working on a multiclass text classification problem.
After splitting the data to train and validation data frames, I've performed text augmentation to balance the data (only on the train data of course).
I've ended up with a balanced trained data and 44325 samples (of trained data).
Later on I've applied the "clean text" task for getting (i.e. stemming and stuff) on the trained data.
train['text'] = train['text'].apply(clean_text)
X_train = train.iloc[:, :-1]
y_train = train.iloc[:, -1:]
X_test = valid.iloc[:, :-1]
y_test = valid.iloc[:, -1:]
y_test = pd.DataFrame(y_test).reset_index(drop=True)
tokenizer = Tokenizer(num_words=vocab_size, oov_token='<OOV>')
tokenizer.fit_on_texts(X_train['text'])
train_seq = tokenizer.texts_to_sequences(X_train['text'])
train_padded = pad_sequences(train_seq, maxlen=max_length, padding=padding_type, truncating=trunc_type)
validation_seq = tokenizer.texts_to_sequences(X_test['text'])
validation_padded = pad_sequences(validation_seq, maxlen=max_length, padding=padding_type, truncating=trunc_type)
print('Shape of train data tensor:', train_padded.shape)
print('Shape of validation data tensor:', validation_padded.shape)
Output:
Shape of train data tensor: (44325, 200)
Shape of validation data tensor: (5466, 200)
Here's the encoding section:
encode = OneHotEncoder()
training_labels = encode.fit_transform(y_train)
validation_labels = encode.transform(y_test)
training_labels = training_labels.toarray()
validation_labels = validation_labels.toarray()
Model:
model = Sequential()
model.add(Embedding(vocab_size, embedding_dim, input_length=train_padded.shape[1]))
model.add(Conv1D(48, len(GROUPS), activation='relu', padding='valid'))
model.add(GlobalMaxPooling1D())
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dropout(0.5))
model.add(Dense(len(GROUPS), activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
epochs = 100
batch_size = 32
history = model.fit(train_padded, training_labels, shuffle=True ,
epochs=epochs, batch_size=batch_size,
validation_split=0.2,
validation_data=(validation_padded, validation_labels),
callbacks=[ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, min_lr=0.0001),
EarlyStopping(monitor='val_loss', mode='min', patience=2, verbose=1),
EarlyStopping(monitor='val_accuracy', mode='max', patience=5, verbose=1)])
Model output:
Epoch 1/100
1109/1109 [==============================] - 88s 79ms/step - loss: 1.2021 - accuracy: 0.5235 - val_loss: 0.8374 - val_accuracy: 0.7232
Epoch 2/100
1109/1109 [==============================] - 87s 79ms/step - loss: 0.9505 - accuracy: 0.6645 - val_loss: 0.7488 - val_accuracy: 0.7461
Epoch 3/100
1109/1109 [==============================] - 86s 77ms/step - loss: 0.8378 - accuracy: 0.7058 - val_loss: 0.6686 - val_accuracy: 0.7663
Epoch 4/100
1109/1109 [==============================] - 88s 79ms/step - loss: 0.7391 - accuracy: 0.7382 - val_loss: 0.6134 - val_accuracy: 0.7891
Epoch 5/100
1109/1109 [==============================] - 89s 80ms/step - loss: 0.6763 - accuracy: 0.7546 - val_loss: 0.5832 - val_accuracy: 0.7997
Epoch 6/100
1109/1109 [==============================] - 87s 79ms/step - loss: 0.6185 - accuracy: 0.7760 - val_loss: 0.5529 - val_accuracy: 0.8050
Epoch 7/100
1109/1109 [==============================] - 87s 79ms/step - loss: 0.5737 - accuracy: 0.7912 - val_loss: 0.5311 - val_accuracy: 0.8153
Epoch 8/100
1109/1109 [==============================] - 88s 80ms/step - loss: 0.5226 - accuracy: 0.8080 - val_loss: 0.5268 - val_accuracy: 0.8226
Epoch 9/100
1109/1109 [==============================] - 88s 79ms/step - loss: 0.4955 - accuracy: 0.8171 - val_loss: 0.5142 - val_accuracy: 0.8285
Epoch 10/100
1109/1109 [==============================] - 88s 80ms/step - loss: 0.4665 - accuracy: 0.8265 - val_loss: 0.5035 - val_accuracy: 0.8338
Epoch 11/100
1109/1109 [==============================] - 88s 79ms/step - loss: 0.4410 - accuracy: 0.8348 - val_loss: 0.5082 - val_accuracy: 0.8399
Epoch 12/100
1109/1109 [==============================] - 88s 80ms/step - loss: 0.4190 - accuracy: 0.8407 - val_loss: 0.5160 - val_accuracy: 0.8414
Epoch 00012: early stopping
... and to the last part I'm unsure of:
def evaluate_preds(y_true, y_preds):
"""
Performs evaluation comparison on y_true labels vs. y_pred labels
on a classification.
"""
accuracy = accuracy_score(y_true, y_preds)
precision = precision_score(y_true, y_preds, average='micro')
recall = recall_score(y_true, y_preds, average='micro')
f1 = f1_score(y_true, y_preds, average='micro')
metric_dict = {"accuracy": round(accuracy, 2),
"precision": round(precision, 2),
"recall": round(recall, 2),
"f1": round(f1, 2)}
print(f"Acc: {accuracy * 100:.2f}%")
print(f"Precision: {precision:.2f}")
print(f"Recall: {recall:.2f}")
print(f"F1 score: {f1:.2f}")
return metric_dict
predicted = model.predict(validation_padded)
evaluate_preds(np.argmax(validation_labels, axis=1), np.argmax(predicted, axis=1))
Output:
Acc: 40.16%
Precision: 0.40
Recall: 0.40
F1 score: 0.40
I can't understand what am I doing wrong.
How come accuracy of the last mentioned method is so low compare to val_accuracy?
Related
I am trying to build an image classification model using an Inception Network as the base. This is a simple binary classification model.
My images are available in many smaller directories within one big directory. Each of them has its own 'image id' and that is how they have been named. In addition to this, I have a few tsv files which contain these image ids and the respective labels ('Positive' or 'Negative').
When I train the model, I see that my accuracy fluctuates without much progress. I was wondering if there is anything wrong with the way that I have prepared my dataset. I have written a few functions for this purpose.
Before I get to these functions, given below is how I have defined my model,
base_model = InceptionV3(weights='imagenet', include_top=False)
x = base_model.output
x = GlobalAveragePooling2D(name='avg_pool')(x)
x = Dropout(0.4)(x)
predictions = Dense(2, activation='sigmoid')(x)
model = Model(inputs=base_model.input, outputs=predictions)
for layer in base_model.layers:
layer.trainable = False
model.compile(optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy'])
These are the functions that I have written in order to prepare my data,
def vectorize_img(img_path):
img = load_img(img_path, target_size=(224, 224)) #size is 224,224 by default
x = img_to_array(img) #change to np array
x = preprocess_input(x) #make input confirm to InceptionV3 input format
return x
def prepare_features(base_dir, limit):
features_dict = dict()
for dir1 in os.listdir(base_dir):
for dir2 in os.listdir(base_dir + dir1):
for file in os.listdir(base_dir + dir1 + '/' + dir2):
if len(features_dict) < limit:
try:
img_path = base_dir + dir1 + '/' + dir2 + '/' + file
x = vectorize_img(img_path)
name_id = file.split('.')[0] #take the file name and use as id in dict
features_dict[name_id] = x
except Exception as e:
print(e)
pass
return features_dict
def prepare_data(file_path, features_dict):
inputs = []
labels = []
df = pd.read_csv(file_path, sep='\t')
df = df[['image_id', 'label_text_image']]
df['class'] = df[['image_id', 'label_text_image']].apply(lambda x: 1 if x['label_text_image'] == 'Positive' else 0, axis = 1)
for index, row in df.iterrows():
try:
inputs.append(features_dict[row['image_id']])
labels.append(row['class'])
except:
pass
return np.asarray(inputs), tf.one_hot(np.asarray(labels), depth=2)
These functions are then called to prepare my dataset,
features_dict = prepare_features('/path/to/img/dir', 8000)
x_train, y_train = prepare_data('/path/to/train/tsv', features_dict)
x_dev, y_dev = prepare_data('/path/to/dev/tsv', features_dict)
x_test, y_test = prepare_data('/path/to/test/tsv', features_dict)
Finally, the model is trained,
EPOCHS = 50
BATCH_SIZE = 32
STEPS_PER_EPOCH = 1
history = model.fit(x=x_train, y=y_train, validation_data=(x_dev, y_dev), epochs=EPOCHS, steps_per_epoch=STEPS_PER_EPOCH, batch_size=BATCH_SIZE)
model.evaluate(x=x_test, y=y_test, batch_size=BATCH_SIZE)
Am I doing something wrong?
Here are the results that my model achieves,
Epoch 1/50
1/1 [==============================] - 158s 158s/step - loss: 0.8298 - accuracy: 0.5000 - val_loss: 0.7432 - val_accuracy: 0.5227
Epoch 2/50
1/1 [==============================] - 113s 113s/step - loss: 0.7775 - accuracy: 0.4688 - val_loss: 0.8225 - val_accuracy: 0.5153
Epoch 3/50
1/1 [==============================] - 113s 113s/step - loss: 0.7663 - accuracy: 0.5625 - val_loss: 0.8431 - val_accuracy: 0.5174
Epoch 4/50
1/1 [==============================] - 156s 156s/step - loss: 1.1292 - accuracy: 0.5312 - val_loss: 0.7763 - val_accuracy: 0.5227
Epoch 5/50
1/1 [==============================] - 114s 114s/step - loss: 0.7452 - accuracy: 0.5312 - val_loss: 0.7332 - val_accuracy: 0.5448
Epoch 6/50
1/1 [==============================] - 156s 156s/step - loss: 0.7884 - accuracy: 0.5312 - val_loss: 0.7072 - val_accuracy: 0.5606
Epoch 7/50
1/1 [==============================] - 114s 114s/step - loss: 0.7856 - accuracy: 0.5312 - val_loss: 0.7195 - val_accuracy: 0.5764
Epoch 8/50
1/1 [==============================] - 156s 156s/step - loss: 0.9203 - accuracy: 0.5312 - val_loss: 0.7348 - val_accuracy: 0.5616
Epoch 9/50
1/1 [==============================] - 156s 156s/step - loss: 0.8639 - accuracy: 0.4062 - val_loss: 0.7275 - val_accuracy: 0.5690
Epoch 10/50
1/1 [==============================] - 156s 156s/step - loss: 0.6170 - accuracy: 0.7188 - val_loss: 0.7125 - val_accuracy: 0.5880
Epoch 11/50
1/1 [==============================] - 156s 156s/step - loss: 0.5756 - accuracy: 0.7188 - val_loss: 0.6979 - val_accuracy: 0.6017
Epoch 12/50
1/1 [==============================] - 113s 113s/step - loss: 0.9976 - accuracy: 0.4375 - val_loss: 0.6834 - val_accuracy: 0.5933
Epoch 13/50
1/1 [==============================] - 156s 156s/step - loss: 0.7025 - accuracy: 0.5938 - val_loss: 0.6863 - val_accuracy: 0.5838
You mentioned that it is a binary classification hence labels are {0,1}. In this case your model output should either be
predictions = Dense(2, activation='softmax')(x)
with categorical labels [0,1] or [1,0]
or
predictions = Dense(1, activation='sigmoid')(x)
with binary label 1 or 0
but you are using output 2 with sigmoid i.e. predictions = Dense(2, activation='sigmoid')(x).
Dataset description:
(1) X_train: (6000,4) shape
(2) y_train: (6000,4) shape
(3) X_validation: (2000,4) shape
(4) y_validation: (2000,4) shape
(5) X_test: (2000,4) shape
(6) y_test: (2000,4) shape
Relationship between X and Y is shown here
For single label classification, the activation function of the last layer is Softmax and the loss function is categorical_crossentrop.
And I know the mathematical calculation method for the loss function.
And for multi-class multi-label classification problems, the activation function of the last layer is sigmoid, and the loss function is binary_crossentrop.
I want to know how the mathematical calculation method of the loss function works
It would be a great help to me if you let me know.
def MinMaxScaler(data):
numerator = data - np.min(data)
denominator = np.max(data) - np.min(data)
return numerator / (denominator + 1e-5)
kki = pd.read_csv(filename,names=['UE0','UE1','UE2','UE3','selected_UE0','selected_UE1','selected_UE2','selected_UE3'])
print(kki)
def LoadData(file):
xy = np.loadtxt(file, delimiter=',', dtype=np.float32)
print("Data set length:", len(xy))
tr_set_size = int(len(xy) * 0.6)
xy[:, 0:-number_of_UEs] = MinMaxScaler(xy[:, 0:-number_of_UEs]) #number_of_UES : 4
X_train = xy[:tr_set_size, 0: -number_of_UEs] #6000 row
y_train = xy[:tr_set_size, number_of_UEs:number_of_UEs*2]
X_valid = xy[tr_set_size:int((tr_set_size/3) + tr_set_size), 0:-number_of_UEs]
y_valid = xy[tr_set_size:int((tr_set_size/3) + tr_set_size), number_of_UEs:number_of_UEs *2]
X_test = xy[int((tr_set_size/3) + tr_set_size):, 0:-number_of_UEs]
y_test = xy[int((tr_set_size/3) + tr_set_size):, number_of_UEs:number_of_UEs*2]
print("Training X shape:", X_train.shape)
print("Training Y shape:", y_train.shape)
print("validation x shape:", X_valid.shape)
print("validation y shape:", y_valid.shape)
print("Test X shape:", X_test.shape)
print("Test Y shape:", y_test.shape)
return X_train, y_train, X_valid, y_valid, X_test, y_test, tr_set_size
X_train, y_train, X_valid, y_valid, X_test, y_test, tr_set_size = LoadData(filename)
model = Sequential()
model.add(Dense(64,activation='relu', input_shape=(X_train.shape[1],)))
model.add(Dense(46, activation='relu'))
model.add(Dense(24, activation='relu'))
model.add(Dense(12, activation='relu'))
model.add(Dense(4, activation= 'sigmoid'))
model.compile( loss ='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
hist = model.fit(X_train, y_train, epochs=5, batch_size=1, verbose= 1, validation_data=(X_valid, y_valid), callbacks= es)
This is a learning process, and even if epochs are repeated,
Accuracy does not improve.
Epoch 1/10
6000/6000 [==============================] - 14s 2ms/step - loss: 0.2999 - accuracy: 0.5345 - val_loss: 0.1691 - val_accuracy: 0.5465
Epoch 2/10
6000/6000 [==============================] - 14s 2ms/step - loss: 0.1554 - accuracy: 0.4883 - val_loss: 0.1228 - val_accuracy: 0.4710
Epoch 3/10
6000/6000 [==============================] - 14s 2ms/step - loss: 0.1259 - accuracy: 0.4710 - val_loss: 0.0893 - val_accuracy: 0.4910
Epoch 4/10
6000/6000 [==============================] - 13s 2ms/step - loss: 0.1094 - accuracy: 0.4990 - val_loss: 0.0918 - val_accuracy: 0.5540
Epoch 5/10
6000/6000 [==============================] - 13s 2ms/step - loss: 0.0967 - accuracy: 0.5223 - val_loss: 0.0671 - val_accuracy: 0.5405
Epoch 6/10
6000/6000 [==============================] - 13s 2ms/step - loss: 0.0910 - accuracy: 0.5198 - val_loss: 0.0836 - val_accuracy: 0.5380
Epoch 7/10
6000/6000 [==============================] - 13s 2ms/step - loss: 0.0870 - accuracy: 0.5348 - val_loss: 0.0853 - val_accuracy: 0.5775
Epoch 8/10
6000/6000 [==============================] - 13s 2ms/step - loss: 0.0859 - accuracy: 0.5518 - val_loss: 0.0515 - val_accuracy: 0.6520
Epoch 9/10
6000/6000 [==============================] - 13s 2ms/step - loss: 0.0792 - accuracy: 0.5508 - val_loss: 0.0629 - val_accuracy: 0.4350
Epoch 10/10
6000/6000 [==============================] - 13s 2ms/step - loss: 0.0793 - accuracy: 0.5638 - val_loss: 0.0632 - val_accuracy: 0.6270
Mistake 1 -
The shape of y_train, y_validation and y_test should be (6000,), (2000,) and (2000,) respectively.
Mistake 2 -
For multi-class classification, the loss should be categorical_crossentropy and activation should be a softmax. So, change these two lines, like this:
model.add(Dense(4, activation= 'softmax'))
model.compile(loss ='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
Suggestion -
Why are you splitting data by yourself? Use scikit-learn train_test_split. This code will give you proper splits:
from sklearn.model_selection import train_test_split
x, x_test, y, y_test = train_test_split(xtrain, labels, test_size=0.2, train_size=0.8)
x_train, x_validation, y_train, y_validation = train_test_split(x, y, test_size = 0.25, train_size =0.75)
I'm trying to figure out why model's Loss value is always 0.0, so the accuracy seems to be constant as well (which is incorrect in my case, afaik).
Code snippet:
model = Sequential()
model.add(Embedding(vocab_size, glove_vectors.vector_size, weights=[embedding_matrix], input_length=X.shape[1]))
model.add(Dropout(0.5))
model.add(LSTM(100, dropout=0.2, recurrent_dropout=0.2))
model.add(Dense(1, activation="sigmoid"))
model.compile(loss='categorical_crossentropy', optimizer="adam", metrics=["accuracy"])
model.summary()
EPOCHS = 20
train_data, test_data, train_labels, test_labels = train_test_split(X, Y, test_size=0.20, random_state = 42)
print(train_data.shape, train_labels.shape)
print(test_data.shape, test_labels.shape)
val_data = (test_data, test_labels)
history = model.fit(train_data, train_labels, validation_data=val_data, epochs=EPOCHS)
score = model.evaluate(test_data, test_labels)
Output:
Epoch 1/20
25/25 [==============================] - 4s 69ms/step - loss: 0.0000e+00 - accuracy: 0.5241 - val_loss: 0.0000e+00 - val_accuracy: 0.4650
Epoch 2/20
25/25 [==============================] - 1s 55ms/step - loss: 0.0000e+00 - accuracy: 0.4927 - val_loss: 0.0000e+00 - val_accuracy: 0.4650
Epoch 3/20
25/25 [==============================] - 1s 55ms/step - loss: 0.0000e+00 - accuracy: 0.5110 - val_loss: 0.0000e+00 - val_accuracy: 0.4650
Epoch 4/20
25/25 [==============================] - 1s 56ms/step - loss: 0.0000e+00 - accuracy: 0.5074 - val_loss: 0.0000e+00 - val_accuracy: 0.4650
Epoch 5/20
25/25 [==============================] - 1s 55ms/step - loss: 0.0000e+00 - accuracy: 0.5363 - val_loss: 0.0000e+00 - val_accuracy: 0.4650
Epoch 6/20
25/25 [==============================] - 1s 53ms/step - loss: 0.0000e+00 - accuracy: 0.5042 - val_loss: 0.0000e+00 - val_accuracy: 0.4650
Epoch 7/20
25/25 [==============================] - 1s 54ms/step - loss: 0.0000e+00 - accuracy: 0.4904 - val_loss: 0.0000e+00 - val_accuracy: 0.4650
In binary classification there will be 1 node in the output layer even though we will be predicting between two classes. In order to get the output in a probability format between 0 and 1 we will use the sigmoid function.
Hence binary_crossentropy is the correct loss function in your case
model.compile(loss='binary_crossentropy', optimizer="adam", metrics=["accuracy"])
I have a time series imbalanced dataset on which I have to perform binary classification. I cannot split training and test sets randomly or even perform stratify on them. The issue is that while training the model validation accuracy is always 1. I realize this has somethings to do with the train-test split but I may be wrong.
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.8, random_state=None, shuffle=False)
from collections import Counter
print(Counter(y))
print(Counter(y_train))
print(Counter(y_test))
Counter({0.0: 55534, 1.0: 10000})
Counter({0.0: 9995, 1.0: 3111})
Counter({0.0: 45539, 1.0: 6889})
model = Sequential()
#First Hidden Layer
model.add(Dense(128, activation='relu', kernel_initializer='random_normal', input_dim=19))
#model.add(Dropout(0.3))
#Second Hidden Layer
model.add(Dense(64, activation='relu', kernel_initializer='random_normal'))
#Output Layer
model.add(Dense(1, activation='sigmoid', kernel_initializer='random_normal'))
history = model.fit(X_train,y_train, batch_size=128, validation_split=0.1, epochs=50)
Train on 11795 samples, validate on 1311 samples
Epoch 1/50
11795/11795 [==============================] - 0s 34us/step - loss: 1.1359 - accuracy: 0.8719 - val_loss: 4.2016e-18 - val_accuracy: 1.0000
Epoch 2/50
11795/11795 [==============================] - 0s 12us/step - loss: 0.1247 - accuracy: 0.9442 - val_loss: 1.0255e-19 - val_accuracy: 1.0000
Epoch 3/50
11795/11795 [==============================] - 0s 13us/step - loss: 0.1177 - accuracy: 0.9462 - val_loss: 3.2516e-23 - val_accuracy: 1.0000
Epoch 4/50
11795/11795 [==============================] - 0s 12us/step - loss: 0.1103 - accuracy: 0.9519 - val_loss: 1.1607e-23 - val_accuracy: 1.0000
Epoch 5/50
11795/11795 [==============================] - 0s 13us/step - loss: 0.0805 - accuracy: 0.9739 - val_loss: 6.2345e-26 - val_accuracy: 1.0000
Appreciate any help on this problem.Thanks
I'm working on a multi-class classification problem with Keras 2.1.3 and a Tensorflow backend. I have two numpy arrays, x and y and I'm using tf.data.Dataset like this:
dataset = tf.data.Dataset.from_tensor_slices(({"sequence": x}, y))
dataset = dataset.apply(tf.contrib.data.batch_and_drop_remainder(self.batch_size))
dataset = dataset.repeat()
xt, yt = dataset.make_one_shot_iterator().get_next()
Then I make my Keras model (omitted for brevity), compile, and fit:
model.compile(
loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'],
)
model.fit(xt, yt, steps_per_epoch=100, epochs=10)
This works perfectly well. But when I add in callbacks, I run into issues. Specifically, if I do this:
callbacks = [
tf.keras.callbacks.ModelCheckpoint("model_{epoch:04d}_{val_acc:.4f}.h5",
monitor='val_acc',
verbose=1,
save_best_only=True,
mode='max'),
tf.keras.callbacks.TensorBoard(os.path.join('.', 'logs')),
tf.keras.callbacks.EarlyStopping(monitor='val_acc', patience=5, min_delta=0, mode='max')
]
model.fit(xt, yt, steps_per_epoch=10, epochs=100, callbacks=callbacks)
I get:
KeyError: 'val_acc'
Also, if I include validation_split=0.1 in my model.fit(...) call, I'm told:
ValueError: If your data is in the form of symbolic tensors, you cannot use validation_split.`
What is the normal way to use callbacks and validation splits with tf.data.Dataset (tensors)?
Thanks!
Using the tensorflow keras API, you can provide a Dataset for training and another for validation.
First some imports
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Dense
import numpy as np
define the function which will split the numpy arrays into training/val
def split(x, y, val_size=50):
idx = np.random.choice(x.shape[0], size=val_size, replace=False)
not_idx = list(set(range(x.shape[0])).difference(set(idx)))
x_val = x[idx]
y_val = y[idx]
x_train = x[not_idx]
y_train = y[not_idx]
return x_train, y_train, x_val, y_val
define numpy arrays and the train/val tensorflow Datasets
x = np.random.randn(150, 9)
y = np.random.randint(0, 10, 150)
x_train, y_train, x_val, y_val = split(x, y)
train_dataset = tf.data.Dataset.from_tensor_slices((x_train, tf.one_hot(y_train, depth=10)))
train_dataset = train_dataset.batch(32).repeat()
val_dataset = tf.data.Dataset.from_tensor_slices((x_val, tf.one_hot(y_val, depth=10)))
val_dataset = val_dataset.batch(32).repeat()
Make the model (notice we are using the tensorflow keras API)
model = keras.models.Sequential([Dense(64, input_shape=(9,), activation='relu'),
Dense(64, activation='relu'),
Dense(10, activation='softmax')
])
model.compile(optimizer='Adam', loss='categorical_crossentropy',
metrics=['accuracy'])
model.summary()
model.fit(train_dataset,
epochs=10,
steps_per_epoch=int(100/32)+1,
validation_data=val_dataset,
validation_steps=2)
and the model trains, kind of (output):
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense (Dense) (None, 64) 640
_________________________________________________________________
dense_1 (Dense) (None, 64) 4160
_________________________________________________________________
dense_2 (Dense) (None, 10) 650
=================================================================
Total params: 5,450
Trainable params: 5,450
Non-trainable params: 0
_________________________________________________________________
Epoch 1/10
4/4 [==============================] - 0s 69ms/step - loss: 2.3170 - acc: 0.1328 - val_loss: 2.3877 - val_acc: 0.0712
Epoch 2/10
4/4 [==============================] - 0s 2ms/step - loss: 2.2628 - acc: 0.2500 - val_loss: 2.3850 - val_acc: 0.0712
Epoch 3/10
4/4 [==============================] - 0s 2ms/step - loss: 2.2169 - acc: 0.2656 - val_loss: 2.3838 - val_acc: 0.0712
Epoch 4/10
4/4 [==============================] - 0s 2ms/step - loss: 2.1743 - acc: 0.3359 - val_loss: 2.3830 - val_acc: 0.0590
Epoch 5/10
4/4 [==============================] - 0s 2ms/step - loss: 2.1343 - acc: 0.3594 - val_loss: 2.3838 - val_acc: 0.0590
Epoch 6/10
4/4 [==============================] - 0s 2ms/step - loss: 2.0959 - acc: 0.3516 - val_loss: 2.3858 - val_acc: 0.0590
Epoch 7/10
4/4 [==============================] - 0s 4ms/step - loss: 2.0583 - acc: 0.3750 - val_loss: 2.3887 - val_acc: 0.0590
Epoch 8/10
4/4 [==============================] - 0s 2ms/step - loss: 2.0223 - acc: 0.4453 - val_loss: 2.3918 - val_acc: 0.0747
Epoch 9/10
4/4 [==============================] - 0s 2ms/step - loss: 1.9870 - acc: 0.4609 - val_loss: 2.3954 - val_acc: 0.1059
Epoch 10/10
4/4 [==============================] - 0s 2ms/step - loss: 1.9523 - acc: 0.4609 - val_loss: 2.3995 - val_acc: 0.1059
Callbacks
Adding callbacks also works,
callbacks = [ tf.keras.callbacks.ModelCheckpoint("model_{epoch:04d}_{val_acc:.4f}.h5",
monitor='val_acc',
verbose=1,
save_best_only=True,
mode='max'),
tf.keras.callbacks.TensorBoard('./logs'),
tf.keras.callbacks.EarlyStopping(monitor='val_acc', patience=5, min_delta=0, mode='max')
]
model.fit(train_dataset, epochs=10, steps_per_epoch=int(100/32)+1, validation_data=val_dataset,
validation_steps=2, callbacks=callbacks)
Output:
Epoch 1/10
4/4
[==============================] - 0s 59ms/step - loss: 2.3274 - acc: 0.1094 - val_loss: 2.3143 - val_acc: 0.0833
Epoch 00001: val_acc improved from -inf to 0.08333, saving model to model_0001_0.0833.h5
Epoch 2/10
4/4 [==============================] - 0s 2ms/step - loss: 2.2655 - acc: 0.1094 - val_loss: 2.3204 - val_acc: 0.1389
Epoch 00002: val_acc improved from 0.08333 to 0.13889, saving model to model_0002_0.1389.h5
Epoch 3/10
4/4 [==============================] - 0s 5ms/step - loss: 2.2122 - acc: 0.1250 - val_loss: 2.3289 - val_acc: 0.1111
Epoch 00003: val_acc did not improve from 0.13889
Epoch 4/10
4/4 [==============================] - 0s 2ms/step - loss: 2.1644 - acc: 0.1953 - val_loss: 2.3388 - val_acc: 0.0556
Epoch 00004: val_acc did not improve from 0.13889
Epoch 5/10
4/4 [==============================] - 0s 2ms/step - loss: 2.1211 - acc: 0.2734 - val_loss: 2.3495 - val_acc: 0.0556
Epoch 00005: val_acc did not improve from 0.13889
Epoch 6/10
4/4 [==============================] - 0s 4ms/step - loss: 2.0808 - acc: 0.2969 - val_loss: 2.3616 - val_acc: 0.0556
Epoch 00006: val_acc did not improve from 0.13889
Epoch 7/10
4/4 [==============================] - 0s 2ms/step - loss: 2.0431 - acc: 0.2969 - val_loss: 2.3749 - val_acc: 0.0712
Epoch 00007: val_acc did not improve from 0.13889
I think your problem is type of your datasets, where xt and yt is not a list or array, so it can not to slice or split.
Let assume x is array with shape (1000,2) fill with random number using numpy and y is class with 10 classes.
# Define x and y
import numpy as np
np.random.seed(2018)
n_cat = 10
x = np.random.rand(1000,2) # Generate random 2 variables and 1000 rows
y = [np.random.randint(1,n_cat) for i in range(len(x))] # Make 10 class
Then you assemble the model and compile
model.compile(
loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'],
)
Create callbacks for checkpoint model
callbacks = [
tf.keras.callbacks.ModelCheckpoint("model_{epoch:04d}_{val_acc:.4f}.h5",
monitor='val_acc',
verbose=1,
save_best_only=True,
mode='max'),
tf.keras.callbacks.TensorBoard(os.path.join('.', 'logs')),
tf.keras.callbacks.EarlyStopping(monitor='val_acc', patience=5, min_delta=0, mode='max')
]
fit the model
model.fit(x, y, batch_size=32, epochs=100, callbacks=callbacks, validation_split=0.1)
it's works for me.