Related
I'm working on a multiclass text classification problem.
After splitting the data to train and validation data frames, I've performed text augmentation to balance the data (only on the train data of course).
I've ended up with a balanced trained data and 44325 samples (of trained data).
Later on I've applied the "clean text" task for getting (i.e. stemming and stuff) on the trained data.
train['text'] = train['text'].apply(clean_text)
X_train = train.iloc[:, :-1]
y_train = train.iloc[:, -1:]
X_test = valid.iloc[:, :-1]
y_test = valid.iloc[:, -1:]
y_test = pd.DataFrame(y_test).reset_index(drop=True)
tokenizer = Tokenizer(num_words=vocab_size, oov_token='<OOV>')
tokenizer.fit_on_texts(X_train['text'])
train_seq = tokenizer.texts_to_sequences(X_train['text'])
train_padded = pad_sequences(train_seq, maxlen=max_length, padding=padding_type, truncating=trunc_type)
validation_seq = tokenizer.texts_to_sequences(X_test['text'])
validation_padded = pad_sequences(validation_seq, maxlen=max_length, padding=padding_type, truncating=trunc_type)
print('Shape of train data tensor:', train_padded.shape)
print('Shape of validation data tensor:', validation_padded.shape)
Output:
Shape of train data tensor: (44325, 200)
Shape of validation data tensor: (5466, 200)
Here's the encoding section:
encode = OneHotEncoder()
training_labels = encode.fit_transform(y_train)
validation_labels = encode.transform(y_test)
training_labels = training_labels.toarray()
validation_labels = validation_labels.toarray()
Model:
model = Sequential()
model.add(Embedding(vocab_size, embedding_dim, input_length=train_padded.shape[1]))
model.add(Conv1D(48, len(GROUPS), activation='relu', padding='valid'))
model.add(GlobalMaxPooling1D())
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dropout(0.5))
model.add(Dense(len(GROUPS), activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
epochs = 100
batch_size = 32
history = model.fit(train_padded, training_labels, shuffle=True ,
epochs=epochs, batch_size=batch_size,
validation_split=0.2,
validation_data=(validation_padded, validation_labels),
callbacks=[ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, min_lr=0.0001),
EarlyStopping(monitor='val_loss', mode='min', patience=2, verbose=1),
EarlyStopping(monitor='val_accuracy', mode='max', patience=5, verbose=1)])
Model output:
Epoch 1/100
1109/1109 [==============================] - 88s 79ms/step - loss: 1.2021 - accuracy: 0.5235 - val_loss: 0.8374 - val_accuracy: 0.7232
Epoch 2/100
1109/1109 [==============================] - 87s 79ms/step - loss: 0.9505 - accuracy: 0.6645 - val_loss: 0.7488 - val_accuracy: 0.7461
Epoch 3/100
1109/1109 [==============================] - 86s 77ms/step - loss: 0.8378 - accuracy: 0.7058 - val_loss: 0.6686 - val_accuracy: 0.7663
Epoch 4/100
1109/1109 [==============================] - 88s 79ms/step - loss: 0.7391 - accuracy: 0.7382 - val_loss: 0.6134 - val_accuracy: 0.7891
Epoch 5/100
1109/1109 [==============================] - 89s 80ms/step - loss: 0.6763 - accuracy: 0.7546 - val_loss: 0.5832 - val_accuracy: 0.7997
Epoch 6/100
1109/1109 [==============================] - 87s 79ms/step - loss: 0.6185 - accuracy: 0.7760 - val_loss: 0.5529 - val_accuracy: 0.8050
Epoch 7/100
1109/1109 [==============================] - 87s 79ms/step - loss: 0.5737 - accuracy: 0.7912 - val_loss: 0.5311 - val_accuracy: 0.8153
Epoch 8/100
1109/1109 [==============================] - 88s 80ms/step - loss: 0.5226 - accuracy: 0.8080 - val_loss: 0.5268 - val_accuracy: 0.8226
Epoch 9/100
1109/1109 [==============================] - 88s 79ms/step - loss: 0.4955 - accuracy: 0.8171 - val_loss: 0.5142 - val_accuracy: 0.8285
Epoch 10/100
1109/1109 [==============================] - 88s 80ms/step - loss: 0.4665 - accuracy: 0.8265 - val_loss: 0.5035 - val_accuracy: 0.8338
Epoch 11/100
1109/1109 [==============================] - 88s 79ms/step - loss: 0.4410 - accuracy: 0.8348 - val_loss: 0.5082 - val_accuracy: 0.8399
Epoch 12/100
1109/1109 [==============================] - 88s 80ms/step - loss: 0.4190 - accuracy: 0.8407 - val_loss: 0.5160 - val_accuracy: 0.8414
Epoch 00012: early stopping
... and to the last part I'm unsure of:
def evaluate_preds(y_true, y_preds):
"""
Performs evaluation comparison on y_true labels vs. y_pred labels
on a classification.
"""
accuracy = accuracy_score(y_true, y_preds)
precision = precision_score(y_true, y_preds, average='micro')
recall = recall_score(y_true, y_preds, average='micro')
f1 = f1_score(y_true, y_preds, average='micro')
metric_dict = {"accuracy": round(accuracy, 2),
"precision": round(precision, 2),
"recall": round(recall, 2),
"f1": round(f1, 2)}
print(f"Acc: {accuracy * 100:.2f}%")
print(f"Precision: {precision:.2f}")
print(f"Recall: {recall:.2f}")
print(f"F1 score: {f1:.2f}")
return metric_dict
predicted = model.predict(validation_padded)
evaluate_preds(np.argmax(validation_labels, axis=1), np.argmax(predicted, axis=1))
Output:
Acc: 40.16%
Precision: 0.40
Recall: 0.40
F1 score: 0.40
I can't understand what am I doing wrong.
How come accuracy of the last mentioned method is so low compare to val_accuracy?
Dataset description:
(1) X_train: (6000,4) shape
(2) y_train: (6000,4) shape
(3) X_validation: (2000,4) shape
(4) y_validation: (2000,4) shape
(5) X_test: (2000,4) shape
(6) y_test: (2000,4) shape
Relationship between X and Y is shown here
For single label classification, the activation function of the last layer is Softmax and the loss function is categorical_crossentrop.
And I know the mathematical calculation method for the loss function.
And for multi-class multi-label classification problems, the activation function of the last layer is sigmoid, and the loss function is binary_crossentrop.
I want to know how the mathematical calculation method of the loss function works
It would be a great help to me if you let me know.
def MinMaxScaler(data):
numerator = data - np.min(data)
denominator = np.max(data) - np.min(data)
return numerator / (denominator + 1e-5)
kki = pd.read_csv(filename,names=['UE0','UE1','UE2','UE3','selected_UE0','selected_UE1','selected_UE2','selected_UE3'])
print(kki)
def LoadData(file):
xy = np.loadtxt(file, delimiter=',', dtype=np.float32)
print("Data set length:", len(xy))
tr_set_size = int(len(xy) * 0.6)
xy[:, 0:-number_of_UEs] = MinMaxScaler(xy[:, 0:-number_of_UEs]) #number_of_UES : 4
X_train = xy[:tr_set_size, 0: -number_of_UEs] #6000 row
y_train = xy[:tr_set_size, number_of_UEs:number_of_UEs*2]
X_valid = xy[tr_set_size:int((tr_set_size/3) + tr_set_size), 0:-number_of_UEs]
y_valid = xy[tr_set_size:int((tr_set_size/3) + tr_set_size), number_of_UEs:number_of_UEs *2]
X_test = xy[int((tr_set_size/3) + tr_set_size):, 0:-number_of_UEs]
y_test = xy[int((tr_set_size/3) + tr_set_size):, number_of_UEs:number_of_UEs*2]
print("Training X shape:", X_train.shape)
print("Training Y shape:", y_train.shape)
print("validation x shape:", X_valid.shape)
print("validation y shape:", y_valid.shape)
print("Test X shape:", X_test.shape)
print("Test Y shape:", y_test.shape)
return X_train, y_train, X_valid, y_valid, X_test, y_test, tr_set_size
X_train, y_train, X_valid, y_valid, X_test, y_test, tr_set_size = LoadData(filename)
model = Sequential()
model.add(Dense(64,activation='relu', input_shape=(X_train.shape[1],)))
model.add(Dense(46, activation='relu'))
model.add(Dense(24, activation='relu'))
model.add(Dense(12, activation='relu'))
model.add(Dense(4, activation= 'sigmoid'))
model.compile( loss ='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
hist = model.fit(X_train, y_train, epochs=5, batch_size=1, verbose= 1, validation_data=(X_valid, y_valid), callbacks= es)
This is a learning process, and even if epochs are repeated,
Accuracy does not improve.
Epoch 1/10
6000/6000 [==============================] - 14s 2ms/step - loss: 0.2999 - accuracy: 0.5345 - val_loss: 0.1691 - val_accuracy: 0.5465
Epoch 2/10
6000/6000 [==============================] - 14s 2ms/step - loss: 0.1554 - accuracy: 0.4883 - val_loss: 0.1228 - val_accuracy: 0.4710
Epoch 3/10
6000/6000 [==============================] - 14s 2ms/step - loss: 0.1259 - accuracy: 0.4710 - val_loss: 0.0893 - val_accuracy: 0.4910
Epoch 4/10
6000/6000 [==============================] - 13s 2ms/step - loss: 0.1094 - accuracy: 0.4990 - val_loss: 0.0918 - val_accuracy: 0.5540
Epoch 5/10
6000/6000 [==============================] - 13s 2ms/step - loss: 0.0967 - accuracy: 0.5223 - val_loss: 0.0671 - val_accuracy: 0.5405
Epoch 6/10
6000/6000 [==============================] - 13s 2ms/step - loss: 0.0910 - accuracy: 0.5198 - val_loss: 0.0836 - val_accuracy: 0.5380
Epoch 7/10
6000/6000 [==============================] - 13s 2ms/step - loss: 0.0870 - accuracy: 0.5348 - val_loss: 0.0853 - val_accuracy: 0.5775
Epoch 8/10
6000/6000 [==============================] - 13s 2ms/step - loss: 0.0859 - accuracy: 0.5518 - val_loss: 0.0515 - val_accuracy: 0.6520
Epoch 9/10
6000/6000 [==============================] - 13s 2ms/step - loss: 0.0792 - accuracy: 0.5508 - val_loss: 0.0629 - val_accuracy: 0.4350
Epoch 10/10
6000/6000 [==============================] - 13s 2ms/step - loss: 0.0793 - accuracy: 0.5638 - val_loss: 0.0632 - val_accuracy: 0.6270
Mistake 1 -
The shape of y_train, y_validation and y_test should be (6000,), (2000,) and (2000,) respectively.
Mistake 2 -
For multi-class classification, the loss should be categorical_crossentropy and activation should be a softmax. So, change these two lines, like this:
model.add(Dense(4, activation= 'softmax'))
model.compile(loss ='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
Suggestion -
Why are you splitting data by yourself? Use scikit-learn train_test_split. This code will give you proper splits:
from sklearn.model_selection import train_test_split
x, x_test, y, y_test = train_test_split(xtrain, labels, test_size=0.2, train_size=0.8)
x_train, x_validation, y_train, y_validation = train_test_split(x, y, test_size = 0.25, train_size =0.75)
I'm trying to figure out why model's Loss value is always 0.0, so the accuracy seems to be constant as well (which is incorrect in my case, afaik).
Code snippet:
model = Sequential()
model.add(Embedding(vocab_size, glove_vectors.vector_size, weights=[embedding_matrix], input_length=X.shape[1]))
model.add(Dropout(0.5))
model.add(LSTM(100, dropout=0.2, recurrent_dropout=0.2))
model.add(Dense(1, activation="sigmoid"))
model.compile(loss='categorical_crossentropy', optimizer="adam", metrics=["accuracy"])
model.summary()
EPOCHS = 20
train_data, test_data, train_labels, test_labels = train_test_split(X, Y, test_size=0.20, random_state = 42)
print(train_data.shape, train_labels.shape)
print(test_data.shape, test_labels.shape)
val_data = (test_data, test_labels)
history = model.fit(train_data, train_labels, validation_data=val_data, epochs=EPOCHS)
score = model.evaluate(test_data, test_labels)
Output:
Epoch 1/20
25/25 [==============================] - 4s 69ms/step - loss: 0.0000e+00 - accuracy: 0.5241 - val_loss: 0.0000e+00 - val_accuracy: 0.4650
Epoch 2/20
25/25 [==============================] - 1s 55ms/step - loss: 0.0000e+00 - accuracy: 0.4927 - val_loss: 0.0000e+00 - val_accuracy: 0.4650
Epoch 3/20
25/25 [==============================] - 1s 55ms/step - loss: 0.0000e+00 - accuracy: 0.5110 - val_loss: 0.0000e+00 - val_accuracy: 0.4650
Epoch 4/20
25/25 [==============================] - 1s 56ms/step - loss: 0.0000e+00 - accuracy: 0.5074 - val_loss: 0.0000e+00 - val_accuracy: 0.4650
Epoch 5/20
25/25 [==============================] - 1s 55ms/step - loss: 0.0000e+00 - accuracy: 0.5363 - val_loss: 0.0000e+00 - val_accuracy: 0.4650
Epoch 6/20
25/25 [==============================] - 1s 53ms/step - loss: 0.0000e+00 - accuracy: 0.5042 - val_loss: 0.0000e+00 - val_accuracy: 0.4650
Epoch 7/20
25/25 [==============================] - 1s 54ms/step - loss: 0.0000e+00 - accuracy: 0.4904 - val_loss: 0.0000e+00 - val_accuracy: 0.4650
In binary classification there will be 1 node in the output layer even though we will be predicting between two classes. In order to get the output in a probability format between 0 and 1 we will use the sigmoid function.
Hence binary_crossentropy is the correct loss function in your case
model.compile(loss='binary_crossentropy', optimizer="adam", metrics=["accuracy"])
I'm working on a multi-class classification problem with Keras 2.1.3 and a Tensorflow backend. I have two numpy arrays, x and y and I'm using tf.data.Dataset like this:
dataset = tf.data.Dataset.from_tensor_slices(({"sequence": x}, y))
dataset = dataset.apply(tf.contrib.data.batch_and_drop_remainder(self.batch_size))
dataset = dataset.repeat()
xt, yt = dataset.make_one_shot_iterator().get_next()
Then I make my Keras model (omitted for brevity), compile, and fit:
model.compile(
loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'],
)
model.fit(xt, yt, steps_per_epoch=100, epochs=10)
This works perfectly well. But when I add in callbacks, I run into issues. Specifically, if I do this:
callbacks = [
tf.keras.callbacks.ModelCheckpoint("model_{epoch:04d}_{val_acc:.4f}.h5",
monitor='val_acc',
verbose=1,
save_best_only=True,
mode='max'),
tf.keras.callbacks.TensorBoard(os.path.join('.', 'logs')),
tf.keras.callbacks.EarlyStopping(monitor='val_acc', patience=5, min_delta=0, mode='max')
]
model.fit(xt, yt, steps_per_epoch=10, epochs=100, callbacks=callbacks)
I get:
KeyError: 'val_acc'
Also, if I include validation_split=0.1 in my model.fit(...) call, I'm told:
ValueError: If your data is in the form of symbolic tensors, you cannot use validation_split.`
What is the normal way to use callbacks and validation splits with tf.data.Dataset (tensors)?
Thanks!
Using the tensorflow keras API, you can provide a Dataset for training and another for validation.
First some imports
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Dense
import numpy as np
define the function which will split the numpy arrays into training/val
def split(x, y, val_size=50):
idx = np.random.choice(x.shape[0], size=val_size, replace=False)
not_idx = list(set(range(x.shape[0])).difference(set(idx)))
x_val = x[idx]
y_val = y[idx]
x_train = x[not_idx]
y_train = y[not_idx]
return x_train, y_train, x_val, y_val
define numpy arrays and the train/val tensorflow Datasets
x = np.random.randn(150, 9)
y = np.random.randint(0, 10, 150)
x_train, y_train, x_val, y_val = split(x, y)
train_dataset = tf.data.Dataset.from_tensor_slices((x_train, tf.one_hot(y_train, depth=10)))
train_dataset = train_dataset.batch(32).repeat()
val_dataset = tf.data.Dataset.from_tensor_slices((x_val, tf.one_hot(y_val, depth=10)))
val_dataset = val_dataset.batch(32).repeat()
Make the model (notice we are using the tensorflow keras API)
model = keras.models.Sequential([Dense(64, input_shape=(9,), activation='relu'),
Dense(64, activation='relu'),
Dense(10, activation='softmax')
])
model.compile(optimizer='Adam', loss='categorical_crossentropy',
metrics=['accuracy'])
model.summary()
model.fit(train_dataset,
epochs=10,
steps_per_epoch=int(100/32)+1,
validation_data=val_dataset,
validation_steps=2)
and the model trains, kind of (output):
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense (Dense) (None, 64) 640
_________________________________________________________________
dense_1 (Dense) (None, 64) 4160
_________________________________________________________________
dense_2 (Dense) (None, 10) 650
=================================================================
Total params: 5,450
Trainable params: 5,450
Non-trainable params: 0
_________________________________________________________________
Epoch 1/10
4/4 [==============================] - 0s 69ms/step - loss: 2.3170 - acc: 0.1328 - val_loss: 2.3877 - val_acc: 0.0712
Epoch 2/10
4/4 [==============================] - 0s 2ms/step - loss: 2.2628 - acc: 0.2500 - val_loss: 2.3850 - val_acc: 0.0712
Epoch 3/10
4/4 [==============================] - 0s 2ms/step - loss: 2.2169 - acc: 0.2656 - val_loss: 2.3838 - val_acc: 0.0712
Epoch 4/10
4/4 [==============================] - 0s 2ms/step - loss: 2.1743 - acc: 0.3359 - val_loss: 2.3830 - val_acc: 0.0590
Epoch 5/10
4/4 [==============================] - 0s 2ms/step - loss: 2.1343 - acc: 0.3594 - val_loss: 2.3838 - val_acc: 0.0590
Epoch 6/10
4/4 [==============================] - 0s 2ms/step - loss: 2.0959 - acc: 0.3516 - val_loss: 2.3858 - val_acc: 0.0590
Epoch 7/10
4/4 [==============================] - 0s 4ms/step - loss: 2.0583 - acc: 0.3750 - val_loss: 2.3887 - val_acc: 0.0590
Epoch 8/10
4/4 [==============================] - 0s 2ms/step - loss: 2.0223 - acc: 0.4453 - val_loss: 2.3918 - val_acc: 0.0747
Epoch 9/10
4/4 [==============================] - 0s 2ms/step - loss: 1.9870 - acc: 0.4609 - val_loss: 2.3954 - val_acc: 0.1059
Epoch 10/10
4/4 [==============================] - 0s 2ms/step - loss: 1.9523 - acc: 0.4609 - val_loss: 2.3995 - val_acc: 0.1059
Callbacks
Adding callbacks also works,
callbacks = [ tf.keras.callbacks.ModelCheckpoint("model_{epoch:04d}_{val_acc:.4f}.h5",
monitor='val_acc',
verbose=1,
save_best_only=True,
mode='max'),
tf.keras.callbacks.TensorBoard('./logs'),
tf.keras.callbacks.EarlyStopping(monitor='val_acc', patience=5, min_delta=0, mode='max')
]
model.fit(train_dataset, epochs=10, steps_per_epoch=int(100/32)+1, validation_data=val_dataset,
validation_steps=2, callbacks=callbacks)
Output:
Epoch 1/10
4/4
[==============================] - 0s 59ms/step - loss: 2.3274 - acc: 0.1094 - val_loss: 2.3143 - val_acc: 0.0833
Epoch 00001: val_acc improved from -inf to 0.08333, saving model to model_0001_0.0833.h5
Epoch 2/10
4/4 [==============================] - 0s 2ms/step - loss: 2.2655 - acc: 0.1094 - val_loss: 2.3204 - val_acc: 0.1389
Epoch 00002: val_acc improved from 0.08333 to 0.13889, saving model to model_0002_0.1389.h5
Epoch 3/10
4/4 [==============================] - 0s 5ms/step - loss: 2.2122 - acc: 0.1250 - val_loss: 2.3289 - val_acc: 0.1111
Epoch 00003: val_acc did not improve from 0.13889
Epoch 4/10
4/4 [==============================] - 0s 2ms/step - loss: 2.1644 - acc: 0.1953 - val_loss: 2.3388 - val_acc: 0.0556
Epoch 00004: val_acc did not improve from 0.13889
Epoch 5/10
4/4 [==============================] - 0s 2ms/step - loss: 2.1211 - acc: 0.2734 - val_loss: 2.3495 - val_acc: 0.0556
Epoch 00005: val_acc did not improve from 0.13889
Epoch 6/10
4/4 [==============================] - 0s 4ms/step - loss: 2.0808 - acc: 0.2969 - val_loss: 2.3616 - val_acc: 0.0556
Epoch 00006: val_acc did not improve from 0.13889
Epoch 7/10
4/4 [==============================] - 0s 2ms/step - loss: 2.0431 - acc: 0.2969 - val_loss: 2.3749 - val_acc: 0.0712
Epoch 00007: val_acc did not improve from 0.13889
I think your problem is type of your datasets, where xt and yt is not a list or array, so it can not to slice or split.
Let assume x is array with shape (1000,2) fill with random number using numpy and y is class with 10 classes.
# Define x and y
import numpy as np
np.random.seed(2018)
n_cat = 10
x = np.random.rand(1000,2) # Generate random 2 variables and 1000 rows
y = [np.random.randint(1,n_cat) for i in range(len(x))] # Make 10 class
Then you assemble the model and compile
model.compile(
loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'],
)
Create callbacks for checkpoint model
callbacks = [
tf.keras.callbacks.ModelCheckpoint("model_{epoch:04d}_{val_acc:.4f}.h5",
monitor='val_acc',
verbose=1,
save_best_only=True,
mode='max'),
tf.keras.callbacks.TensorBoard(os.path.join('.', 'logs')),
tf.keras.callbacks.EarlyStopping(monitor='val_acc', patience=5, min_delta=0, mode='max')
]
fit the model
model.fit(x, y, batch_size=32, epochs=100, callbacks=callbacks, validation_split=0.1)
it's works for me.
I've spend the last 2 weeks struggling with my NN. The aim is to predict trip durations of taxi courses based on several
numerical variables (latitudes and longitudes)
categorical variables (numerically encoded) (hour of the day, day of the week, etc)
Here is the simplest version
X_train = trainData.as_matrix(columns=["fareDistance","hour","day","pickup_longitude","pickup_latitude","dropoff_longitude","dropoff_latitude"])
Y_train = np.array(trainData["trip_duration"])
model = Sequential()
model.add(Dense(32, input_dim=7, activation='linear'))
model.add(Dense(12, activation='linear'))
model.add(Dense(1, activation='linear'))
model.compile(loss='mean_absolute_percentage_error', optimizer='adagrad', metrics=['accuracy'])
model.summary()
model.fit(X_train, Y_train, epochs=10, validation_split=0.2)
I also tried to merge two different models for numerical variables on one hand and categorical on the other but it didn't change a thing. Depending on the combinations of Loss and optimization function either the loss and accuracy remain quite the same (acc. 0.0016) or I don't even have non null acc.
A friend of mine replicated the NN in pure TensorFlow and got the same kind of results
Train on 233383 samples, validate on 58346 samples
Epoch 1/20 233383/233383 [==============================] - 15s - loss: 45.9550 - acc: 0.0016 - val_loss: 46.2514 - val_acc: 0.0014
Epoch 2/20 233383/233383 [==============================] - 15s - loss: 45.8675 - acc: 0.0014 - val_loss: 46.2675 - val_acc: 0.0015
Epoch 3/20 233383/233383 [==============================] - 15s - loss: 45.8465 - acc: 0.0015 - val_loss: 46.2131 - val_acc: 0.0013
Epoch 4/20 233383/233383 [==============================] - 15s - loss: 45.8283 - acc: 0.0014 - val_loss: 46.2478 - val_acc: 0.0016
Epoch 5/20 233383/233383 [==============================] - 15s - loss: 45.8214 - acc: 0.0015 - val_loss: 46.2043 - val_acc: 0.0013
Epoch 6/20 233383/233383 [==============================] - 14s - loss: 45.8122 - acc: 0.0014 - val_loss: 46.2526 - val_acc: 0.0014
Epoch 7/20 233383/233383 [==============================] - 12s - loss: 45.7990 - acc: 0.0015 - val_loss: 46.1821 - val_acc: 0.0014
Epoch 8/20 233383/233383 [==============================] - 12s - loss: 45.7964 - acc: 0.0016 - val_loss: 46.1761 - val_acc: 0.0013
Epoch 9/20 233383/233383 [==============================] - 11s - loss: 45.7898 - acc: 0.0015 - val_loss: 46.1804 - val_acc: 0.0016
Am I missing something -- like something big, obvious -- which would explain why any attempt to change activation, loss or optimization function ends up doing the same?
Thanks in advance
D.
try this:
X_train = trainData.as_matrix(columns=["fareDistance","hour","day","pickup_longitude","pickup_latitude","dropoff_longitude","dropoff_latitude"])
Y_train = np.array(trainData["trip_duration"])
model = Sequential()
model.add(Dense(32, input_dim=7, activation='elu'))
model.add(Dense(12, activation='elu'))
model.add(Dense(1, kernel_initializer='normal'))
model.compile(loss='mean_absolute_percentage_error', optimizer='rmsprop')
model.summary()
model.fit(X_train, Y_train, epochs=10, validation_split=0.2)
you can also try the adam optimizer.
model.compile(loss='mean_absolute_percentage_error', optimizer='adam')
Update:
If the code above didn't help you it means your input data either not normalized or very dirty.