When I was running a tensorflow model on python, the accuracy of my model can't be improved by training. Even if I change my training data to a quite regular one, the model still didn't work. What's the problem?
Code:
train_x = np.array([1] * 1000 + [2] * 1000 + [3] * 1000)
train_y = np.zeros((3000, 3))
train_y[:1000,0] = 1
train_y[1000:2000,1] = 1
train_y[2000:3000,2] = 1
val_x = train_x
val_y = train_y
model = tf.keras.Sequential()
model.add(layers.Dense(3, activation='relu'))
model.add(layers.Dense(3, activation='relu'))
model.compile(optimizer=tf.keras.optimizers.Adam(0.1),
loss=tf.keras.losses.categorical_crossentropy,
metrics=[tf.keras.metrics.categorical_accuracy])
model.fit(train_x, train_y, epochs = 10, batch_size = 32, verbose = 1,
shuffle = False,
validation_data=(val_x, val_y))
And training result
Epoch 1/10
94/94 [==============================] - 0s 2ms/step - loss: 10.7836 - categorical_accuracy: 0.3120 - val_loss: 10.7454 - val_categorical_accuracy: 0.3333
Epoch 2/10
94/94 [==============================] - 0s 1ms/step - loss: 10.7454 - categorical_accuracy: 0.3333 - val_loss: 10.7454 - val_categorical_accuracy: 0.3333
Epoch 3/10
94/94 [==============================] - 0s 1ms/step - loss: 10.7454 - categorical_accuracy: 0.3333 - val_loss: 10.7454 - val_categorical_accuracy: 0.3333
Epoch 4/10
94/94 [==============================] - 0s 1ms/step - loss: 10.7454 - categorical_accuracy: 0.3333 - val_loss: 10.7454 - val_categorical_accuracy: 0.3333
Epoch 5/10
94/94 [==============================] - 0s 2ms/step - loss: 10.7454 - categorical_accuracy: 0.3333 - val_loss: 10.7454 - val_categorical_accuracy: 0.3333
So where I should adjust to get better performance, and which thing I have done wrong?
The problem is that with 3 input neurons and 1 feature (i.e., 1 column), the neural network doesn't have enough if... then combinations to learn the pattern you're trying to teach it. If you one-hot encode your input, it will effectively learn to multiply every input column by one and it will give the right answer.
You have the wrong activation function. For multi-class problems, use 'softmax'.
Your optimizer's learning rate is a little too high, so the step is too high and jumping all over the cost function. Use 0.01 at most.
Fully-working example:
import numpy as np
import tensorflow as tf
train_x = np.array([1] * 1000 + [2] * 1000 + [3] * 1000)
train_x = tf.keras.utils.to_categorical(train_x - 1)
train_y = np.zeros((3000, 3))
train_y[:1000,0] = 1
train_y[1000:2000,1] = 1
train_y[2000:3000,2] = 1
val_x = train_x
val_y = train_y
model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(3, activation='relu'))
model.add(tf.keras.layers.Dense(3, activation='softmax'))
model.compile(optimizer=tf.keras.optimizers.Adam(0.01),
loss=tf.keras.losses.categorical_crossentropy,
metrics=[tf.keras.metrics.categorical_accuracy])
model.fit(train_x, train_y, epochs = 10, batch_size = 32, verbose = 1,
shuffle = False,
validation_data=(val_x, val_y))
Epoch 9/10
32/3000 [..............................] - ETA: 0s - loss: 0.0067 - cat_acc: 1.0000
608/3000 [=====>........................] - ETA: 0s - loss: 0.0063 - cat_acc: 1.0000
1184/3000 [==========>...................] - ETA: 0s - loss: 0.0244 - cat_acc: 1.0000
1760/3000 [================>.............] - ETA: 0s - loss: 0.0553 - cat_acc: 1.0000
2272/3000 [=====================>........] - ETA: 0s - loss: 0.0550 - cat_acc: 1.0000
2848/3000 [===========================>..] - ETA: 0s - loss: 0.0447 - cat_acc: 1.0000
Related
I'm working on a CNN model with Keras for Human vs Horses dataset to predict some images.
with following codes I build the model and save in a file:
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from keras.optimizers import RMSprop
training_dir = 'horse-or-human/training'
train_datagen = ImageDataGenerator(
rescale=1/255,
rotation_range=40,
width_shift_range= 0.2,
height_shift_range= 0.2,
shear_range=0.2,
zoom_range= 0.2,
horizontal_flip= True,
fill_mode='nearest'
)
train_generator = train_datagen.flow_from_directory(training_dir , target_size=(300,300) , class_mode='binary')
model = tf.keras.models.Sequential([
tf.keras.layers.Conv2D(16 , (3,3), activation=tf.nn.relu , input_shape = (300,300,3)),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(32 , (3,3), activation=tf.nn.relu),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(64 , (3,3), activation=tf.nn.relu),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(64 , (3,3), activation=tf.nn.relu),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(64 , (3,3), activation=tf.nn.relu),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512 ,activation=tf.nn.relu ),
tf.keras.layers.Dense(1, activation = tf.nn.sigmoid)
])
model.compile(optimizer = RMSprop(learning_rate = 0.001) , metrics=['accuracy'] , loss='binary_crossentropy' )
validation_dir = 'horse-or-human/validation'
validation_datagen = ImageDataGenerator(rescale=1/255)
validation_generator = validation_datagen.flow_from_directory(
validation_dir ,
target_size=(300,300) ,
class_mode='binary'
)
model.fit(train_generator , epochs= 15 ,validation_data=validation_generator)
model.save('human-horses-model.h5')
And this part of my code that using that model to predict s specific image :
import tensorflow as tf
from ipyfilechooser import FileChooser
import keras.utils as image
import numpy as np
model = tf.keras.models.load_model('human-horses-model.h5')
fc = FileChooser()
display(fc)
img = image.load_img(fc.selected , target_size=(300,300))
img = image.img_to_array(img)
img /= 255.
img = np.expand_dims(img , axis=0)
output = model.predict(img)
if output[0]> 0.5 :
print('selected Image is a Human')
else :
print('selected Image is a Horses')
And following is output of each epochs:
Found 256 images belonging to 2 classes.
Epoch 1/15
33/33 [==============================] - 83s 2s/step - loss: 0.7800 - accuracy: 0.5686 - val_loss: 0.6024 - val_accuracy: 0.5859
Epoch 2/15
33/33 [==============================] - 73s 2s/step - loss: 0.6430 - accuracy: 0.6777 - val_loss: 0.8060 - val_accuracy: 0.5586
Epoch 3/15
33/33 [==============================] - 77s 2s/step - loss: 0.5252 - accuracy: 0.7595 - val_loss: 0.7498 - val_accuracy: 0.6875
Epoch 4/15
33/33 [==============================] - 79s 2s/step - loss: 0.4754 - accuracy: 0.7731 - val_loss: 1.7478 - val_accuracy: 0.5938
Epoch 5/15
33/33 [==============================] - 77s 2s/step - loss: 0.3966 - accuracy: 0.8130 - val_loss: 2.0004 - val_accuracy: 0.5234
Epoch 6/15
33/33 [==============================] - 73s 2s/step - loss: 0.4196 - accuracy: 0.8442 - val_loss: 0.3918 - val_accuracy: 0.8281
Epoch 7/15
33/33 [==============================] - 73s 2s/step - loss: 0.2859 - accuracy: 0.8802 - val_loss: 1.6727 - val_accuracy: 0.6680
Epoch 8/15
33/33 [==============================] - 74s 2s/step - loss: 0.2489 - accuracy: 0.8929 - val_loss: 3.1737 - val_accuracy: 0.6484
Epoch 9/15
33/33 [==============================] - 76s 2s/step - loss: 0.2829 - accuracy: 0.8948 - val_loss: 1.8389 - val_accuracy: 0.7109
Epoch 10/15
33/33 [==============================] - 76s 2s/step - loss: 0.2140 - accuracy: 0.9250 - val_loss: 1.8419 - val_accuracy: 0.7930
Epoch 11/15
33/33 [==============================] - 73s 2s/step - loss: 0.2341 - accuracy: 0.9299 - val_loss: 1.5261 - val_accuracy: 0.6914
Epoch 12/15
33/33 [==============================] - 74s 2s/step - loss: 0.1576 - accuracy: 0.9464 - val_loss: 0.9359 - val_accuracy: 0.8398
Epoch 13/15
33/33 [==============================] - 75s 2s/step - loss: 0.2002 - accuracy: 0.9250 - val_loss: 1.9854 - val_accuracy: 0.7344
Epoch 14/15
33/33 [==============================] - 79s 2s/step - loss: 0.1854 - accuracy: 0.9406 - val_loss: 0.7637 - val_accuracy: 0.8164
Epoch 15/15
33/33 [==============================] - 80s 2s/step - loss: 0.1160 - accuracy: 0.9611 - val_loss: 1.6901 - val_accuracy: 0.7656
My model always return 1 or a number very near to 1 that show all images are Human while in real that are Horse.
I searched a lot but did not find the answer!
Can anyone help me to find and solve the problem.
Please use class_mode='sparse' as you are using the image dataset and Adam optimzer for better results or change the kernel size to (7,7) or (5,5) to convolve the large image dataset efficiently.
(I have replicated the same code and attached the gist for your reference which predicts the expected result as you can see in the image below)
You can also try using Pretrained model or Strategies to prevent overfitting for fine tune the model performance.
Input X = [[1,1,1,1,1], [1,2,1,3,7], [3,1,5,7,8]] etc..
Output Y = [[0.77],[0.63],[0.77],[1.26]] etc..
input x mean some combination example
["car", "black", "sport", "xenon", "5dor"]
["car", "red", "sport", "noxenon", "3dor"] etc...
output mean some score of combination.
What i need? i need to predict is combination good or bad....
Dataset size 10k..
Model:
model.add(Dense(20, input_dim = 5, activation = 'relu'))
model.add(Dense(20, activation = 'relu'))
model.add(Dense(1, activation = 'linear'))
optimizer = adam, loss = mse, validation split 0.2, epoch 30
Tr:
Epoch 1/30
238/238 [==============================] - 0s 783us/step - loss: 29.8973 - val_loss: 19.0270
Epoch 2/30
238/238 [==============================] - 0s 599us/step - loss: 29.6696 - val_loss: 19.0100
Epoch 3/30
238/238 [==============================] - 0s 579us/step - loss: 29.6606 - val_loss: 19.0066
Epoch 4/30
238/238 [==============================] - 0s 583us/step - loss: 29.6579 - val_loss: 19.0050
Epoch 5/30
not good no sens...
i need some good documentation how to proper setup or build model...
Just tried to reproduce. My results differ from yours. Please check:
import tensorflow as tf
from tensorflow.keras.layers import Input, Dense
from tensorflow.keras import Model
inputA = Input(shape=(5, ))
x = Dense(20, activation='relu')(inputA)
x = Dense(20, activation='relu')(x)
x = Dense(1, activation='linear')(x)
model = Model(inputs=inputA, outputs=x)
model.compile(optimizer = 'adam', loss = 'mse')
input = tf.random.uniform([10000, 5], 0, 10, dtype=tf.int32)
labels = tf.random.uniform([10000, 1])
model.fit(input, labels, epochs=30, validation_split=0.2)
Results:
Epoch 1/30 250/250 [==============================] - 1s 3ms/step -
loss: 0.1980 - val_loss: 0.1082
Epoch 2/30 250/250 [==============================] - 1s 2ms/step -
loss: 0.0988 - val_loss: 0.0951
Epoch 3/30 250/250 [==============================] - 1s 2ms/step -
loss: 0.0918 - val_loss: 0.0916
Epoch 4/30 250/250 [==============================] - 1s 2ms/step -
loss: 0.0892 - val_loss: 0.0872
Epoch 5/30 250/250 [==============================] - 0s 2ms/step -
loss: 0.0886 - val_loss: 0.0859
Epoch 6/30 250/250 [==============================] - 1s 2ms/step -
loss: 0.0864 - val_loss: 0.0860
Epoch 7/30 250/250 [==============================] - 1s 3ms/step -
loss: 0.0873 - val_loss: 0.0863
Epoch 8/30 250/250 [==============================] - 1s 2ms/step -
loss: 0.0863 - val_loss: 0.0992
Epoch 9/30 250/250 [==============================] - 0s 2ms/step -
loss: 0.0876 - val_loss: 0.0865
The model should work on real figures.
I found some reports where the loss and accuracy does not change. But here the loss changes — it is giant! — and the accuracy does not change.
I've tried to increase the dataframe, decrease the dataframe, change the activation function. etc. But nothing changes the accuracy.
I have also tried to change the batch_size and the epochs. The curious thing is that when I use train_test_split, it works. Any suggestions?
Thank you!
Base = pd.read_csv(r"C:\...\Teste 6x3 Janela1.csv", sep=";", decimal=",")
x_treinamento = Base.iloc[Linha_i:Linha_f, 20:134]
y_treinamento = Base.iloc[Linha_i:Linha_f, 12]
x_teste = Base.iloc[Linha_i_Teste:Linha_f_Teste, 20:134]
y_teste = Base.iloc[Linha_i_Teste:Linha_f_Teste, 12]
rna = Sequential()
rna.add(Dense(units = 226, activation = 'relu', kernel_initializer = 'random_uniform', input_dim = 113))
rna.add(Dense(units = 226, activation = 'relu'))
rna.add(Dense(units = 1, activation = 'sigmoid'))
rna.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['binary_accuracy'])
rna.fit(x_treinamento, y_treinamento, batch_size = 2, epochs = 100)
Epoch 1/100
89/89 [==============================] - 0s 3ms/step - loss: -54259.0588 - binary_accuracy: 0.2472
Epoch 2/100
89/89 [==============================] - 0s 1ms/step - loss: -783880.1310 - binary_accuracy: 0.2472
Epoch 3/100
89/89 [==============================] - 0s 1ms/step - loss: -4232499.0056 - binary_accuracy: 0.2472
Epoch 4/100
89/89 [==============================] - 0s 1ms/step - loss: -13589123.4944 - binary_accuracy: 0.2472
Epoch 5/100
89/89 [==============================] - 0s 1ms/step - loss: -32081543.0112 - binary_accuracy: 0.2472
Epoch 6/100
89/89 [==============================] - 0s 1ms/step - loss: -63825447.9551 - binary_accuracy: 0.2472
.
.
.
Epoch 96/100
89/89 [==============================] - 0s 1ms/step - loss: -240481326482.6966 - binary_accuracy: 0.2472
Epoch 97/100
89/89 [==============================] - 0s 1ms/step - loss: -246697877964.2247 - binary_accuracy: 0.2472
Epoch 98/100
89/89 [==============================] - 0s 1ms/step - loss: -253059365623.3708 - binary_accuracy: 0.2472
Epoch 99/100
89/89 [==============================] - 0s 1ms/step - loss: -259567427491.9550 - binary_accuracy: 0.2472
Epoch 100/100
89/89 [==============================] - 0s 1ms/step - loss: -266072634862.7416 - binary_accuracy: 0.2472
I'll answer here 'cause I don't have enough reputation to comment.
Try do randomize your train and test data, when you do not do that, the model can learn with the sequence of data and overfit or underfit. sklearn.train_test_split() can do that for you.
I found! I solved with:
ss = StandardScaler()
x_treinamento = ss.fit_transform(x_treinamento)
encoder = LabelEncoder()
encoder.fit(y_treinamento)
y_treinamento = encoder.transform(y_treinamento)
I wrote this code a few days ago and I had a few bugs but with some help, I was able to fix them. The Model is not learning. I tried different batch sizes, different amount of epochs, different activation functions, checked my data a few times for flaws I wasn't able to find any. It is due in a week or so for a school project. Any help will be very much valued.
Here is the code.
from keras.layers import Dense, Input, Concatenate, Dropout
from sklearn.preprocessing import MinMaxScaler
from keras.models import Model
from keras.layers import LSTM
import tensorflow as tf
import NetworkRequest as NR
import ParseNetworkRequest as PNR
import numpy as np
def buildModel():
_Price = Input(shape=(1, 1))
_Volume = Input(shape=(1, 1))
PriceLayer = LSTM(128)(_Price)
VolumeLayer = LSTM(128)(_Volume)
merged = Concatenate(axis=1)([PriceLayer, VolumeLayer])
Dropout(0.2)
dense1 = Dense(128, input_dim=2, activation='relu', use_bias=True)(merged)
Dropout(0.2)
dense2 = Dense(64, input_dim=2, activation='relu', use_bias=True)(dense1)
Dropout(0.2)
output = Dense(1, activation='softmax', use_bias=True)(dense2)
opt = tf.keras.optimizers.Adam(learning_rate=1e-3, decay=1e-6)
_Model = Model(inputs=[_Price, _Volume], output=output)
_Model.compile(optimizer=opt, loss='mse', metrics=['accuracy'])
return _Model
if __name__ == '__main__':
api_key = "47BGPYJPFN4CEC20"
stock = "DJI"
Index = ['4. close', '5. volume']
RawData = NR.Initial_Network_Request(api_key, stock)
Closing = PNR.Parse_Network_Request(RawData, Index[0])
Volume = PNR.Parse_Network_Request(RawData, Index[1])
Length = len(Closing)
scalar = MinMaxScaler(feature_range=(0, 1))
Closing_scaled = scalar.fit_transform(np.reshape(Closing[:-1], (-1, 1)))
Volume_scaled = scalar.fit_transform(np.reshape(Volume[:-1], (-1, 1)))
Labels_scaled = scalar.fit_transform(np.reshape(Closing[1:], (-1, 1)))
Train_Closing = Closing_scaled[:int(0.9 * Length)]
Train_Closing = np.reshape(Train_Closing, (Train_Closing.shape[0], 1, 1))
Train_Volume = Volume_scaled[:int(0.9 * Length)]
Train_Volume = np.reshape(Train_Volume, (Train_Volume.shape[0], 1, 1))
Train_Labels = Labels_scaled[:int((0.9 * Length))]
Train_Labels = np.reshape(Train_Labels, (Train_Labels.shape[0], 1))
# -------------------------------------------------------------------------------------------#
Test_Closing = Closing_scaled[int(0.9 * Length):(Length - 1)]
Test_Closing = np.reshape(Test_Closing, (Test_Closing.shape[0], 1, 1))
Test_Volume = Volume_scaled[int(0.9 * Length):(Length - 1)]
Test_Volume = np.reshape(Test_Volume, (Test_Volume.shape[0], 1, 1))
Test_Labels = Labels_scaled[int(0.9 * Length):(Length - 1)]
Test_Labels = np.reshape(Test_Labels, (Test_Labels.shape[0], 1))
Predict_Closing = Closing_scaled[-1]
Predict_Closing = np.reshape(Predict_Closing, (Predict_Closing.shape[0], 1, 1))
Predict_Volume = Volume_scaled[-1]
Predict_Volume = np.reshape(Predict_Volume, (Predict_Volume.shape[0], 1, 1))
Predict_Label = Labels_scaled[-1]
Predict_Label = np.reshape(Predict_Label, (Predict_Label.shape[0], 1))
model = buildModel()
model.fit(
[
Train_Closing,
Train_Volume
],
[
Train_Labels
],
validation_data=(
[
Test_Closing,
Test_Volume
],
[
Test_Labels
]
),
epochs=10,
batch_size=Length
)
This is the output when I run it.
Using TensorFlow backend.
2020-01-01 16:31:47.905012: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2199985000 Hz
2020-01-01 16:31:47.906105: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x49214f0 executing computations on platform Host. Devices:
2020-01-01 16:31:47.906137: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): Host, Default Version
/home/martin/PycharmProjects/MarketPredictor/Model.py:26: UserWarning: Update your `Model` call to the Keras 2 API: `Model(inputs=[<tf.Tenso..., outputs=Tensor("de...)`
_Model = Model(inputs=[_Price, _Volume], output=output)
Train on 4527 samples, validate on 503 samples
Epoch 1/10
4527/4527 [==============================] - 1s 179us/step - loss: 0.4716 - accuracy: 2.2090e-04 - val_loss: 0.6772 - val_accuracy: 0.0000e+00
Epoch 2/10
4527/4527 [==============================] - 0s 41us/step - loss: 0.4716 - accuracy: 2.2090e-04 - val_loss: 0.6772 - val_accuracy: 0.0000e+00
Epoch 3/10
4527/4527 [==============================] - 0s 42us/step - loss: 0.4716 - accuracy: 2.2090e-04 - val_loss: 0.6772 - val_accuracy: 0.0000e+00
Epoch 4/10
4527/4527 [==============================] - 0s 42us/step - loss: 0.4716 - accuracy: 2.2090e-04 - val_loss: 0.6772 - val_accuracy: 0.0000e+00
Epoch 5/10
4527/4527 [==============================] - 0s 43us/step - loss: 0.4716 - accuracy: 2.2090e-04 - val_loss: 0.6772 - val_accuracy: 0.0000e+00
Epoch 6/10
4527/4527 [==============================] - 0s 39us/step - loss: 0.4716 - accuracy: 2.2090e-04 - val_loss: 0.6772 - val_accuracy: 0.0000e+00
Epoch 7/10
4527/4527 [==============================] - 0s 42us/step - loss: 0.4716 - accuracy: 2.2090e-04 - val_loss: 0.6772 - val_accuracy: 0.0000e+00
Epoch 8/10
4527/4527 [==============================] - 0s 39us/step - loss: 0.4716 - accuracy: 2.2090e-04 - val_loss: 0.6772 - val_accuracy: 0.0000e+00
Epoch 9/10
4527/4527 [==============================] - 0s 42us/step - loss: 0.4716 - accuracy: 2.2090e-04 - val_loss: 0.6772 - val_accuracy: 0.0000e+00
Epoch 10/10
4527/4527 [==============================] - 0s 38us/step - loss: 0.4716 - accuracy: 2.2090e-04 - val_loss: 0.6772 - val_accuracy: 0.0000e+00
Process finished with exit code 0
The loss is high, and the accuracy is 0.
Please help.
You're using activation functions and metrics made for a classification task, not a stock forecasting task (with a continuous target).
For continuous targets, your final activation layer should be linear. Metrics should be mse or mae, not accuracy.
accuracy would only be satisfied is the dji prediction is exactly equal to the actual price. Since dji has at least 7 digits, it's nearly impossible.
Here's my suggestion:
Use a simpler network: Not sure how big is your dataset, but sometimes using dense. layer isn't helpful. Looks like the weights of there intermediate layers are not changing at all. Try the model with just one dense layer.
Reduce dropout: Try with using one dropout layer with Dropout(0.1).
Adam defaults: Start with using adam optimizer with its default parameters.
Metric selection: As mentioned by Nicolas's answer, use a regression metric instead of accuracy.
How do I define a custom keras metric for computing accuracy like so,
y_true = [12.5, 45.5]
y_predicted = [14.5, 29]
splits = [-float("inf"), 10, 20, 30, float("inf")]
"""
Splits to Classes translation =>
Class 0: -inf to 9
Class 1: 10 to 19
Class 2: 20 to 29
Class 3: 30 to inf
"""
# using the above translation,
y_true_classes = [1, 3]
y_predicted_classes = [1, 2]
accuracy = K.equal( y_true_classes, y_predicted_classes ) # => 0.5 here
return accuracy
Here is an idea on how you might you around implementing this (although probably not the best one).
def convert_to_classes(vals, splits):
out = tf.zeros_like(vals, dtype=tf.int32)
for split in splits:
out = tf.where(vals > split, out + 1, out)
return out
def my_acc(splits):
def custom_acc(y_true, y_pred):
y_true = convert_to_classes(y_true, splits)
y_pred = convert_to_classes(y_pred, splits)
return K.mean(K.equal(y_true, y_pred))
return custom_acc
The function convert_to_classes converts the floats to bucks, assuming the bounds are always +-inf.
The closure my_acc lets you define the splits (without +-inf) at compile time (added statically to the graph), and then returns a metric function as expected with keras.
Testing using tensorflow:
y_true = tf.constant([12.5, 45.5])
y_pred = tf.constant([14.5, 29])
with tf.Session() as sess:
print(sess.run(my_acc((10, 20, 30))(y_true, y_pred)))
gives the expected 0.5 accuracy.
And quick test with Keras:
x = np.random.randn(100, 10)*100
y = np.random.randn(100)*100
model = Sequential([Dense(20, activation='relu'),
Dense(1, activation=None)])
model.compile(optimizer='Adam',
loss='mse',
metrics=[my_acc(splits=(10, 20, 30))])
model.fit(x, y, batch_size=32, epochs=10)
Given the metric (named as the inner function in the closure custom_acc)
100/100 [==============================] - 0s 2ms/step - loss: 10242.2591 - custom_acc: 0.4300
Epoch 2/10
100/100 [==============================] - 0s 53us/step - loss: 10101.9658 - custom_acc: 0.4200
Epoch 3/10
100/100 [==============================] - 0s 53us/step - loss: 10011.4662 - custom_acc: 0.4300
Epoch 4/10
100/100 [==============================] - 0s 51us/step - loss: 9899.7181 - custom_acc: 0.4300
Epoch 5/10
100/100 [==============================] - 0s 50us/step - loss: 9815.1607 - custom_acc: 0.4200
Epoch 6/10
100/100 [==============================] - 0s 74us/step - loss: 9736.5554 - custom_acc: 0.4300
Epoch 7/10
100/100 [==============================] - 0s 50us/step - loss: 9667.0845 - custom_acc: 0.4400
Epoch 8/10
100/100 [==============================] - 0s 58us/step - loss: 9589.5439 - custom_acc: 0.4400
Epoch 9/10
100/100 [==============================] - 0s 61us/step - loss: 9511.8003 - custom_acc: 0.4400
Epoch 10/10
100/100 [==============================] - 0s 51us/step - loss: 9443.9730 - custom_acc: 0.4400