Keras Accuracy NAN - python

I am very new to machine learning and am trying to create a Keras model using data I have collected. It is perfectly uniform and loads in fine.
Here is a sample:
n,d0,d1,d2,d3,d4,d5,d6,d7,d8,output
30,85.1,65.0,32.2,38.2,191.9,72.1,118.2,121.5,110.3,0.0
417,232.8,51.3,39.8,66.0,173.4,246.7,285.4,265.6,217.0,1.0
496,194.2,72.7,214.8,41.6,155.2,195.2,208.3,31.0,15.6,2.0
361,206.1,52.8,63.0,105.1,168.5,156.0,145.7,127.4,70.6,1.0
408,202.5,48.4,47.4,79.1,223.8,236.6,260.3,247.4,206.2,1.0
Here is my Keras code:
import numpy as np
import pandas
from tensorflow import keras
from sklearn.model_selection import train_test_split
data = pandas.read_csv("data.csv")
x = data[[f"d{i}" for i in range(9)]]
y = data[["output"]]
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.1)
model = keras.models.Sequential()
model.add(keras.layers.Dense(12, input_dim=9, activation="relu"))
model.add(keras.layers.Dense(8, activation="relu"))
model.add(keras.layers.Dense(1, activation="softmax"))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x_train, y_train, epochs=150, batch_size=10)
print(model.predict(np.array([[0, 1, 2, 3, 4, 5, 6, 7, 8]])))
_, acc = model.evaluate(x, y)
print('Accuracy: %.2f' % (acc*100))
I don't see any issues, but I can't predict. Please could someone help?

I think you need to fix your output and your compile step.
If you're doing regression (ie. predicting true, continuous values) then you need to change your loss to RMSE or some other continuous loss, and not use a softmax classification. I don't think the accuracy metric will work either.
If you're doing classification then you need to change the number of outputs to the number of classes that you have. You also can't use binary cross entropy because you have more than one class, so use CategoricalCrossentropy. Note that according to the docs you have to change your current label representation (0,1,2..) to a one-hot representation. You can easily do this by:
y_one_hot = keras.utils.to_categorical(y)

Related

Prediction using Bidirectional LSTM

I have a table with 1799 users and 31 features which are arranged in rows and columns respectively. The last column is a 2-type condition feature that tells the model which condition the users belong to. I understood that by using LSTM I need to make my input to be 3-d. So, I used reshape(31,1) as I don't have time series data. I also understood that input_shape took in the number of features. My issue is that I want to predict a new set of users who also have the same 30 features and give me a classification result about which user belongs to which condition. It would be better if the result can tell me what is the probability of each of the conditions predicted. So, I tried to use model.predict to do the mentioned tasks. It gave me a result of a numpy array predict_prob with a shape=(200, 31, 1). I am confused at the part that the data structure should be [(31x1)x200] and the output should be the conditions of the users which should be (200,). How come the result is in 3-d and how should I convert it to dataframe format so that I can read it in .csv format? Thank you in advance.
X = raw_data[feature_names]
P = predict_data_raw[feature_names]
P1 = predict_data_raw[feature_names1]
#Training
y = raw_data['Conditions']
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=22, test_size=0.1)
X_test = np.expand_dims(X_test, axis=2)
# fit and evaluate a model
model = Sequential()
model.add(Reshape((31,1)))
model.add(Bidirectional(LSTM(10, return_sequences=True),input_shape=(31,)))
model.add(Dropout(0.5))
model.add(Dense(8, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
# compile the keras model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit the keras model on the dataset
LSTM = model.fit(X_train, y_train, epochs=5, batch_size=10)
# evaluate the keras model
_, accuracy = model.evaluate(X_test)
print('Accuracy: %.2f' % (accuracy*100))
predict_prob=model.predict([X_test])
df = pd.DataFrame(predict_prob, columns=["Prediction"])

I'm getting a ValueError: No gradients provided for any variable

I'm having a bit of trouble trying to get my code to work
import tensorflow as tf
from tensorflow import keras
import numpy as np
import pandas as pd
import csv
from sklearn.model_selection import train_test_split
batch_size = 1
csv = "EmergeSync.csv"
val_csv = "EmergeSync.csv"
dataframe = pd.read_csv(csv)
#Split the data
train, test_ds = train_test_split(dataframe, train_size=0.8, test_size=0.2)
train_ds, val_ds = train_test_split(train, train_size=0.8, test_size=0.2)
#Building the model
model = keras.Sequential()
units = 7
model.add(keras.layers.Flatten())
model.add(keras.layers.Dense(units=units, activation='linear'))
model.add(keras.layers.Dense(units=units, activation='linear'))
model.add(keras.layers.Dense(units=units, activation='linear'))
model.compile(optimizer="adam", loss="mean_squared_error", metrics=["accuracy"])
num_epochs = 2
history = model.fit(train_ds, epochs=num_epochs, steps_per_epoch=5, batch_size=batch_size, shuffle=True, validation_data=val_ds, verbose=1)
print(history)
I get the following error:
ValueError: No gradients provided for any variable: ['sequential/dense/kernel:0', 'sequential/dense/bias:0', 'sequential/dense_1/kernel:0', 'sequential/dense_1/bias:0', 'sequential/dense_2/kernel:0', 'sequential/dense_2/bias:0'].
I have no idea what is causing this error. If anyone could help me, that would be great!
So basically the error is correct there were no gradients found by the optimizer and can no longer update your network.
Now you need to ask yourself how are the gradients calculated. There are calculated by taking the partial derivative of your loss function w.r.t to all the parameters.
Your loss function is mean_square_error, so it looks something like (y-y')**2.
Here y being your original expected value and y' is what your model outputs.
If one of the two does not exist then the gradients cannot be calculated.
In your case, you are not supplying y to the model and due to this reason it is unable to calculate the gradients and unable to update the parameter values.
You will have to do the following.
history = model.fit(x=train_ds, y=np.zeros((10880,7)), epochs=num_epochs, steps_per_epoch=5, batch_size=batch_size, shuffle=True, validation_data=val_ds, verbose=1)
I do not know your y so I took dummy data, but you will have to call the fit API in an above-shown manner.
I have tried your code and it works on my system. I hope this answer finds you well.

Tensorflow model accuracy low

So my main goal is to use data from 2018 and try to predict data for 2019. I'm using a GRU model and I have the following code. I have a few issues, I'm not sure if the code is actually correct or if I am missing something, and also for model.fit should I use validation_split=0.1 or validation_data=X_test,y_test since I'm using a different dataframe for tesing.
Regarding the accuracy, it is very small and doesn't make any sense and I have no idea why.
import pandas as pd
import tensorflow as tf
from keras.layers.core import Dense
from keras.layers.recurrent import GRU
from keras.models import Sequential
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
from tensorboardcolab import TensorBoardColab, TensorBoardColabCallback
df = pd.read_csv('IF 10 PERCENT.csv',index_col=None)
#Loading Second Dataframe
df2 = pd.read_csv('2019 10minutes IF 10 PERCENT.csv',index_col=None)
tbc=TensorBoardColab() # Tensorboard
X_train= df[['WindSpeed_mps','AmbTemp_DegC','RotorSpeed_rpm','RotorSpeedAve','NacelleOrientation_Deg','MeasuredYawError','Pitch_Deg','WindSpeed1','WindSpeed2','WindSpeed3','GeneratorTemperature_DegC','GearBoxTemperature_DegC']]
X_train=X_train.values
y_train= df['Power_kW']
y_train=y_train.values
X_test= df2[['WindSpeed_mps','AmbTemp_DegC','RotorSpeed_rpm','RotorSpeedAve','NacelleOrientation_Deg','MeasuredYawError','Pitch_Deg','WindSpeed1','WindSpeed2','WindSpeed3','GeneratorTemperature_DegC','GearBoxTemperature_DegC']]
X_test=X_test.values
y_test= df2['Power_kW']
y_test=y_test.values
# conversion to numpy array
# scaling values for model
x_scale = MinMaxScaler()
y_scale = MinMaxScaler()
X_train= x_scale.fit_transform(X_train)
y_train= y_scale.fit_transform(y_train.reshape(-1,1))
X_test=x_scale.fit_transform(X_test)
y_test=y_scale.fit_transform(y_test.reshape(-1,1))
X_train = X_train.reshape((-1,1,12))
X_test = X_test.reshape((-1,1,12))
# splitting train and test
# creating model using Keras
model = Sequential()
model.add(GRU(units=512, return_sequences=True, input_shape=(1,12)))
model.add(GRU(units=256, return_sequences=True))
model.add(GRU(units=256))
model.add(Dense(units=1, activation='sigmoid'))
model.compile(loss=['mse'], optimizer='adam',metrics=['accuracy'])
model.summary()
#model.fit(X_train, y_train, batch_size=250, epochs=10, validation_split=0.1, verbose=1, callbacks=[TensorBoardColabCallback(tbc)])
model.fit(X_train, y_train, batch_size=250, epochs=10, validation_data=(X_test,y_test), verbose=1, callbacks=[TensorBoardColabCallback(tbc)])
score = model.evaluate(X_test, y_test)
print('Score: {}'.format(score))
print('Accuracy: {}'.format(acc))
y_predicted = model.predict(X_test)
y_predicted = y_scale.inverse_transform(y_predicted)
y_t
est = y_scale.inverse_transform(y_test)
plt.plot(y_predicted, label='Predicted')
plt.plot(y_test, label='Measurements')
plt.legend()
plt.show()
Thank you
It sounds to me that you are trying to solve a regression problem here. if it is so, It does not make sense to measure accuracy as a metric, since accuracy is about to measure the exact label matching. MSE should be pretty good for the regression

Displaying predicted values of 'y' with corresponding actual 'x' values in Keras

I am attempting to write a neural network that uses 100 data points evenly spaced along a y = sin(x) graph, between 0 and (5/2)pi. So the x values go from 0 to 7.854, and the corresponding y values oscillate between 1 and -1.
I have split the data into training and validation points, and it seems to have trained properly, however when I try to print the 'test' data, it is 100% correct, no matter how many neurons, epochs that I use. I think it is simply displaying the actual values, no predictions.
I think that I must have done something wrong in the final predictions part of the code, but I am not sure how to display the predicted values of y by the network for corresponding x values. Any help would be greatly appreciated!
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Activation
import pandas as pd
import io
import os
import requests
import numpy as np
from sklearn import metrics
from sklearn.model_selection import train_test_split
from tensorflow.keras.callbacks import EarlyStopping
df = pd.read_csv("C:\\Users\\Dan\\Desktop\\UNI\\PROJECT\\y_sinx_values.csv")
x = df['x'].values
y = df['y'].values
# Split into training and validation sets
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.25, random_state=42)
model = Sequential()
model.add(Dense(100, input_shape = (1,), activation='relu'))
model.add(Dense(50, activation='relu'))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer = 'adam')
monitor = EarlyStopping(monitor='val_loss', min_delta=1e-3, patience=1, verbose=1, mode='auto', restore_best_weights=True)
model.fit(x_train,y_train,validation_data=(x_test,y_test),callbacks=[monitor],verbose=2, epochs=2)
pred = model.predict(x_test)
score = np.sqrt(metrics.mean_squared_error(pred,y_test))
print(f"Final score (RMSE): {score}")
# Sample predictions
for i in range(25):
print(f"{i+1}. x: {x_test[i]}, y: {y_test[i]}")
# Sample predictions
for i in range(25):
print(f"{i+1}. x: {x_test[i]}, y: {y_test[i]}")
Yes, you are printing your input test data. You most likely want something like:
# Sample predictions
for i in range(25):
print(f"{i+1}. x: {x_test[i]}, y: {pred[i]}")
Note the y_test[i] has been changed to pred[i].

How can I get the history of the different fits when using cross vaidation over a KerasRegressor?

I have a regression problem and I am using a keras fully connected layer to model my problem. I am using cross_val_score and my question is: how can I extract the model and the history of each train/validation combination the cross_val_score does?
Assuming this example:
from sklearn import datasets
from sklearn.model_selection import cross_val_score, KFold
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
seed = 1
diabetes = datasets.load_diabetes()
X = diabetes.data[:150]
y = diabetes.target[:150]
def baseline_model():
model = Sequential()
model.add(Dense(10, input_dim=10, activation='relu'))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
return model
estimator = KerasRegressor(build_fn=baseline_model, nb_epoch=100, batch_size=100, verbose=False)
kfold = KFold(n_splits=10, random_state=seed)
results = cross_val_score(estimator, X, y, cv=kfold)
print("Results: %.2f (%.2f) MSE" % (results.mean(), results.std()))
My understanding is that I only get the overall mse over each fold, so to say.
But I want to compare the train to validation mse over the epochs of the model for each fold, i.e. for 10 in this case.
When not using kfold, but simple train/validation split, then one can do:
hist = model.fit(X_tr, y_tr, validation_data=val_data,
epochs=100, batch_size=100,
verbose=1)
plt.plot(history.history['loss'])
plt.plot(history.history['loss'])
This would return a plot representing the evolution of the mse w.r.t. to the epochs for the train and validation datasets, allowing to spot over/underfitting.
How to do this for each fold when using cross validation?
You can go for a "manual" CV procedure, and plot the loss (or any other available metric you might want to use) for each fold, i.e. something like this:
from sklearn.metrics import mean_squared_error
cv_mse = []
for train_index, val_index in kfold.split(X):
history = estimator.fit(X[train_index], y[train_index])
pred = estimator.predict(X[val_index])
err = mean_squared_error(y[val_index], pred)
cv_mse.append(err)
plt.plot(history.history['loss'])
In which case, the cv_mse list will contain the final MSE for each fold, and you also get the respective plots for its evolution per epoch for each fold.

Categories

Resources