I try to run SHAP with my basic model LSTM with 3D input.
import shap
import tensorflow.keras.backend
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from tensorflow.keras.models import load_model
import shap
tf.compat.v1.disable_eager_execution()
regressor = load_model(Model_name)
pred_x = regressor.predict_classes(X_Training)
#explainer = shap.DeepExplainer((regressor.layers[0].input, regressor.layers[-1].output),data)
background = X_Training[np.random.choice(X_Training.shape[0], 200, replace=False)]
explainer = shap.DeepExplainer(regressor,background)
test = X_Training[np.random.choice(X_Training.shape[0], 20, replace=False)]
shap_val = explainer.shap_values(test,check_additivity=False)
shap_val = np.array(shap_val)
shap_val = np.reshape(shap_val,(int(shap_val.shape[1]),int(shap_val.shape[2]),int(shap_val.shape[3])))
shap_abs = np.absolute(shap_val)
sum_0 = np.sum(shap_abs,axis=0)
f_names = ['A','B','C','D','E','F','G','H','K','L','M','Results']
shap.initjs()
shap.summary_plot(
shap_val[0],
#X_Testing,
feature_names=f_names[:-1],
max_display=50,
plot_type='bar')
The output is as picture below.
The model summary()
I checked the shap_values and all values are zeros.
The versions of shap and tensorflow are 0.37.0 and 2.3.0 respectively.
Related
import numpy as np
import pandas as pd
import matplotlib
import xgboost as xgb
from sklearn.metrics import roc_auc_score
from xgboost.sklearn import XGBClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import KFold
train = pd.read_csv("train_final.csv")
y = train['Y']
YValues = []
for x in range(len(y)):
YValues.append(y[x])
print(YValues)
print(type(YValues))
YVal = np.array(YValues)
train = train.drop(['Y'], axis=1)
test = pd.read_csv("test_final.csv")
dtrain = xgb.DMatrix(train, label = y)
dtest = xgb.DMatrix(test)
xgb2_hyperparams = XGBClassifier()
xgb2_hyperparams = xgb2.predict_proba(test)
xgb2_hyperparams_test = xgb2.predict_proba(train)
print('Accuracy: ', roc_auc_score(YVal, xgb2_hyperparams_test))
np.savetxt("xgboostHyperParams.csv", xgb2_hyperparams, delimiter=",")
print(xgb2_hyperparams)
I've explicitly created YVal to be a 1D np-array but it is still saying that YVal is an array of shape (2603, 2) and I'm not sure what is up with that. I originally tried fiddling with y but that led to more errors and at this point, I'm not sure why Python is so adamant about the (2603, 2) shape - I'm not sure what I'm missing that it is always reading it as (2603, 2) no matter whether it is data type series, ndarray or array.
You have to convert it to an array. Pandas essentially creates a container around your data. Try:
import numpy as np
import pandas as pd
import matplotlib
import xgboost as xgb
from sklearn.metrics import roc_auc_score
from xgboost.sklearn import XGBClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import KFold
train = pd.read_csv("train_final.csv")
y = train['Y'].values
YValues = []
for x in range(len(y)):
YValues.append(y[x])
print(YValues)
print(type(YValues))
YVal = np.array(YValues)
train = train.drop(['Y'], axis=1).values
test = pd.read_csv("test_final.csv")
dtrain = xgb.DMatrix(train, label = y)
dtest = xgb.DMatrix(test)
xgb2_hyperparams = XGBClassifier()
xgb2_hyperparams = xgb2.predict_proba(test)
xgb2_hyperparams_test = xgb2.predict_proba(train)
print('Accuracy: ', roc_auc_score(YVal, xgb2_hyperparams_test))
np.savetxt("xgboostHyperParams.csv", xgb2_hyperparams, delimiter=",")
print(xgb2_hyperparams)
I have tried the methods mentioned in the post "https://stackoverflow.com/questions/32419510/how-to-get-reproducible-results-in-keras" but unable to reproduce the result in term of accuracy where I got different prediction for different run. The experiment I am running is about classification and the model used is CNN. The code below is the only part I do to try to reproduce the result.
import os
random_seed = 0
os.environ['PYTHONHASHSEED']=str(random_seed)
from __future__ import print_function, division
import tensorflow as tf
import keras.backend as K
session_conf = tf.compat.v1.ConfigProto(intra_op_parallelism_threads=1, inter_op_parallelism_threads=1)
session_conf.gpu_options.allow_growth = True
sess = tf.compat.v1.Session(graph=tf.compat.v1.get_default_graph(), config=session_conf)
K.set_session(sess)
# for later session
#tf.compat.v1.keras.backend.set_session(sess)
import csv
import glob
import numpy as np
import time
from keras.layers import *
import keras.backend
from keras.layers.advanced_activations import *
from keras.layers.convolutional import *
from keras.models import *
from keras.utils import *
from sklearn.utils import shuffle
import math
from keras import optimizers
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
random_seed = 0
os.environ['PYTHONHASHSEED']=str(random_seed)
np.random.seed(random_seed)
tf.random.set_seed(random_seed)
random.seed(seed)
session_conf = tf.compat.v1.ConfigProto(intra_op_parallelism_threads=1, inter_op_parallelism_threads=1)
session_conf.gpu_options.allow_growth = True
sess = tf.compat.v1.Session(graph=tf.compat.v1.get_default_graph(), config=session_conf)
K.set_session(sess)
# # for later versions:
# session_conf = tf.compat.v1.ConfigProto(intra_op_parallelism_threads=1, inter_op_parallelism_threads=1)
# session_conf.gpu_options.allow_growth = True
# sess = tf.compat.v1.Session(graph=tf.compat.v1.get_default_graph(), config=session_conf)
# tf.compat.v1.keras.backend.set_session(sess)
random_seed = 0
np.random.seed(random_seed)
tf.random.set_seed(random_seed)
The post is a popular post from a few years ago, is there any changes that need to make? Or is there any other recent method that I can use to reproduce the result in Keras and Tensorflow?
I am new to python and ml and was just trying to work around with mnist
my program looks like this
from sklearn.datasets import fetch_openml
import matplotlib as mpl
import matplotlib.pyplot as plt
from sklearn.linear_model import SGDClassifier
import numpy as np
mist=fetch_openml('mnist_784',version=1)
mist.keys()
x,y=mist['data'],mist['target']
x.shape
images = x[1]
images=images.reshape(28,28)
plt.imshow(images)
plt.show()
y=y.astype(np.uint8)
y[0]
xtrain,xtest,ytrain,ytest=x[:60000],x[60000:],y[:60000],y[60000:]
ytrain_5=(ytrain==5)
ytest_5=(ytest==5)
sgd = SGDClassifier(random_state=42)
sgd.fit(xtrain,ytrain_5)
sgd.predict([images])
And is throwing and error:
Found array with dim 3. Estimator expected <= 2.
Use this line:
sgd.predict(images.reshape(1, 784))
The algorithm was trained with a flattened array of shape (70000, 784), so you need to flatten images to shape (1, 784) before you pass it through. You had previously reshaped it to (28, 28).
Full code:
from sklearn.datasets import fetch_openml
import matplotlib as mpl
import matplotlib.pyplot as plt
from sklearn.linear_model import SGDClassifier
import numpy as np
mist=fetch_openml('mnist_784',version=1)
x,y=mist['data'],mist['target']
images = x[1]
images=images.reshape(28,28)
y=y.astype(np.uint8)
xtrain,xtest,ytrain,ytest=x[:60000],x[60000:],y[:60000],y[60000:]
ytrain_5=(ytrain==5)
ytest_5=(ytest==5)
sgd = SGDClassifier(random_state=42)
sgd.fit(xtrain,ytrain_5)
sgd.predict(images.reshape(1, 784))
I created a multiple input one output LSTM that estimated the total price with a dataset of daily room rates for a hotel by month, but the model I created doesn't work well. Below I shared the model's code and the link to the data set.
data = pd.read_csv("/content/drive/My Drive/hotels.csv")
data
enter image description here
new_data = data.loc[:,['date','days','price','total']]
new_data.info()
date = new_data.date.values
dates = []
for i in date:
dates.append(i.split('/')[0])
new_data['date'] = dates
new_data
enter image description here
new_data = new_data.astype('float32')
new_data.info()
enter image description here
import pickle
filehandler = open(b"Hotels.obj","wb")
pickle.dump(new_data,filehandler)
file = open("/content/Hotels.obj",'rb')
object_file = pickle.load(file)
object_file
enter image description here
from math import sqrt
from matplotlib import pyplot
from pandas import read_csv
from pandas import DataFrame
from pandas import concat
from sklearn.preprocessing import MinMaxScaler
from sklearn.preprocessing import LabelEncoder
from sklearn.metrics import mean_squared_error
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import LSTM
from tensorflow.keras.layers import Concatenate
from sklearn.model_selection import train_test_split
from tensorflow.keras.layers import Add
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import RMSprop,Adam
from sklearn.preprocessing import MinMaxScaler
import tensorflow as tf
import datetime
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from packaging import version
print("TensorFlow version: ", tf.__version__)
assert version.parse(tf.__version__).release[0] >= 2, \
"This notebook requires TensorFlow 2.0 or above."
file = open('/content/Hotels.obj', 'rb')
scaler = MinMaxScaler(feature_range=(0, 1))
train_size = int(len(object_file) * 0.76)
test_size = len(object_file) - train_size
days = object_file["days"].values.reshape(-1,1)
price = object_file["price"].values.reshape(-1,1)
total = object_file["total"].values.reshape(-1,1)
date = object_file["date"].values.reshape(-1,1)
days_ = scaler.fit_transform(days)
total_ = scaler.fit_transform(total)
price_ = scaler.fit_transform(price)
date_ = scaler.fit_transform(date)
days_train = days_[0:train_size].reshape(train_size,1,1)
days_test = days_[train_size:len(days_)].reshape(test_size,1,1)
date_train = date_[0:train_size].reshape(train_size,1,1)
date_test = date_[train_size:len(days_)].reshape(test_size,1,1)
price_train = price_[0:train_size].reshape(train_size,1,1)
price_test = price_[train_size:len(price_)].reshape(test_size,1,1)
total_train = total_[0:train_size].reshape(train_size,1)
total_test = total_[train_size:len(total_)].reshape(test_size,1)
def buildModel(dataLength,labelLength):
date = tf.keras.Input(shape=(1,1),name='date')
days = tf.keras.Input(shape=(1,1),name='days')
price = tf.keras.Input(shape=(1,1),name='price')
dateLayers = LSTM(100,return_sequences=False)(date)
daysLayers = LSTM(100,return_sequences=False)(days)
priceLayers = LSTM(100,return_sequences=False)(price)
output = tf.keras.layers.concatenate(inputs=[dateLayers,daysLayers, priceLayers],axis=1)
output = Dense(labelLength,activation='relu',name='weightedAverage_output_3')(output)
model = Model(inputs=[date,days,price],outputs=[output])
optimizer = Adam(lr=0.001, beta_1=0.9, beta_2=0.999)
model.compile(optimizer=optimizer,loss='mse',metrics=['accuracy'])
return model
object_file = pickle.load(file)
logdir="logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=logdir)
rnn = buildModel(train_size,1)
rnn.fit([date_train,days_train,price_train],
[total_train],
validation_data = ([date_test,days_test,price_test],[total_test]),
epochs = 1,
batch_size = 10,
callbacks=[tensorboard_callback]
)
result = rnn.predict([date_test,days_test,price_test])
scaler.inverse_transform(result)
enter image description here
When I increase the number of epoch, the model is being overfit.I can't get the result I want.How can I do this?
Data set link : https://www.kaggle.com/leomauro/argodatathon2019#hotels.csv
Your results are poor because your metrics is accuracy. If I understand correctly, you're predicting a continuous variable — you're not classifying. So, it makes no sense to look at accuracy.
Metrics should be mae for mean absolute error. I think you'll be satisfied with your model performance then.
Re-scaling your target makes no sense here. It's the inner workings of the neural network that prefer an input between 0 and 1.
I have the below code.
I would like to see how the weights and bias changes during training.
Ideally I would like to see it in tensorboard.
Would someone be able to show me how to do this.
from time import time
import numpy as np
import matplotlib.pyplot as plt
import keras
import tensorflow as tf
from keras.callbacks import TensorBoard
from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
x = scaler.fit_transform(np.array([[1965.0], [1980.0]])).reshape(-1,1)
y = scaler.fit_transform(np.array([[320.0], [345.0]])).reshape(-1,1)
tensorboard = TensorBoard(log_dir='logs/{}'.format(time()), write_grads=True)
model = keras.Sequential([keras.layers.Dense(1, activation='linear')])
model.compile(optimizer='sgd',
loss="mean_squared_error")
model.fit(x=x, y=y, epochs=1000, callbacks=[tensorboard])
yHat = model.predict(x)
Based on the Keras documentation, all you need to do maybe is just run the command line:
tensorboard --logdir=logs
Notice that the logdir setting is pointing to the root of your log directory.