Develop Reference

Can't flatten output from Keras model

I have the below model built with Keras, and I am training it using StratifiedKFold. Training works nice, performance is good. Now I am trying to explain the model predictions using the SHAP library. My dateset shape is (107012, 67) and the below is the the code I wrote that encodes my data, trains and makes predictions. original_X is the variable I am reading my data in using Pandas. Most of my data is categorical and only one column contains continuous values.
ohe = OneHotEncoder()
mms = MinMaxScaler()
ct = make_column_transformer(
(ohe, categorical_columns_encode),
(mms, numerical_columns_encode),
remainder='passthrough')
ct.fit(original_X.astype(str))
X = ct.transform(original_X.astype(str))
print(X.shape) # Shape of the encoded value (107012, 47726)
recall = Recall(name="recall")
prec = Precision(name="precision")
ba = BinaryAccuracy()
def get_model():
network = Sequential()
network.add(Input(shape=X_1.shape))
network.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))
network.add(Dropout(0.5))
network.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))
network.add(Dropout(0.5))
network.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))
# network.add(Flatten())
network.add(Dense(1, activation='sigmoid'))
network.compile(loss='binary_crossentropy',
optimizer=Adam(learning_rate=0.001),
metrics=[recall, prec, ba])
return network
classifier = KerasClassifier(build_fn=get_model)
kfold = RepeatedStratifiedKFold(n_splits=3, n_repeats=3, random_state=42)
callback = EarlyStopping(
monitor='val_recall',
min_delta=0,
patience=0,
verbose=1,
mode="auto",
baseline=None,
restore_best_weights=True
)
epochs_per_fold = []
for train, validation in kfold.split(X_1, y_1):
X_train, X_validation = X_1[train], X_1[validation]
y_train, y_validation = y_1[train], y_1[validation]
# Printing the distribution of classes in the training set
counter = Counter(y_train)
print("Number of class distributions of the training set ", counter)
print("Minority case percentage of the training set ", counter[1] / (counter[0] + counter[1]))
# Training our model and saving the history of the training
history = classifier.fit(
x=X_train,
y=y_train,
verbose=1,
epochs=30,
shuffle=True,
callbacks=[callback],
class_weight={0: 1.0, 1: 3.0},
validation_data=(X_validation, y_validation))
# predict classes for our validation set in order to manually verify the metrics
yhat_classes = (classifier.predict(X_validation) > 0.5).astype("int32")
TP = 0
FP = 0
TN = 0
FN = 0
# Record our preditions for the confusion matrix for manually verifying our metrics
for p,t in zip(y_validation, yhat_classes):
if p == 1 and t == 1:
TP += 1
elif p == 0 and t == 1:
FP += 1
elif p == 1 and t == 0:
FN += 1
elif p == 0 and t == 0:
TN += 1
print("\n")
print(" "*16, "T F")
print("Positive result ", TP, FP, )
print("Negative result ", TN, FN, )
print("\n")
# Printing the built in classification report of our model
print(classification_report(y_validation, yhat_classes))
report_dict = classification_report(y_validation, yhat_classes, output_dict=True)
# Record the average number of epochs of training
epochs_per_fold.append(len(history.history['recall']))
print(yhat_classes)
Here I am trying to use DeepExplainer from the Shap libeary to look inside my predictions.
# we use the first 100 training examples as our background dataset to integrate over
background = X_2[np.random.choice(X_2.shape[0], 100, replace=False)]
explainer = shap.DeepExplainer(get_model(), background)
When the code reaches the explainer declaration the below error is thrown.
Your TensorFlow version is newer than 2.4.0 and so graph support has been removed in eager mode. See PR #1483 for discussion.
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-113-d24b2d1e3b91> in <module>()
----> 1 explainer = shap.DeepExplainer(get_model(), background)
1 frames
/usr/local/lib/python3.7/dist-packages/shap/explainers/_deep/deep_tf.py in __init__(self, model, data, session, learning_phase_flags)
100 self.model_output = _get_model_output(model)
101 assert type(self.model_output) != list, "The model output to be explained must be a single tensor!"
--> 102 assert len(self.model_output.shape) < 3, "The model output must be a vector or a single value!"
103 self.multi_output = True
104 if len(self.model_output.shape) == 1:
AssertionError: The model output must be a vector or a single value!
My questions are:
How can I flatten the output of my model from within the get_model function?
Is there a better approach to explaining my predictions with Shap?
Let me know if I need to share any extra information on this.

Adding a Flatten layer after a Dense layer is causing the error. Observe, that the line which causes the error is,
assert len(self.model_output.shape) < 3, "The model output must be a vector or a single value!"
Considering a 2D input, the output of a Dense layer is ( None , units ). So, if we have a Dense( 32 ) layer and the batch size is set to 16, then the output of such a layer will be a tensor of shape ( 16 , 32 ). The Flatten layer preserves the 0th axis ( i.e. the batch dimension ), and hence a tensor of shape ( 16 , 32 ) could be flattened further.
On the other hand, if you had a tensor of shape ( 16 , 32 , 3 ) ( for ex. output of a Conv2D layer with 3 filters ) then the output of the Flatten layer will be a tensor of shape ( 16 , 96 ).
Since you have 2D input, just remove the Flatten layer. If you were
trying to reshape the output, use a Reshape layer instead.

ValueError: No gradients provided for any variable when using model.fit

I'm trying to use the features extracted from two pre-trained models (resnet and mobilenet) as inputs to train a functional model using Keras. I need to classify images as categories 1,2 or 3 using a softmax layer.
My model.fit function is giving me the following error:
ValueError: No gradients provided for any variable: ['dense_66/kernel:0', 'dense_66/bias:0',
'dense_64/kernel:0', 'dense_64/bias:0', 'dense_67/kernel:0', 'dense_67/bias:0',
'dense_65/kernel:0', 'dense_65/bias:0', 'dense_68/kernel:0', 'dense_68/bias:0',
'dense_69/kernel:0', 'dense_69/bias:0', 'dense_70/kernel:0', 'dense_70/bias:0'].
Here's the relevant part of code:
Creating the dataset
def datasetgenerator(url,BATCH_SIZE,IMG_SIZE):
data=image_dataset_from_directory(url,
shuffle=True,
batch_size=BATCH_SIZE,
image_size=IMG_SIZE,
label_mode='int'
)
return data
BATCH_SIZE = 20
IMG_SIZE = (160, 160)
train_dir='wound_dataset2/train'
train_dataset = datasetgenerator(url=train_dir,BATCH_SIZE=BATCH_SIZE,IMG_SIZE= IMG_SIZE)
val_dir='wound_dataset2/val'
validation_dataset = datasetgenerator(url=val_dir,BATCH_SIZE=BATCH_SIZE,IMG_SIZE= IMG_SIZE)
test_dir='wound_dataset2/test'
test_dataset = datasetgenerator(url=test_dir,BATCH_SIZE=BATCH_SIZE,IMG_SIZE= IMG_SIZE)
print(train_dataset)
Feature extraction
mobilenet_features = np.empty([20, 1280])
resnet_features = np.empty([20, 2048])
for data in train_dataset:
image_batch, label_batch = data
image_batch = data_augmentation(image_batch)
preprocess_input_image_resnet = preprocess_input_resnet(image_batch)
preprocess_input_image_mobilenet = preprocess_input_mobilenet(image_batch)
feature_batch_resnet = base_model_resnet(preprocess_input_image_resnet)
feature_batch_average_resnet = global_average_layer(feature_batch_resnet)
feature_batch_mobilenet = base_model_mobilenet(preprocess_input_image_mobilenet)
feature_batch_average_mobilenet = global_average_layer(feature_batch_mobilenet)
mobilenet_features = np.concatenate((mobilenet_features, np.array(feature_batch_average_mobilenet)))
resnet_features = np.concatenate((resnet_features, np.array(feature_batch_average_resnet)))
Model Generation
from tensorflow.keras.layers import concatenate
# define two sets of inputs
inputA = tf.keras.Input(shape=(1280,))
inputB = tf.keras.Input(shape=(2048,))
# the first branch operates on the first input
x = tf.keras.layers.Dense(8, activation="relu")(inputA)
x = tf.keras.layers.Dense(4, activation="relu")(x)
x = tf.keras.Model(inputs=inputA, outputs=x)
# the second branch opreates on the second input
y = tf.keras.layers.Dense(64, activation="relu")(inputB)
y = tf.keras.layers.Dense(32, activation="relu")(y)
y = tf.keras.layers.Dense(4, activation="relu")(y)
y = tf.keras.Model(inputs=inputB, outputs=y)
# combine the output of the two branches
combined = concatenate([x.output, y.output])
fc_layers = [1024, 1024]
dropout = 0.5
# apply a FC layer and then a regression prediction on the
# combined outputs
z = Flatten()(combined)
for fc in fc_layers:
# New FC layer, random init
z = Dense(fc, activation='relu')(z)
z = Dropout(dropout)(z)
# New softmax layer
predictions = Dense(3, activation='softmax')(z)
# our model will accept the inputs of the two branches and
# then output a single value
model = tf.keras.Model(inputs=[x.input, y.input], outputs=z)
Training
model.compile(optimizer=tf.keras.optimizers.Adam(1e-3),
loss= tf.keras.losses.CategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
history = model.fit((mobilenet_features, resnet_features), batch_size=20, epochs=10)
I'm trying this as a method to improve accuracy over what I got using transfer learning. Any help would be appreciated.

z = Flatten()(combined)
z = Dense(fc, activation='relu')(z)
z = Dropout(dropout)(z)
z = Dense(fc, activation='relu')(z)
z = Dropout(dropout)(z)
predictions = Dense(3, activation='softmax')(z)
# use the prediction as output layer
model = tf.keras.Model(inputs=[x.input, y.input], outputs=predictions)
#add target tensor to the fit method
history = model.fit((mobilenet_features, resnet_features),youTarget, batch_size=20, epochs=10)

Keras LSTM Input/Output Dimension

I am constructing an LSTM predictor with Keras. My input array is historical price data. I segment the data into window_size blocks, in order to predict prediction length blocks ahead. My data is a list of 4246 floating point numbers. I seperate my data into 4055 arrays each of length 168 in order to predict 24 units ahead.
This gives me an x_train set with dimension (4055,168). I then scale my data and try to fit the data but run into a dimension error.
df = pd.DataFrame(data)
print(f"Len of df: {len(df)}")
min_max_scaler = MinMaxScaler()
H = 24
window_size = 7*H
num_pred_blocks = len(df)-window_size-H+1
x_train = []
y_train = []
for i in range(num_pred_blocks):
x_train_block = df['C'][i:(i + window_size)]
x_train.append(x_train_block)
y_train_block = df['C'][(i + window_size):(i + window_size + H)]
y_train.append(y_train_block)
LEN = int(len(x_train)*window_size)
x_train = min_max_scaler.fit_transform(x_train)
batch_size = 1
def build_model():
model = Sequential()
model.add(LSTM(input_shape=(window_size,batch_size),
return_sequences=True,
units=num_pred_blocks))
model.add(TimeDistributed(Dense(H)))
model.add(Activation("linear"))
model.compile(loss="mse", optimizer="rmsprop")
return model
num_epochs = epochs
model= build_model()
model.fit(x_train, y_train, batch_size = batch_size, epochs = 50)
The error being returned is as such.
ValueError: Error when checking model target: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 1 array(s), but instead got the following list of 4055 arrays: [array([[0.00630006],
Am I not segmenting correctly? Loading correctly? Should the number of units be different than the number of prediction blocks? I appreciate any help. Thanks.
Edit
The suggestions to convert them to Numpy arrays is correct but MinMixScalar() returns a numpy array. I reshaped the arrays into the proper dimension but now my computer is having CUDA memory error. I consider the problem solved. Thank you.
df = pd.DataFrame(data)
min_max_scaler = MinMaxScaler()
H = prediction_length
window_size = 7*H
num_pred_blocks = len(df)-window_size-H+1
x_train = []
y_train = []
for i in range(num_pred_blocks):
x_train_block = df['C'][i:(i + window_size)].values
x_train.append(x_train_block)
y_train_block = df['C'][(i + window_size):(i + window_size + H)].values
y_train.append(y_train_block)
x_train = min_max_scaler.fit_transform(x_train)
y_train = min_max_scaler.fit_transform(y_train)
x_train = np.reshape(x_train, (len(x_train), 1, window_size))
y_train = np.reshape(y_train, (len(y_train), 1, H))
batch_size = 1
def build_model():
model = Sequential()
model.add(LSTM(batch_input_shape=(batch_size, 1, window_size),
return_sequences=True,
units=100))
model.add(TimeDistributed(Dense(H)))
model.add(Activation("linear"))
model.compile(loss="mse", optimizer="rmsprop")
return model
num_epochs = epochs
model = build_model()
model.fit(x_train, y_train, batch_size = batch_size, epochs = 50)

I don't think you passed the batch size in the model.
input_shape=(window_size,batch_size) is the data dimension. which is correct, but you should use input_shape=(window_size, 1)
If you want to use batch, you have to add another dimension, like this LSTM(n_neurons, batch_input_shape=(n_batch, X.shape[1], X.shape[2])) (Cited from the Keras)
in your case:
def build_model():
model = Sequential()
model.add(LSTM(input_shape=(batch_size, 1, window_size),
return_sequences=True,
units=num_pred_blocks))
model.add(TimeDistributed(Dense(H)))
model.add(Activation("linear"))
model.compile(loss="mse", optimizer="rmsprop")
return model
You also need to use np.shape to change the dimension of the of your data, it should be (batch_dim, data_dim_1, data_dim_2). I use numpy, so numpy.reshape() will work.
First your data should be row-wise, so for each row, you should have a shape of (1, 168), then add the batch dimension, it will be (batch_n, 1, 168).
Hope this help.

That's probably because x_train and y_train were not updated to numpy arrays. Take a closer look at this issue on github.
model = build_model()
x_train, y_train = np.array(x_train), np.array(y_train)
model.fit(x_train, y_train, batch_size = batch_size, epochs = 50)

How to set up LSTM network for predict multi-sequence?

I am learning how to set up the RNN-LSTM network for prediction. I have created the dataset with one input variable.
x y
1 2.5
2 6
3 8.6
4 11.2
5 13.8
6 16.4
...
By the following python code, I have created the window data, like [x(t-2), x(t-1), x(t)] to predict [y(t)]:
df= pd.read_excel('dataset.xlsx')
# split a univariate dataset into train/test sets
def split_dataset(data):
train, test = data[:-328], data[-328:-6]
return train, test
train, test = split_dataset(df.values)
# scale train and test data
def scale(train, test):
# fit scaler
scaler = MinMaxScaler(feature_range=(0,1))
scaler = scaler.fit(train)
# transform train
#train = train.reshape(train.shape[0], train.shape[1])
train_scaled = scaler.transform(train)
# transform test
#test = test.reshape(test.shape[0], test.shape[1])
test_scaled = scaler.transform(test)
return scaler, train_scaled, test_scaled
scaler, train_scaled, test_scaled = scale(train, test)
def to_supervised(train, n_input, n_out=7):
# flatten data
data = train
X, y = list(), list()
in_start = 0
# step over the entire history one time step at a time
for _ in range(len(data)):
# define the end of the input sequence
in_end = in_start + n_input
out_end = in_end + n_out
# ensure we have enough data for this instance
if out_end <= len(data):
x_input = data[in_start:in_end, 0]
x_input = x_input.reshape((len(x_input), 1))
X.append(x_input)
y.append(data[in_end:out_end, 0])
# move along one time step
in_start += 1
return np.array(X), np.array(y)
train_x, train_y = to_supervised(train_scaled, n_input = 3, n_out = 1)
test_x, test_y = to_supervised(test_scaled, n_input = 3, n_out = 1)
verbose, epochs, batch_size = 0, 20, 16
n_timesteps, n_features, n_outputs = train_x.shape[1], train_x.shape[2], train_y.shape[1]
model = Sequential()
model.add(LSTM(200, return_sequences= False, input_shape = (train_x.shape[1],train_x.shape[2])))
model.add(Dense(1))
model.compile(loss = 'mse', optimizer = 'adam')
history = model.fit(train_x, train_y, epochs=epochs, verbose=verbose, validation_data = (test_x, test_y))
However, I have other questions about this:
Q1: What is the meaning of units in LSTM? [model.add(LSTM(units, ...))]
(I have tried different units for the model, it would be more accurate as units increased.)
Q2: How many layers should I set?
Q3: How can I predict multi-steps ? e.g base on (x(t),x(t-1)) to predict y(t), y(t+1) I have tried to set the n_out = 2 in the to_supervised function, but when I applied the same method, it returned the error
train_x, train_y = to_supervised(train_scaled, n_input = 3, n_out = 2)
test_x, test_y = to_supervised(test_scaled, n_input = 3, n_out = 2)
verbose, epochs, batch_size = 0, 20, 16
n_timesteps, n_features, n_outputs = train_x.shape[1], train_x.shape[2], train_y.shape[1]
model = Sequential()
model.add(LSTM(200, return_sequences= False, input_shape = (train_x.shape[1],train_x.shape[2])))
model.add(Dense(1))
model.compile(loss = 'mse', optimizer = 'adam')
history = model.fit(train_x, train_y, epochs=epochs, verbose=verbose, validation_data = (test_x, test_y))
ValueError: Error when checking target: expected dense_27 to have shape (1,) but got array with shape (2,)
Q3(cont): What should I add or change in the model setting?
Q3(cont): What is the return_sequences ? When should I set True?

Q1. Units in LSTM is the number of neurons in your LSTM layer.
Q2. That depends on your model / data. Try changing them around to see the effect.
Q3. That depends which apporach you take.
Q4. Ideally you'll want to predict a single time step every time.
It is possible to predict several at a time, but in my experience you will get better results like as i have described below
e.g
use y(t-1), y(t) to predict y_hat(t+1)
THEN
use y(t), y_hat(t+1) to predict y_hat(t+2)
Are you sure you're actually using X to predict Y in this case?
how does train x/y and test x/y look like?

Re Q1: It is the number of LSTM cells (=LSTM units), which consist of several neurons themselves but have (in the standard case as given) only one output each. Thus, the number of units corresponds directly to the dimensionality of your output.

How to set up the LSTM network?

I am learning how to set up the RNN-LSTM network for prediction. I have created the dataset with one-variables.
x y
1 2.5
2 6
3 8.6
4 11.2
5 13.8
6 16.4
...
And the relationship of the y(t) = 2.5x(t) + x(t-1) -0.9*x(t-2). And I am trying to set up the RNN-LSTM to learn the pattern, but it occurred the error of my program. My program is like below:
df= pd.read_excel('dataset.xlsx')
def split_dataset(data):
# split into standard weeks
train, test = data[:-328], data[-328:-6]
# restructure into windows of weekly data
train = np.array(np.split(train, len(train)/1))
test = np.array(np.split(test, len(test)/1))
return train, test
verbose, epochs, batch_size = 0, 20, 16
train, test = split_dataset(df.values)
train_x, train_y = train[:,:,0], train[:,:,1]
model = Sequential()
model.add(LSTM(200, return_sequences=True, input_shape = train_x.shape))
model.compile(loss='mse', optimizer='adam')
It occurred the ValueError:
ValueError: Error when checking input: expected lstm_35_input to have 3 dimensions, but got array with shape (8766, 1)
Any experienced DS or pythoner can teach me how to set up the network?
Thanks

For LSTM based RNN, the input should be of 3 dimensions (batch, time, data_point). I assume index of your x variable is its time. In this case, you have to transform your input into batches of some window, say a window of 3, then your input is:
batch # input target
0 x[0:3] y[3]
1 x[1:4] y[4]
2 x[2:5] y[5]
Note: your y's start from t=3 since you are using last 3 time steps to predict the next 4th value. If your y's are already calculated from the last three time steps as you have said, then y's should start from 0 index, i.e. at batch 0 you have y[0] as the target
UPDATE as per below comment
If you want to have multiple sequences, then you can model it as a sequence to sequence problem and will be an N to M mapping, you need five x values to predict three y's:
batch # input target
0 x[0:5] y[3:6]
1 x[1:6] y[4:7]
2 x[2:7] y[5:8]

In current, I have created the data window and it look like work for I mentioned case.
Below is my code:
df= pd.read_excel('dataset.xlsx')
# split a univariate dataset into train/test sets
def split_dataset(data):
train, test = data[:-328], data[-328:-6]
return train, test
train, test = split_dataset(df.values)
# scale train and test data to [-1, 1]
def scale(train, test):
# fit scaler
scaler = MinMaxScaler(feature_range=(0,1))
scaler = scaler.fit(train)
# transform train
#train = train.reshape(train.shape[0], train.shape[1])
train_scaled = scaler.transform(train)
# transform test
#test = test.reshape(test.shape[0], test.shape[1])
test_scaled = scaler.transform(test)
return scaler, train_scaled, test_scaled
scaler, train_scaled, test_scaled = scale(train, test)
def to_supervised(train, n_input, n_out=7):
# flatten data
data = train
X, y = list(), list()
in_start = 0
# step over the entire history one time step at a time
for _ in range(len(data)):
# define the end of the input sequence
in_end = in_start + n_input
out_end = in_end + n_out
# ensure we have enough data for this instance
if out_end <= len(data):
x_input = data[in_start:in_end, 0]
x_input = x_input.reshape((len(x_input), 1))
X.append(x_input)
y.append(data[in_end:out_end, 0])
# move along one time step
in_start += 1
return np.array(X), np.array(y)
train_x, train_y = to_supervised(train_scaled, n_input = 3, n_out = 1)
test_x, test_y = to_supervised(test_scaled, n_input = 3, n_out = 1)
verbose, epochs, batch_size = 0, 20, 16
n_timesteps, n_features, n_outputs = train_x.shape[1], train_x.shape[2], train_y.shape[1]
model = Sequential()
model.add(LSTM(200, return_sequences= False, input_shape = (train_x.shape[1],train_x.shape[2])))
model.add(Dense(1))
model.compile(loss = 'mse', optimizer = 'adam')
history = model.fit(train_x, train_y, epochs=epochs, verbose=verbose, validation_data = (test_x, test_y))
However, I have other questions about this:
Q1: What is the meaning of units in LSTM? [model.add(LSTM(units, ...))]
(I have tried different units for the model, it would be more accurate as units increased.)
Q2: How many layers should I set?
Q3: How can I predict multi-steps ? e.g base on (x(t),x(t-1)) to predict y(t), y(t+1)
I have tried to set the n_out = 2 in the to_supervised function, but when I applied the same method, it returned the error
train_x, train_y = to_supervised(train_scaled, n_input = 3, n_out = 2)
test_x, test_y = to_supervised(test_scaled, n_input = 3, n_out = 2)
verbose, epochs, batch_size = 0, 20, 16
n_timesteps, n_features, n_outputs = train_x.shape[1], train_x.shape[2], train_y.shape[1]
model = Sequential()
model.add(LSTM(200, return_sequences= False, input_shape = (train_x.shape[1],train_x.shape[2])))
model.add(Dense(1))
model.compile(loss = 'mse', optimizer = 'adam')
history = model.fit(train_x, train_y, epochs=epochs, verbose=verbose, validation_data = (test_x, test_y))
ValueError: Error when checking target: expected dense_27 to have shape (1,) but got array with shape (2,)
Q3(cont): What should I add or change in the model setting?
Q3(cont): What is the return_sequences ? When should I set True?