I am trying a deep neural network prediction but getting error:
InvalidArgumentError: logits and labels must have the same first dimension, got logits shape [32,4] and labels shape [128]
Here are the features:
new_features.shape
(19973, 8)
new_features[0].shape
(8,)
Here are the label/output
output.shape
(19973, 4)
output[0].shape
(4,)
Here is the keras code
model = Sequential(
[
Dense(units=8, input_shape=new_features[0].shape, name="layer1"),
Dense(units=1024, activation="relu", name="layer2"),
Dense(units=1024, activation="relu", name="layer3"),
Dense(units=4, name="layer4", activation="softmax"),
]
)
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
model.fit(new_features, output, epochs=2)
The features and labels contain float values.
The problem is in your target shape. First of all your target in classification problems must be int
if you have 1D integer encoded target you can use sparse_categorical_crossentropy as loss function
X = np.random.randint(0,10, (1000,100))
y = np.random.randint(0,3, 1000)
model = Sequential([
Dense(128, input_dim = 100),
Dense(3, activation='softmax'),
])
model.summary()
model.compile(loss='sparse_categorical_crossentropy',optimizer='adam',metrics=['accuracy'])
history = model.fit(X, y, epochs=3)
Otherwise, if you have one-hot encoded your target in order to have 2D shape (n_samples, n_class) you can use categorical_crossentropy
X = np.random.randint(0,10, (1000,100))
y = pd.get_dummies(np.random.randint(0,3, 1000)).values
model = Sequential([
Dense(128, input_dim = 100),
Dense(3, activation='softmax'),
])
model.summary()
model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy'])
history = model.fit(X, y, epochs=3)
Related
I am trying to create a Multitask NN using Tensorflow. Following is the architecture that I am trying to develop:
METRICS= [tf.keras.metrics.TruePositives(name='TP'),
tf.keras.metrics.FalsePositives(name='FP'),
tf.keras.metrics.TrueNegatives(name='TN'),
tf.keras.metrics.FalseNegatives(name='FN'),
tf.keras.metrics.Precision(name='precision'),
tf.keras.metrics.Recall(name='recall'),
tf.keras.metrics.AUC(curve='PR', name='PR-AUC')]
input_shape = (X_train.shape[1],)
inputlayer = tf.keras.layers.Input(shape=input_shape)
l1 = tf.keras.layers.Dense(input_shape[0]*2, activation= 'relu')(inputlayer)
l2 = tf.keras.layers.Dropout(0.1)(l1)
l3 = tf.keras.layers.Dense(int(input_shape[0]/2), activation='relu')(l2)
output1 = tf.keras.layers.Dense(1, activation='sigmoid', name = 'output1')(l3)
output2 = tf.keras.layers.Dense(10, activation='softmax', name = 'output2')(l3)
output3 = tf.keras.layers.Dense(12, activation='softmax', name = 'output3')(l3)
model = tf.keras.Model(inputs=inputlayer, outputs=[output1, output2, output3])
model.compile(loss={"output1": 'binary_crossentropy',
"output2": 'categorical_crossentropy',
"output3": 'categorical_crossentropy'},
optimizer=tf.keras.optimizers.Adam(learning_rate=.01),
metrics = METRICS, loss_weights = [1, 1e-1, 1e-1])
And this is the model architecture:
Then I tried to train the model like this:
BATCH_SIZE= 20
model.fit(X_train, [y1_train,y2_train,y3_train], batch_size=BATCH_SIZE, epochs=10, verbose=0)
But I got the following issue:
ValueError: Shapes (None, 1) and (None, 10) are incompatible
I already verified the labels of each output and they are respectively 2, 10 and 12
I couldn't understood what the problem is exactly, can anyone give me a suggestion please?
I think you might have mixed up the order of your labels. Here is a working example:
import tensorflow as tf
METRICS= [tf.keras.metrics.TruePositives(name='TP'),
tf.keras.metrics.FalsePositives(name='FP'),
tf.keras.metrics.TrueNegatives(name='TN'),
tf.keras.metrics.FalseNegatives(name='FN'),
tf.keras.metrics.Precision(name='precision'),
tf.keras.metrics.Recall(name='recall'),
tf.keras.metrics.AUC(curve='PR', name='PR-AUC')]
input_shape = (31,)
inputlayer = tf.keras.layers.Input(shape=input_shape)
l1 = tf.keras.layers.Dense(input_shape[0]*2, activation= 'relu')(inputlayer)
l2 = tf.keras.layers.Dropout(0.1)(l1)
l3 = tf.keras.layers.Dense(int(input_shape[0]/2), activation='relu')(l2)
output1 = tf.keras.layers.Dense(1, activation='sigmoid', name = 'output1')(l3)
output2 = tf.keras.layers.Dense(10, activation='softmax', name = 'output2')(l3)
output3 = tf.keras.layers.Dense(12, activation='softmax', name = 'output3')(l3)
model = tf.keras.Model(inputs=inputlayer, outputs=[output1, output2, output3])
model.compile(loss={"output1": 'binary_crossentropy',
"output2": 'categorical_crossentropy',
"output3": 'categorical_crossentropy'},
optimizer=tf.keras.optimizers.Adam(learning_rate=.01),
metrics = METRICS, loss_weights = [1, 1e-1, 1e-1])
y1_train, y2_train, y3_train = tf.random.uniform((50, 1), maxval=2), tf.random.uniform((50, 10), maxval=11), tf.random.uniform((50, 12), maxval=13)
model.fit(tf.random.normal((50, 31)), [y1_train,y2_train,y3_train], batch_size=20, epochs=10)
You need to make sure that y1_train, y2_train, and y3_train are in the correct order and have the correct shape, that is (samples, 1), (samples, 10), and (samples, 12).
I'm working on my own dataset and trying to fine-tune the parameters using the Keras tuner.I'm using the basic model-building function:
from tensorflow.keras import layers
from keras_tuner import RandomSearch
def build_model(hp):
model = keras.Sequential()
model.add(layers.Flatten())
model.add(
layers.Dense(
units=hp.Int("units", min_value=32, max_value=512, step=32),
activation="relu",
)
)
model.add(layers.Dense(10, activation="softmax"))
model.compile(
optimizer=keras.optimizers.Adam(
hp.Choice("learning_rate", values=[1e-2, 1e-3, 1e-4])
),
loss="categorical_crossentropy",
metrics=["accuracy"],
)
return model
After showing the search space summary and trying to search the best values from the following code statement:
tuner.search(X_train, y_train, epochs=15, validation_data=(X_test, y_test))
the following error was raised:
logits and labels must have the same first dimension, got logits shape
[32,10] and labels shape [6144]
I tried to change the loss to 'sparse_categorical_crossentropy' and 'categorical_crossentropy' but it didn't work.
My dataset shape is as follows:
x_test (201, 16)
y_test (201, 16, 12)
X_train (1803, 16)
y_train (1803, 16, 12)
x_val (500, 16)
y_val (500, 16, 12)
I am trying to add a SVM layer to my ResNet model but I am getting the error InvalidArgumentError: Incompatible shapes: [64,1] vs. [64,7,7,1] [[node hinge/mul (defined at <ipython-input-39-8ab0138abfe4>:17) ]] [Op:__inference_train_function_110660]
My code is as follows:
base_model = ResNet50(include_top=False, weights="imagenet", input_shape=(img_height, img_width,3 ))
x = base_model.output
x = Sequential()(x)
x = Dense(64, activation="relu")(x)
x = Dense(1, kernel_regularizer=l2(0.0001))(x)
x = tf.keras.layers.Activation("linear")(x)
model = Model(inputs = base_model.input, outputs = x)
model.compile(loss='hinge',
optimizer='adadelta',
metrics=['accuracy'])
history = model.fit(train_generator, validation_data=validation_generator, epochs=10)
The output of ResNet is all feature maps you get from image you need to add a flatten layer (or global pooling if you want) before passing it to the fully connected part for classfication
from tf.keras.layers import Flatten
base_model = ResNet50(include_top=False, weights="imagenet", input_shape=(img_height, img_width,3 ))
x = base_model.output
# Flatten layer
x = Flatten()(x)
x = Dense(64, activation="relu")(x)
x = Dense(1, kernel_regularizer=l2(0.0001))(x)
x = tf.keras.layers.Activation("linear")(x)
model = Model(inputs = base_model.input, outputs = x)
I have a pandas dataframe of features and samples, and a single series with binary category (0 or 1) values. With that I'm trying to train a neural network, but I am getting the error:
TensorFlow incompatible shapes binary classification
Here is a summary of the code:
X_train, X_test, y_train, y_test = train_test_split(df_x, series_y, random_state=1, test_size=0.25)
best_weight_path = 'best_weights.hdf5'
x = df_x.to_numpy()
y = series_y.to_numpy()
numpy_x_train = X_train.to_numpy()
numpy_y_train = y_train.to_numpy()
numpy_x_test = X_test.to_numpy()
numpy_y_test = y_test.to_numpy()
model = Sequential()
model.add(Dense(20, input_dim=x.shape[1], activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(2, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam')
monitor = EarlyStopping(monitor='val_loss', min_delta=1e-3, patience=5, verbose=1, mode='auto')
checkpointer = ModelCheckpoint(filepath=best_weight_path, verbose=0, save_best_only=True)
model.fit(x, y, validation_data=(numpy_x_test, numpy_y_test), callbacks=[monitor, checkpointer], verbose=0, epochs=1000)
ValueError: Shapes (None, 1) and (None, 2) are incompatible
Shouldn't the last dense layer have 2 units as there are two possible outcomes, so where is the shape (None, 1) coming from?
The problem is related to the correct choice of an appropriate loss function according to the format of your labels. you have 2 possibilities when using softmax in classification task:
1 possibility: if you have 1D integer encoded target, you can use sparse_categorical_crossentropy as loss function (this seems to be your case)
n_class = 2
n_features = 100
n_sample = 1000
X = np.random.randint(0,10, (n_sample,n_features))
y = np.random.randint(0,n_class, n_sample)
inp = Input((n_features,))
x = Dense(128, activation='relu')(inp)
out = Dense(n_class, activation='softmax')(x)
model = Model(inp, out)
model.compile(loss='sparse_categorical_crossentropy',optimizer='adam',metrics=['accuracy'])
history = model.fit(X, y, epochs=3)
2 possibility: if you have one-hot encoded your target in order to have 2D shape (n_samples, n_class), you can use categorical_crossentropy
n_class = 2
n_features = 100
n_sample = 1000
X = np.random.randint(0,10, (n_sample,n_features))
y = pd.get_dummies(np.random.randint(0,n_class, n_sample)).values
inp = Input((n_features,))
x = Dense(128, activation='relu')(inp)
out = Dense(n_class, activation='softmax')(x)
model = Model(inp, out)
model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy'])
history = model.fit(X, y, epochs=3)
I'm trying to build a 1D CNN model and can't seem to crack the data shape problem after trying a lot of ways to go about it.
Here is my code:
scaler = StandardScaler()
x_scaled = scaler.fit_transform(data)
# data.shape = (16611, 6001)
trainX, testX, trainy, testy = train_test_split(x_scaled, target, test_size=0.3)
# converting to 3D for input
dtrainX, dtestX, dtrainy, dtesty = dstack(trainX), dstack(testX), dstack(trainy), dstack(testy)
# dtrainX.shape = (1, 6001, 11627)
# dtrainy.shape = (1, 1, 11627)
verbose, epochs, batch_size = 0, 10, 32
n_timesteps, n_features, n_outputs = dtrainX.shape[1], dtrainX.shape[2], dtrainy.shape[0]
Ntrainy = np.array(dtrainy)
Ntrainy = np.squeeze(Ntrainy, axis=1)
# Ntrainy.shape = (1,11627)
model = Sequential()
model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(n_timesteps,n_features)))
model.add(Conv1D(filters=64, kernel_size=3, activation='relu'))
model.add(Dropout(0.5))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(100, activation='relu'))
model.add(Dense(n_outputs, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(dtrainX, Ntrainy, epochs=epochs, batch_size=batch_size, verbose=verbose)
Throws me this error:
Error when checking target: expected dense_2 to have shape (1,) but got array with shape (11627,).
I don't understand what I'm doing wrong any help would be great!!
The actual shape is of your data is:
X = (outputs, n_timesteps, n_features)
Y = (1, 1, n_features)
To make it work, dtrainX and Ntrainy shape should be reshaped to:
X = (n_features, n_timesteps, outputs)
Y = (n_features, 1, 1)
You could do that to both dtrainX and Ntrainy to make the model work:
# X
dtrainX = np.transpose(dtrainX, (2,1,0))
# Y
Ntrainy = np.transpose(Ntrainy, (1,0))