Develop Reference

ValueError: 'logits' and 'labels' must have the same shape

I am working on my first neural network, and i'm stuck on one error. Here is the code:
import pandas as pd
from sklearn.model_selection import train_test_split
df = pd.read_csv('iris.csv')
X = pd.get_dummies(df.drop(['variety'], axis=1))
y = df['variety'].apply(lambda x: 0 if x=='Setosa' else (1 if x=='Versicolor' else 2))
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.2)
print(y_train.head())
from keras.models import Sequential, load_model
from keras.layers import Dense
from sklearn.metrics import accuracy_score
model = Sequential()
model.add(Dense(units=8, activation='relu', input_dim=len(X_train.columns)))
model.add(Dense(units=3, activation='sigmoid'))
model.add(flatten())
model.compile(loss='binary_crossentropy', optimizer='sgd', metrics='accuracy')
model.fit(X_train, y_train, epochs=50, batch_size=1)
I am working off of a tutorial on tensorflow, and am using https://www.kaggle.com/datasets/arshid/iris-flower-dataset as the dataset to train on. I used the code from the tutorial, but changed it to fit my dataset. Still, I get the ValueError. Any help?

Why predict works without fit the model in Keras

Check the following code:
import numpy as np
import keras
from keras.models import Sequential
from keras.layers import Conv1D, MaxPooling1D, Flatten
from sklearn.model_selection import train_test_split
# Data
X = np.random.rand(1000, 100, 1)
y = np.random.randint(0, 2, (1000, 1))
# Splitting into train and test
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
# Conv1D
model = Sequential()
model.add(Conv1D(32, kernel_size=3, activation='relu', input_shape=(100, 1)))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
# Predict before fitting the model
cnn_features_train = model.predict(X_train)
cnn_features_test = model.predict(X_test)
Why this runs without throwing an error? The weights are not yet stabilished by the .fit method, how can it predict something?
If i try to do the same thing (predict before fitting the model) using Sklearn i get the expected error, for example:
from sklearn.ensemble import RandomForestClassifier
# Data
X = np.random.rand(1000, 100, 1)
y = np.random.randint(0, 2, (1000, 1))
# Splitting into train and test
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
# Random Forest
rf = RandomForestClassifier()
rf.predict(X_test)
The error:
sklearn.exceptions.NotFittedError: This RandomForestClassifier instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator.

Keras is different from sklearn. The .predict ()without calling .fit() helps users in preparing and debugging the correct shapes of the tensor.

Why do loss graphs disappear once some metrics are added to a model?

In the course of assessing a trained model synthesized for the regression problem below, I have some confusion in plotting the resulting history. In particular, when I don't consider any metrics
import pandas as pd
import matplotlib.pyplot as plt
import tensorflow as tf
from sklearn.datasets import fetch_california_housing
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
housing = fetch_california_housing()
X_train_full, X_test, y_train_full, y_test = train_test_split(
housing.data, housing.target)
X_train, X_valid, y_train, y_valid = train_test_split(
X_train_full, y_train_full)
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_valid = scaler.fit_transform(X_valid)
X_test = scaler.fit_transform(X_test)
model = tf.keras.Sequential([
tf.keras.layers.Dense(30, tf.keras.activations.relu, input_shape=X_train.shape[1:]),
tf.keras.layers.Dense(1)
])
model.compile(loss=tf.keras.losses.mean_squared_error,
optimizer=tf.keras.optimizers.SGD())
history = model.fit(X_train, y_train, epochs=20,
validation_data=(X_valid, y_valid))
pd.DataFrame(history.history).plot()
plt.grid(True)
plt.show()
the final plot includes loss and val_loss graphs as expected.
But once I add a metrics to my model, say, tf.keras.metrics.MeanSquaredError(), the resulting plot generated by
import pandas as pd
import matplotlib.pyplot as plt
import tensorflow as tf
from sklearn.datasets import fetch_california_housing
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
housing = fetch_california_housing()
X_train_full, X_test, y_train_full, y_test = train_test_split(
housing.data, housing.target)
X_train, X_valid, y_train, y_valid = train_test_split(
X_train_full, y_train_full)
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_valid = scaler.fit_transform(X_valid)
X_test = scaler.fit_transform(X_test)
model = tf.keras.Sequential([
tf.keras.layers.Dense(30, tf.keras.activations.relu, input_shape=X_train.shape[1:]),
tf.keras.layers.Dense(1)
])
model.compile(loss=tf.keras.losses.mean_squared_error,
optimizer=tf.keras.optimizers.SGD(),
metrics=[tf.keras.metrics.MeanSquaredError()])
history = model.fit(X_train, y_train, epochs=20,
validation_data=(X_valid, y_valid))
pd.DataFrame(history.history).plot()
plt.grid(True)
plt.show()
lacks loss and val_loss sketches.
What's the problem here?
Edit:
Here is the content of history.history:
{'loss': [0.880902886390686, 0.6208109855651855, 0.5102624297142029, 0.47074252367019653, 0.4556053578853607, 0.4464321732521057, 0.44210636615753174, 0.43378400802612305, 0.42544370889663696, 0.428415447473526], 'mean_squared_error': [0.880902886390686, 0.6208109855651855, 0.5102624297142029, 0.47074252367019653, 0.4556053578853607, 0.4464321732521057, 0.44210636615753174, 0.43378400802612305, 0.42544370889663696, 0.428415447473526], 'val_loss': [0.6332216262817383, 0.514700710773468, 0.4509757459163666, 0.46695834398269653, 0.5228265523910522, 0.6748611330986023, 0.6648175716400146, 0.7329052090644836, 0.8352308869361877, 1.081600546836853], 'val_mean_squared_error': [0.6332216262817383, 0.514700710773468, 0.4509757459163666, 0.46695834398269653, 0.5228265523910522, 0.6748611330986023, 0.6648175716400146, 0.7329052090644836, 0.8352308869361877, 1.081600546836853]}

Your loss is the mean squared error and your metric is the mean squared error, which is exactly the same. It means they are overlapping when you plot them !

Why the neural network is not learning?

I am training a neural network with a simple dataset. I have tried different combinations of parameters, optimizers, learning rates ... but even after 20 epochs the network is still not learning anything.
I wonder where in the following code lies the problem?
from tensorflow.keras.models import Sequential, load_model
from tensorflow.keras.layers import Input, Dense, Flatten
from tensorflow import keras
from livelossplot import PlotLossesKeras
from keras.models import Model
from sklearn.datasets import make_classification
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import pandas as pd
seed = 42
X, y = make_classification(n_samples=100000, n_features=2, n_redundant=0,
n_informative=2, random_state=seed)
print(f"Number of features: {X.shape[1]}")
print(f"Number of samples: {X.shape[0]}")
df = pd.DataFrame(np.concatenate((X,y.reshape(-1,1)), axis=1))
df.set_axis([*df.columns[:-1], 'Class'], axis=1, inplace=True)
df['Class'] = df['Class'].astype('int')
X = df.drop('Class', axis=1)
y = df['Class']
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)
print(f"Train set: {X_train.shape}")
print(f"Validation set: {X_val.shape}")
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train.astype(np.float64))
X_val_scaled = scaler.transform(X_val.astype(np.float64))
inputs = Input(shape=X_train_scaled.shape[1:])
h0 = Dense(5, activation='relu')(inputs)
h1 = Dense(5, activation='relu')(h0)
preds = Dense(1, activation = 'sigmoid')(h1)
model = Model(inputs=inputs, outputs=preds)
opt = keras.optimizers.Adam(lr=0.0001)
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
history = model.fit(X_train_scaled, y_train, batch_size=128, epochs=20, verbose=0,
validation_data=(X_val_scaled, y_val),
callbacks=[PlotLossesKeras()])
score_train = model.evaluate(X_train_scaled, y_train, verbose=0)
score_test = model.evaluate(X_val_scaled, y_val, verbose=0)
print('Train score:', score_train[0])
print('Train accuracy:', score_train[1])
print('Test score:', score_test[0])
print('Test accuracy:', score_test[1])
The code produces the following kind of output

You have used wrong loss function, change this line
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
to, for example,
model.compile(optimizer=opt, loss='mse', metrics=['accuracy'])
Categorical cross-entropy needs a one-hot encoded y which means, you have to have a 0 or a 1 for every class. MSE is just mean squared error, so it will work. But you might try some other losses as well.
your y:
[1,0,1]
one-hot encoded y:
[[0,1], [1,0], [0,1]]

ValueError: Error when checking input: expected dense_1_input to have shape (2198,) but got array with shape (1,)

I am trying to build a text classification model with cnn but I got this error. I have also tried to make input_dim=(2198,) but does not work also.
So, how can I determine input dimension right and is there any processing on text data like this ?Normalization for example.
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from keras.models import Sequential
from keras.layers import Dense
from keras.utils import to_categorical
import matplotlib.pyplot as plt
# read dataset
paths = {
'amazon': 'D:\\NLP\\sentiment labelled sentences\\amazon_cells_labelled.txt',
'imdb': 'D:\\NLP\\sentiment labelled sentences\\imdb_labelled.txt',
'yelp': 'D:\\NLP\\sentiment labelled sentences\\yelp_labelled.txt'
}
data_fame = []
for source, path in paths.items():
df = pd.read_csv(path, names=['sentence', 'label'], sep='\t')
df['source'] = source # Add another column filled with the source name amazon, imdb, yelp
data_fame.append(df)
df = pd.concat(data_fame)
# split data frame into features and labels
X, y = [], []
for i in range(len(df)):
X.append(df.iloc[i][0])
y.append(df.iloc[i][1])
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
x_train = np.array(x_train)
x_test = np.array(x_test)
y_train = np.array(y_train)
y_test = np.array(y_test)
y_train = to_categorical(y_train, 2)
y_test = to_categorical(y_test, 2)
x_train = np.expand_dims(x_train, axis=0)
x_test = np.expand_dims(x_test, axis=0)
# define CNN model
model = Sequential()
model.add(Dense(10, activation='relu', input_dim=x_train.shape[0]))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
history = model.fit(x_train, y_train, epochs=90, batch_size=16, validation_data=(x_test, y_test))
model.save('model.h5')