Print results more meaningfully in Python code? - python

I'm performing SVM on a dataset with 10 folds for different values of C and gamma
from sklearn.datasets import load_digits, load_iris, load_breast_cancer, load_wine
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.utils import shuffle
from sklearn import preprocessing
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import StratifiedKFold
from sklearn.metrics import accuracy_score, zero_one_loss, confusion_matrix
import pandas as pd
import numpy as np
z = pd.read_csv('/home/user/iris.csv', header=0)
X = z.iloc[:, :-1]
y = z.iloc[:, -1:]
X = np.array(X)
y = np.array(y)
# Performing standard scaling
scaler = preprocessing.MinMaxScaler()
X_scaled = scaler.fit_transform(X)
c = [0.1, 0.5]
gamma_values = [1e-1, 1]
for z in c:
for v in gamma_values:
# Defining the SVM with 'rbf' kernel
svc = SVC(kernel='rbf',C=z, gamma=v, random_state=50)
skf = StratifiedKFold(n_splits=10, shuffle=True)
acc_score = []
#skf.get_n_splits(X, y)
for train_index, test_index in skf.split(X, y):
X_train, X_test = X_scaled[train_index], X_scaled[test_index]
y_train, y_test = y[train_index], y[test_index]
# Training the model
svc.fit(X_train, np.ravel(y_train))
# Prediction on test dataste
y_pred = svc.predict(X_test)
# Obtaining the accuracy scores of the model
score = accuracy_score(y_test, y_pred)
acc_score.append(score)
print(np.array(acc_score))
#print the accuracy score for each of the C values
print('Mean accuracy score: %0.3f' % np.array(acc_score).mean())
which results in output like below
[0.52 0.6 0.49 0.6 0.55 0.6 0.5 0.51 0.63 0.54]
Mean accuracy score: 0.554
[0.51 0.45 0.54 0.42 0.53 0.45 0.52 0.48 0.5 0.39]
Mean accuracy score: 0.479
[0.73 0.76 0.7 0.64 0.61 0.68 0.71 0.61 0.71 0.71]
Mean accuracy score: 0.686
[0.76 0.6 0.66 0.61 0.67 0.66 0.69 0.74 0.63 0.65]
Mean accuracy score: 0.667
However, I want to print the results more meaningfully like below :
[0.52 0.6 0.49 0.6 0.55 0.6 0.5 0.51 0.63 0.54]
Mean accuracy score for (C=0.1,gamma=0.1): 0.554
[0.51 0.45 0.54 0.42 0.53 0.45 0.52 0.48 0.5 0.39]
Mean accuracy score (C=0.1, gamma = 1): 0.479
[0.73 0.76 0.7 0.64 0.61 0.68 0.71 0.61 0.71 0.71]
Mean accuracy score (C=0.5, gamma = 0.1): 0.686
[0.76 0.6 0.66 0.61 0.67 0.66 0.69 0.74 0.63 0.65]
Mean accuracy score (C=0.5, gamma = 1): 0.667
How can I print the results more meaningfully in the existing code ?


try this:
# (1)
print('Mean accuracy score (C=%0.1f, gamma=%0.1f): %0.3f' % (z, v, np.array(acc_score).mean()))
# (2)
print("Mean accuracy score (C={}, gamma={}): {}".format(z, v, np.array(acc_score).mean()))
# (3)
print("Mean accuracy score (C="+str(z)+", gamma="+str(v)+"): "+str(np.array(acc_score).mean()))
output:
Mean accuracy score (C=0.1, gamma=0.1): 0.554


You can use fancy output formatting:
print(f'Mean accuracy score (C={z:.1f}, gamma={v:.1f}): {np.array(acc_score).mean()}')

Related

Scikit Learn Dataset

In Scikit learn, when doing X,Y = make_moons(500,noise = 0.2) and after printing X and Y, I see that they are like arrays with a bunch of entries but with no commas?
I have data that I want to use instead of the Scikit learn moons dataset, but I dont understand what data type these Scikit learn data sets are and how I can make my data follow this data type.

The first one X is a 2d array:
array([[-6.72300890e-01, 7.40277997e-01],
[ 9.60230259e-02, 9.95379113e-01],
[ 3.20515776e-02, 9.99486216e-01],
[ 8.71318704e-01, 4.90717552e-01],
....
[ 1.61911895e-01, -4.55349012e-02]])
Which contains the x-axis, and y-axis position of points.
The second part of the tuple: y, is an array that contains the labels (0 or 1 for binary classification).
array([0, 0, 0, 0, 1, ... ])
To use this data in a simple classification task, you could do the following:
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
# Create dataset
X, y = make_moons(500,noise = 0.2)
# Split dataset in a train part and a test part
train_X, test_X, train_y, test_y = train_test_split(X, y)
# Create the Logistic Regression classifier
log_reg = LogisticRegression()
# Fit the logistic regression classifier
log_reg.fit(train_X, train_y)
# Use the trained model to predit con the train and predict samples
train_y_pred = log_reg.predict(train_X)
test_y_pred = log_reg.predict(test_X)
# Print classification report on the training data
print(classification_report(train_y, train_y_pred))
# Print classification report on the test data
print(classification_report(test_y, test_y_pred))
The results are:
On training data
precision recall f1-score support
0 0.88 0.87 0.88 193
1 0.86 0.88 0.87 182
accuracy 0.87 375
macro avg 0.87 0.87 0.87 375
weighted avg 0.87 0.87 0.87 375
On test data
precision recall f1-score support
0 0.81 0.89 0.85 57
1 0.90 0.82 0.86 68
accuracy 0.86 125
macro avg 0.86 0.86 0.86 125
weighted avg 0.86 0.86 0.86 125
As we can see, the f1_score is not very different between the train and the test set, the model is not overfitting.

Sklearn Linear SVM cannot train in multilabel classification

I want to train linear SVM with multilabel classification with the following code:
from sklearn.svm import LinearSVC
from sklearn.multioutput import MultiOutputClassifier
import numpy as np
data = np.loadtxt('tictac_multi.txt')
X = data[:,:9]
y = data[:,9:]
clf = MultiOutputClassifier(LinearSVC(random_state=0, tol=1e-5, C=100, penalty='l2',max_iter=2000))
clf.fit(X, y)
print(clf.score(X, y))
The dataset can be found here https://www.connellybarnes.com/work/class/2016/deep_learning_graphics/proj1/tictac_multi.txt
I try to adjust different parameters such as C, tol, max_iter, and others. The linear SVM model still cannot be trained well. The training accuracy is still less than 0.01 whatever I adjust any parameters...
The above code's output is:
Warning (from warnings module):
File "C:\Users\hyu14\AppData\Local\Programs\Python\Python38-32\lib\site-packages\sklearn\svm\_base.py", line 946
warnings.warn("Liblinear failed to converge, increase "
ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.
Warning (from warnings module):
File "C:\Users\hyu14\AppData\Local\Programs\Python\Python38-32\lib\site-packages\sklearn\svm\_base.py", line 946
warnings.warn("Liblinear failed to converge, increase "
ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.
Warning (from warnings module):
File "C:\Users\hyu14\AppData\Local\Programs\Python\Python38-32\lib\site-packages\sklearn\svm\_base.py", line 946
warnings.warn("Liblinear failed to converge, increase "
ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.
Warning (from warnings module):
File "C:\Users\hyu14\AppData\Local\Programs\Python\Python38-32\lib\site-packages\sklearn\svm\_base.py", line 946
warnings.warn("Liblinear failed to converge, increase "
ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.
Warning (from warnings module):
File "C:\Users\hyu14\AppData\Local\Programs\Python\Python38-32\lib\site-packages\sklearn\svm\_base.py", line 946
warnings.warn("Liblinear failed to converge, increase "
ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.
Warning (from warnings module):
File "C:\Users\hyu14\AppData\Local\Programs\Python\Python38-32\lib\site-packages\sklearn\svm\_base.py", line 946
warnings.warn("Liblinear failed to converge, increase "
ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.
Warning (from warnings module):
File "C:\Users\hyu14\AppData\Local\Programs\Python\Python38-32\lib\site-packages\sklearn\svm\_base.py", line 946
warnings.warn("Liblinear failed to converge, increase "
ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.
Warning (from warnings module):
File "C:\Users\hyu14\AppData\Local\Programs\Python\Python38-32\lib\site-packages\sklearn\svm\_base.py", line 946
warnings.warn("Liblinear failed to converge, increase "
ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.
Warning (from warnings module):
File "C:\Users\hyu14\AppData\Local\Programs\Python\Python38-32\lib\site-packages\sklearn\svm\_base.py", line 946
warnings.warn("Liblinear failed to converge, increase "
ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.
0.011601282246985194
The accuracy is 0.0116 from the current code.

It seems like a "TicTacToe" dataset (from the filename and the format).
Assuming that the first nine columns of the datset provide the description of the 9 cells in a specific moment of the game and that the other nine represent the cells corresponding to the good moves, you can train a classifier cell by cell, in order to predict if a cell is a good move or not.
So, you actually need to train 9 binary classifiers, not one. I sketched a very simple approach in the following code based on this idea. Start with simple cross-validation, after splitting the dataset in train/test (80/20):
import numpy as np
from sklearn.svm import LinearSVC
from sklearn.model_selection import cross_validate, train_test_split
from sklearn.metrics import classification_report
import pandas as pd
# Load data, creating a Dataframe holding input and outputs
df = pd.read_csv('tictac_multi.txt', sep=' ', header=None)[list(range(18))].copy()
df.columns = pd.MultiIndex.from_product((('input', 'output'), [f'x{i}' for i in range(1, 10)]))
# split dataset 80/20 (also shuffle it)
X_train, X_test, y_train, y_test = train_test_split(df['input'].values, df['output'].values, test_size=0.2, random_state=42)
# Get scores from cross validation
scores = {
s: cross_validate(
LinearSVC(random_state=0, dual=False, class_weight='balanced', tol=1e-5),
X_train, y_train[:, i], cv=5, scoring=['balanced_accuracy', 'precision', 'recall', 'f1_weighted'],
n_jobs=-1,
) for i, (s, clf) in enumerate(sorted(clfs.items()))
}
As you can see, I've used some non-default options (dual=False, class_weight='balanced') for the classifier: they are only an educated guess, you should investigate more to better understand the data and the problem and then look for the best parameters for your model (e.g., a grid search).
Here the scores:
{'x1': {'fit_time': array([0.01000571, 0.00814652, 0.00937247, 0.00622296, 0.00536656]),
'score_time': array([0.01159358, 0.00597596, 0.00835085, 0.00647163, 0.00619125]),
'test_balanced_accuracy': array([0.52209841, 0.51820565, 0.53743952, 0.55455645, 0.53620968]),
'test_precision': array([0.25454545, 0.25 , 0.26611227, 0.27659574, 0.26295585]),
'test_recall': array([0.5060241 , 0.52016129, 0.51612903, 0.5766129 , 0.55241935]),
'test_f1_weighted': array([0.56543736, 0.55328701, 0.58232694, 0.57711117, 0.56292617])},
'x2': {'fit_time': array([0.00737047, 0.00885296, 0.00616217, 0.00707698, 0.0071764 ]),
'score_time': array([0.00650406, 0.00595641, 0.00623679, 0.00636506, 0.00567913]),
'test_balanced_accuracy': array([0.57367382, 0.5342687 , 0.55287658, 0.56565243, 0.57909451]),
'test_precision': array([0.22520661, 0.20041754, 0.21073559, 0.22037422, 0.23175966]),
'test_recall': array([0.5828877 , 0.51336898, 0.56684492, 0.56684492, 0.57446809]),
'test_f1_weighted': array([0.6183652 , 0.60068273, 0.59707974, 0.61584554, 0.63060231])},
'x3': {'fit_time': array([0.0067966 , 0.00759745, 0.00617337, 0.00679278, 0.00650382]),
'score_time': array([0.00605631, 0.00537109, 0.00551271, 0.00665474, 0.00649571]),
'test_balanced_accuracy': array([0.52683332, 0.54103562, 0.56227539, 0.53312408, 0.51986383]),
'test_precision': array([0.25502008, 0.26639344, 0.28367347, 0.26035503, 0.25 ]),
'test_recall': array([0.51626016, 0.52845528, 0.56275304, 0.53441296, 0.53036437]),
'test_f1_weighted': array([0.56805171, 0.58208858, 0.59506983, 0.56776364, 0.55079222])},
'x4': {'fit_time': array([0.00649667, 0.00767159, 0.00802064, 0.00769711, 0.00611663]),
'score_time': array([0.00572419, 0.00529647, 0.00616765, 0.00592041, 0.00609517]),
'test_balanced_accuracy': array([0.53369766, 0.57259312, 0.57644138, 0.55746825, 0.51877354]),
'test_precision': array([0.19791667, 0.22290389, 0.22540984, 0.21489362, 0.18930041]),
'test_recall': array([0.51351351, 0.58602151, 0.59139785, 0.54301075, 0.49462366]),
'test_f1_weighted': array([0.6005693 , 0.615313 , 0.61784599, 0.61784823, 0.58924053])},
'x5': {'fit_time': array([0.00650501, 0.005898 , 0.00682783, 0.00629449, 0.00635648]),
'score_time': array([0.00553894, 0.0059135 , 0.00625896, 0.00583744, 0.00580502]),
'test_balanced_accuracy': array([0.51108635, 0.50499149, 0.52183641, 0.53230958, 0.51296946]),
'test_precision': array([0.30185185, 0.29735234, 0.31163708, 0.322 , 0.30522088]),
'test_recall': array([0.53094463, 0.47557003, 0.51465798, 0.52272727, 0.49350649]),
'test_f1_weighted': array([0.5248707 , 0.53861778, 0.54612005, 0.55679291, 0.54217533])},
'x6': {'fit_time': array([0.00703621, 0.00908065, 0.00665092, 0.00619102, 0.00814819]),
'score_time': array([0.00568652, 0.00626183, 0.00584817, 0.00574327, 0.00552726]),
'test_balanced_accuracy': array([0.55457928, 0.55569106, 0.50701258, 0.53690769, 0.56919396]),
'test_precision': array([0.2145749 , 0.21621622, 0.18480493, 0.20416667, 0.22540984]),
'test_recall': array([0.56084656, 0.55026455, 0.47619048, 0.51851852, 0.57894737]),
'test_f1_weighted': array([0.60241544, 0.61008882, 0.5813744 , 0.60080544, 0.6130977 ])},
'x7': {'fit_time': array([0.0070405 , 0.00908256, 0.00702643, 0.00635576, 0.00632381]),
'score_time': array([0.00546646, 0.00674367, 0.00542998, 0.00671315, 0.00549483]),
'test_balanced_accuracy': array([0.53124816, 0.52187224, 0.54180051, 0.57438252, 0.52764072]),
'test_precision': array([0.27054108, 0.26235741, 0.27659574, 0.30364372, 0.26824034]),
'test_recall': array([0.52325581, 0.53488372, 0.55642023, 0.58365759, 0.48638132]),
'test_f1_weighted': array([0.56745684, 0.54860915, 0.56677092, 0.5996452 , 0.57954721])},
'x8': {'fit_time': array([0.00761437, 0.00997519, 0.006984 , 0.00623441, 0.00683069]),
'score_time': array([0.00540686, 0.00635052, 0.00645804, 0.00535131, 0.00548935]),
'test_balanced_accuracy': array([0.51471322, 0.56996108, 0.52712724, 0.5443143 , 0.55319282]),
'test_precision': array([0.18661258, 0.22292994, 0.192607 , 0.20408163, 0.20874751]),
'test_recall': array([0.49462366, 0.56451613, 0.53513514, 0.54054054, 0.56756757]),
'test_f1_weighted': array([0.58328382, 0.62374708, 0.57815794, 0.60051373, 0.59779516])},
'x9': {'fit_time': array([0.00723267, 0.0069263 , 0.00828266, 0.00672913, 0.00750995]),
'score_time': array([0.00545311, 0.00556946, 0.00732398, 0.0056181 , 0.00555682]),
'test_balanced_accuracy': array([0.53490307, 0.55281703, 0.58447809, 0.52272419, 0.54294236]),
'test_precision': array([0.26388889, 0.27868852, 0.29811321, 0.25506073, 0.27198364]),
'test_recall': array([0.53413655, 0.54618474, 0.63453815, 0.5060241 , 0.532 ]),
'test_f1_weighted': array([0.56987212, 0.58922553, 0.59075641, 0.56631422, 0.5819019 ])}}
As you can see, they are not very good, but far from 0.
Now, train the models on the whole train dataset, and assess performance on test data:
def train_clfs(clfs, X, y):
return {s: clf.fit(X, y[:, i]) for i, (s, clf) in enumerate(sorted(clfs.items()))}
def get_predictions(clfs, inp):
return {s: clf.predict(inp) for s, clf in clfs.items()}
# Train the classifiers
clfs = {s: LinearSVC(random_state=0, dual=False, class_weight='balanced', tol=1e-5) for s in sorted(df['output'].columns)}
clfs = train_clfs(clfs, X_train, y_train)
# Try them on the test values
pred = get_predictions(clfs, X_test)
# Get the classification report for each classifier
cl_report = {s: classification_report(y_test[:, i], p) for i, (s, p) in enumerate(sorted(pred.items()))}
And here the performances:
x1
precision recall f1-score support
0 0.76 0.52 0.62 988
1 0.25 0.49 0.33 323
accuracy 0.51 1311
macro avg 0.50 0.51 0.48 1311
weighted avg 0.63 0.51 0.55 1311
x2
precision recall f1-score support
0 0.87 0.56 0.68 1086
1 0.22 0.58 0.31 225
accuracy 0.57 1311
macro avg 0.54 0.57 0.50 1311
weighted avg 0.75 0.57 0.62 1311
x3
precision recall f1-score support
0 0.79 0.50 0.61 998
1 0.26 0.57 0.36 313
accuracy 0.52 1311
macro avg 0.53 0.54 0.49 1311
weighted avg 0.66 0.52 0.55 1311
x4
precision recall f1-score support
0 0.84 0.54 0.65 1061
1 0.22 0.57 0.32 250
accuracy 0.54 1311
macro avg 0.53 0.55 0.49 1311
weighted avg 0.72 0.54 0.59 1311
x5
precision recall f1-score support
0 0.72 0.53 0.61 926
1 0.31 0.50 0.38 385
accuracy 0.52 1311
macro avg 0.51 0.52 0.50 1311
weighted avg 0.60 0.52 0.54 1311
x6
precision recall f1-score support
0 0.85 0.57 0.69 1077
1 0.22 0.54 0.31 234
accuracy 0.57 1311
macro avg 0.53 0.56 0.50 1311
weighted avg 0.74 0.57 0.62 1311
x7
precision recall f1-score support
0 0.81 0.55 0.65 1021
1 0.25 0.53 0.34 290
accuracy 0.55 1311
macro avg 0.53 0.54 0.50 1311
weighted avg 0.68 0.55 0.59 1311
x8
precision recall f1-score support
0 0.84 0.55 0.66 1069
1 0.21 0.53 0.30 242
accuracy 0.55 1311
macro avg 0.52 0.54 0.48 1311
weighted avg 0.72 0.55 0.60 1311
x9
precision recall f1-score support
0 0.79 0.54 0.64 1006
1 0.26 0.52 0.35 305
accuracy 0.54 1311
macro avg 0.52 0.53 0.49 1311
weighted avg 0.67 0.54 0.57 1311

Sklearn - plotting classification report gives a different output than basic avg?

I wanted to leverage this answer How to plot scikit learn classification report? turning an sklearn classification report into a heatmap.
It's all working with their sample report, however my classification report looks slightly different and is thus screwing up the functions.
Their report (notice the avg / total):
sampleClassificationReport =
precision recall f1-score support
Acacia 0.62 1.00 0.76 66
Blossom 0.93 0.93 0.93 40
Camellia 0.59 0.97 0.73 67
Daisy 0.47 0.92 0.62 272
Echium 1.00 0.16 0.28 413
avg / total 0.77 0.57 0.49 858
My report with metrics.classification_report(valid_y, y_pred) :
precision recall f1-score support
0 1.00 0.18 0.31 11
1 0.00 0.00 0.00 14
2 0.00 0.00 0.00 19
3 0.50 0.77 0.61 66
4 0.39 0.64 0.49 47
5 0.00 0.00 0.00 23
accuracy 0.46 180
macro avg 0.32 0.27 0.23 180
weighted avg 0.35 0.46 0.37 180
The issue, from the selected answer in the heatmap link, is here:
for line in lines[2 : (len(lines) - 2)]:
t = line.strip().split()
if len(t) < 2: continue
classes.append(t[0])
v = [float(x) for x in t[1: len(t) - 1]]
support.append(int(t[-1]))
class_names.append(t[0])
print(v)
plotMat.append(v)
Because I get the error:
ValueError: could not convert string to float: 'avg'
So the problem truly is how my classification report is being outputted. What can I change here to match the sample?
EDIT: what Ive tried:
df = pd.DataFrame(metrics.classification_report(valid_y, y_pred)).T
df['support'] = df.support.apply(int)
df.style.background_gradient(cmap='viridis',
subset=pd.IndexSlice['0':'9', :'f1-score'])
Error:
ValueError: DataFrame constructor not properly called!

With the advent of output_dict param in classification_report, there is no hassle for parsing the report. You can directly use the output of classification report to be read as pd.DataFrame. Then, you could use the pd.Style option to render the heat map.
Example:
from sklearn.metrics import classification_report
import numpy as np
import pandas as pd
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split, GridSearchCV
X, y = make_classification(n_samples=1000, n_features=30,
n_informative=12,
n_clusters_per_class=1, n_classes=10,
class_sep=2.0, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, stratify=y)
clf = LogisticRegression(max_iter=1000, random_state=42).fit(X_train, y_train)
df = pd.DataFrame(classification_report(clf.predict(X_test),
y_test, digits=2,
output_dict=True)).T
df['support'] = df.support.apply(int)
df.style.background_gradient(cmap='viridis',
subset=pd.IndexSlice['0':'9', :'f1-score'])

classification_report output with missing accuracy data

I'm taking a course and doing some examples my output comes wrong.
import pandas as pd
df = pd.read_csv(r'E:\Python Projects\Python-Data-Science-and-Machine-Learning-Bootcamp\Machine Learning\Árvores de decisão e Florestas Aleatórias\kyphosis.csv')
from sklearn.model_selection import train_test_split
x = df.drop('Kyphosis', axis=1)
y = df['Kyphosis']
X_train, X_test, y_train, y_test = train_test_split(x,y,test_size=0.33)
from sklearn.tree import DecisionTreeClassifier
dtree = DecisionTreeClassifier()
dtree.fit(X_train, y_train)
pred = dtree.predict(X_test)
from sklearn.metrics import classification_report
print(classification_report(y_test, pred))
These 2 data are missing

This is how classification_report returns the text summary, nothing is missing.
Look into the documentation: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html
>>> from sklearn.metrics import classification_report
>>> y_true = [0, 1, 2, 2, 2]
>>> y_pred = [0, 0, 2, 2, 1]
>>> target_names = ['class 0', 'class 1', 'class 2']
>>> print(classification_report(y_true, y_pred, target_names=target_names))
precision recall f1-score support
<BLANKLINE>
class 0 0.50 1.00 0.67 1
class 1 0.00 0.00 0.00 1
class 2 1.00 0.67 0.80 3
<BLANKLINE>
accuracy 0.60 5
macro avg 0.50 0.56 0.49 5
weighted avg 0.70 0.60 0.61 5
<BLANKLINE>
>>> y_pred = [1, 1, 0]
>>> y_true = [1, 1, 1]
>>> print(classification_report(y_true, y_pred, labels=[1, 2, 3]))
precision recall f1-score support
<BLANKLINE>
1 1.00 0.67 0.80 3
2 0.00 0.00 0.00 0
3 0.00 0.00 0.00 0
<BLANKLINE>
micro avg 1.00 0.67 0.80 3
macro avg 0.33 0.22 0.27 3
weighted avg 1.00 0.67 0.80 3
<BLANKLINE>
The reported averages include macro average (averaging the unweighted mean per label), weighted average (averaging the support-weighted mean per label), and sample average (only for multilabel classification). Micro average (averaging the total true positives, false negatives and false positives) is only shown for multi-label or multi-class with a subset of classes, because it corresponds to accuracy otherwise.
Your accuracy is simply 74%.

Your classification report is not missing anything; it is a peculiarity of scikit-learn that it chooses to display the accuracy there, but there is no "precision accuracy" or "recall accuracy". Your actual accuracy is what is shown under the f1-score column; here is an example with toy data from the documentation:
from sklearn.metrics import classification_report
y_true = [0, 1, 2, 2, 2]
y_pred = [0, 0, 2, 2, 1]
target_names = ['class 0', 'class 1', 'class 2']
print(classification_report(y_true, y_pred, target_names=target_names))
Result:
precision recall f1-score support
class 0 0.50 1.00 0.67 1
class 1 0.00 0.00 0.00 1
class 2 1.00 0.67 0.80 3
accuracy 0.60 5
macro avg 0.50 0.56 0.49 5
weighted avg 0.70 0.60 0.61 5
i.e. the accuracy here is 0.6, something that you can directly verify:
from sklearn.metrics import accuracy_score
accuracy_score(y_true, y_pred)
# 0.6
You are right that it's odd, though, and it can certainly be confusing. Not a great design choice...

GridSearchCV returns a different value

I split my dataset in two parts: training set and test set. For now just forget the test set and use the training set with the function GridSearchCV of the package sklearn.model_selection to search the best parameters for an SVM:
Cs = [0.001, 0.01, 0.1, 1, 10, 100, 1000]
gammas = [0.001, 0.01, 0.1, 1]
# Set the parameters by cross-validation
param_grid = [{'kernel': ['rbf'], 'gamma': gammas, 'C': Cs}]
clf = GridSearchCV(svm.SVC(), param_grid = param_grid, cv=nfolds, verbose=1.)
clf.fit(x_train, labels)
after found my best C and gamma parameters, I create an SVM and I fit it with the training set (used before to search the best C and gamma):
model = svm.SVC(kernel='rbf', C = clf.best_params_['C'], gamma = clf.best_params_['gamma'])
model.fit(x_train, y_train)
At this point I tried one thing, I used the predict() function of the GridSearchCV object and the one of the svm.SVC object:
predicted_label1 = model.predict(x_test)
predicted_label2 = clf.predict(x_test)
and then I used the classification_report(y_test, predicted_label) to valuate my two predicted_label vectors. In my mind I should obtain the same values but this not happens...Here my output:
precision recall f1-score support
0.0 0.24 0.97 0.39 357
1.0 0.00 0.00 0.00 358
2.0 0.00 0.00 0.00 357
3.0 0.00 0.00 0.00 357
avg / total 0.06 0.24 0.10 1429
fine parametri
training set and test set saved
Create SVM classifier
precision recall f1-score support
0.0 0.70 0.63 0.66 357
1.0 0.89 0.90 0.90 358
2.0 0.89 0.94 0.91 357
3.0 0.85 0.88 0.86 357
avg / total 0.83 0.84 0.83 1429
The first is from the GridSearchCV and the second from the SVM...
Is this normal?
What does GridSearchCV returns? Does it fit with the passed training set?

Categories

Resources