I have used fasttext train_supervised utility to train a classification model according to their webpage https://fasttext.cc/docs/en/supervised-tutorial.html .
model = fasttext.train_supervised(input='train.txt', autotuneValidationFile='validation.txt', autotuneDuration=600)
After I got the model how could I explore what kind of best parameters for the model like in sklearn after a set of best parameters trained, we could always check the values for these parameters but I could not find any document to explain this.
I also used this trained model to make prediction on my data
model.predict(test_df.iloc[2, 1])
It will return the label with a probability like this
(('__label__2',), array([0.92334366]))
I'm wondering if I have 5 labels, every time when make prediction,is it possible for each text to get all the probability for each label?
Like for the above test_df text,
I could get something like
model.predict(test_df.iloc[2, 1])
(('__label__2',), array([0.92334366])),(('__label__1',), array([0.82334366])),
(('__label__3',), array([0.52333333])),(('__label__0',), array([0.07000000])),
(('__label__4',), array([0.00002000]))
could find anything related to make change to get such prediction results.
Any suggestion?
Thanks.
As you can see here in the documentation, when using predict method, you should specify k parameter to get the top-k predicted classes.
model.predict("Why not put knives in the dishwasher?", k=5)
OUTPUT:
((u'__label__food-safety', u'__label__baking', u'__label__equipment',
u'__label__substitutions', u'__label__bread'), array([0.0857 , 0.0657,
0.0454, 0.0333, 0.0333]))
Related
I have trained my model and saved it as json file and then i have saved the wights and load it as h5
Then i have used it in another script in order to feed some records to test the accuracy of the model what i want to do is:
Making the model say what is the predicted label and then i must compare it with the true label from the csv file but I don’t know how to get the predicted values?
I tried this
predict_x= loaded_model(X_test) y_pred=np.argmax(predict_x,axis=1)
And then
for i in y_pred
to loop through y_pred, i thought i could see my y classes which are from 0-4 while actually i found numers like 17,22,13…
Can in one till me what I should do?
Thank you in advance
………………………………………………….
So I am working on a model that attempts to use RandomForest to classify samples into 1 of 7 classes. I'm able to build and train the model, but when it comes to evaluating it using roc_auc function, I'm able to perform 'ovr' (oneVsrest) but 'ovo' is giving me some trouble.
roc_auc_score(y_test, rf_probs, multi_class = 'ovr', average = 'weighted')
The above works wonderfully, I get my output, however, when I switch multi_class to 'ovo' which I understand might be better with class imbalances, I get the following error:
roc_auc_score(y_test, rf_probs, multi_class = 'ovo')
IndexError: too many indices for array
(I pasted the whole traceback below!)
Currently my data is set up as follow:
y_test (61,1)
y_probs (61, 7)
Do I need to reshape my data in a special way to use 'ovo'?
In the documentation, https://thomasjpfan.github.io/scikit-learn-website/modules/generated/sklearn.metrics.roc_auc_score.html, it says "binary y_true, y_score is supposed to be the score of the class with greater label. The multiclass case expects shape = [n_samples, n_classes] where the scores correspond to probability estimates."
Additionally, the whole traceback seems to hint as using maybe using a more binary array (hopefully that's the right term! I'm new to this!)
Very, very thankful for any ideas/thoughts!
#Tirth Patel provided the right answer, I needed to reshape my test set using one hot encoding. Thank you!
I am using SVM classifier for multi class classification.
I want svc.predict to return the result along with probabilities for the other classes also.
The result i got is like this:
print(svclassifier.predict([[79,93,60,50,50,80,81,88,87,100,100,71,100,83,100,100,75,70,100,60]]))
Expected OutPut is the category number= 27
Output Obtained is: 27
But I want the output in the form of probabilities....
Yes, you can use predict_proba method (see documentation).
Please also consider the note seen in documentation:
The probability model is created using cross validation, so the
results can be slightly different than those obtained by predict.
Also, it will produce meaningless results on very small datasets
Can someone help me with the code for getting the predicted probability values? My model is working fine and is giving me predictions as 1 and 0 however I need the probability values also. The code is in two python files. The first file uses the training data set to create the map file. The second python file (scoring file) uses the map file on the test data to predict. Can someone let me know the code I should insert to get the probability values.
The below code is from the scoring file and here I need the code the get the probability values
pred = model.predict(X.values)
data["Predicted"] = pred
# I NEED THE CODE HERE TO GET THE PROBABILITY VALUES.
data.to_excel(r'result.xlsx', index=False)
Thanks a lot
Check if your model has predict_proba method.
The usage is same as the same predict method.
prob = model.predict_proba(X.values)
Edit:
Some of the learning model implementations from sklearn provide the predict_proba method. It is not a metric but as I said, a method of the class of the learning model.
For example:
from sklearn.tree import DecisionTreeClassifier
# after split you have X_train,y_train,X_testy_test
model = DecisionTreeClassifier()
model.fit(X_train,y_train)
proba = model.predict_proba(X_test)
I am no longer able to edit my question so putting it here.
Thanks for all the help. I am using random forest model. This is my code and the line 4 below is giving an error. If I remove line 4, the code runs but in the final excel file I do not get the probabilities but only the predictions as 1 and 0. Can someone please let me know how to resolve this error. The last line of the error says
ValueError: Wrong number of items passed 2, placement implies 1
pred = model.predict(X.values)
data["Predicted"] = pred
prob = model.predict_proba(X.values)
data["Pred Value"]= prob - this line causes error
data.to_excel(r'result.xlsx', index=False)'
Thanks
I'm using the MaxEnt classifier from the Python NLTK library. For my dataset, I have many possible labels, and as expected, MaxEnt returns just one label. I have trained my dataset and get about 80% accuracy. I've also tested my model on unknown data items, and the results are good. However, for any given unknown input, I want to be able to print/display a ranking of all the possible labels based on some internal criteria MaxEnt used to select the one, such as confidence/probability. For example, suppose I had a,b,c as possible labels and I use MaxEnt.classify(input), I get currently one label, let's say c. However, I want to be able to view something like a (0.9), b(0.7), c(0.92), so I can see why c was selected, and possibly choose multiple labels based on those parameters. Apologies for my fuzzy terminology, I'm fairly new to NLP and machine learning.
Solution
Based on the accepted answer, here's a skeleton code example to demonstrate what I wanted and how it can be achieved. More classifier examples on the NLTK website.
import nltk
contents = read_data('mydataset.csv')
data_set = [(feature_sets(input), label) for (label, input) in contents] # User-defined feature_sets() function
train_set, test_set = data_set[:1000], data_set[1000:]
labels = [label for (input, label) in train_set]
maxent = nltk.MaxentClassifier.train(train_set)
maxent.classify(feature_sets(new_input)) # Returns one label
multi_label = maxent.prob_classify(feature_sets(new_input)) # Returns a DictionaryProbDist object
for label in labels:
multi_label.prob(label)
Try prob_classify(input)
It returns dictionary with probability for each label, see docs.