The dataset i am working on has 7 input features and 4 output class. The length of my dataset is 160. Will neural network be a good choice here? If so, how should i take my inputs to the neural network. Since I have 4 output class, i am going to use Softmax in the final layer.
If neural network makes no sense in such a small dataset, then what are the possible good Machine Learning Algorithms for have a great result in this kind of problems?.
Thanks 😊
What kind of a dataset do you have? I am assuming a tabular dataset.
You can use a neural network if you must. However, for such a small dataset, a neural network isn't usually advisable. You should rather look into the following classifiers:
Decision Tree
Naive Bayes
Multi-class Logistic Regression
Support Vector Machine
Ensemble models (Random Forest and/or Gradient Boosting)
Related
I have a question regarding the process to make a late fusion between SVM (Linear) and a NeuralNetwork (NN),
I have done some research and I found that concatenated the clf.predict_prob of SVM and Model.predic of NN, I should train the new model, however, these scores are for the test data and I cannot figure what to do with the training data.
In other words, I train the new model with the concatenated probability scores of the test data from my two models (SVM and NN) and I test this new model with the same concatenated data, and I'm not really sure of this.
Can you please give me an insight into if this is correct?
After a lot of searching and research I found the solution:
The solution is to train and test a new classifier, in my case it was another Neural Network, with the concatenated probability scores obtained from both data sets (training and test), of the two classifiers, the Linear SVM and the Neural Network.
An example of this of three Linear SVM Late fusion was implemented in python, and can be found in the following link:
https://github.com/JMalhotra7/Learning-image-by-parts-using-early-and-late-fusion-of-auto-encoder-features
I've worked with supervised learning models for a while now. I'm quite familiar with regression models (outputting continuous results for a target variable) and classification models (binary or multi-class classifications). However, what happens when I need to train a model for loan approval with the desired outputs as followed:
If the loan is disapproved, the decision will simply be 'disapprove'
If the loan is approved, the decision will be 'approve' and the model also gives suggestion on 'interest rate' and 'loan duration'
Can machine learning models yield such outputs?
Yup! You can do that by having multiple output heads from a neural network. Here is the keras api for that.
Although, you'll need to have the data for the other decisions too.
Yes of course most machine learning algorithms can work with more than 1 output layer. You can either use Keras neural network for creating multiple output predictor models or you can use a decision tree from sklearn. it works with 2 output layers checked
I used gridsearchcv to determine which hyperparameters in the mlpclassifier can make the accuracy from my neural network higher. I figured out that the amount of layers and nodes makes a difference but I'm trying to figure out which other configurations can make a difference in accuracy (F1 score actualy). But from my experience it lookes like parameters like "activation", "learning_rate", "solver" don't really change anything.
I need to do a research on which other hyperparameters can make a difference in the accuracy from predictions via the neural network.
Does someone have some tips/ideas on which parameters different from the amount of layers / nodes that can make a difference in the accuracy from my neural network predictions?
It all depends on your dataset. Neural network are not magical tools that can learn everything and also they require a lot of data compared to traditional machine learning models. In case of MLP, making a model extremely complex by adding a lot of layers is never a good idea as it makes the model more complex, slow and can lead to overfitting as well. Learning rate is an important factor as it is used to find the best solution for the model. A model makes mistakes and learns from it and the speed of learning is controlled by learning rate. If learning rate is too small, your model will take a long time to reach the best possible stage but if it is too high the model might just skip the best stage. The choice of activation function is again dependent on the use case and the data but for simpler datasets, activation function will not make a huge differnece.
In traditional deep learning models, a neural network is build up of several layers which might not always be dense. All the layers in MLP as dense i.e. feed forward. To improve your model, you can try a combination of dense layers along with cnn, rnn, lstm, gru or other layers. Which layer to use depends completely on your dataset. If you are using a very simple dataset for a school project, then experiment with traditional machine learning methods like random forest as you might get better results.
If you want to stick to neural nets, read about other types of layers, dropout, regularization, pooling, etc.
I have a working example of a multiclass classifier (using sklearn.svm) on text data. At one pass, I can only train/test one feature. Is it possible to stack several features in one classifier? For concreteness, my data has the following characteristics:
feature 1: 1c1, 1c2, 1c3, 1c4
feature 2: 2c1,2c2
feature 3: 3c1,3c2,3c3,3c4,3c5
feature 4: 4c1,4c2,4c3
Currently, I can run a training pass for feature 1 and repeat for feature 2 etc.
How can I stack them together to get an output vector like [1c4,2c1,3c5,4c2]? This is not a multi-label problem because feature sets {1..n} are mutually exclusive.
Apparently, there is no way to do this, per Alan Sz's answer
One obvious advantage of artificial neural networks over support vector machines is that artificial neural networks may have any number of outputs, while support vector machines have only one. The most direct way to create an n-ary classifier with support vector machines is to create n support vector machines and train each of them one by one. On the other hand, an n-ary classifier with neural networks can be trained in one go.
Lately I was on a Data Science meetup in my city, there was a talk about connecting Neural Networks with SVM. Unfortunately presenter had to quit right after presentation, so I wasn't able to ask some questions.
I was wondering how is that possible ? He was talking about using neural networks for his classification, and later on, he was using SVM classifier to improve his accuracy and precision by about 10%.
I am using Keras for Neural Networks and SKlearn for the rest of ML.
This is completely possible and actually quite common. You just select the output of a layer of the neural network and use that as a feature vector to train a SVM. Generally one normalizes the feature vectors as well.
Features learned by (Convolutional) Neural Networks are powerful enough that they generalize to different kinds of objects and even completely different images. For examples see the paper CNN Features off-the-shelf: an Astounding Baseline for Recognition.
About implementation, you just have to train a neural network, then select one of the layers (usually the ones right before the fully connected layers or the first fully connected one), run the neural network on your dataset, store all the feature vectors, then train an SVM with a different library (e.g sklearn).