Let me make my question clearer:
I am using python gensim.models.Word2Vec to train a word embedding model. Based on my understanding, the model training is in essence a machine learning issue---to train a neural network via a prediction task. For example, if I select parameters to train a skip-gram model, then the model is trained by predicting context words from target word. Once the model is well-trained, word vectors are just obtained from the model.
If my understanding is correct, so since in fact it is a machine learning process and the training goal is to perform well in the prediction task, there should be a loss function during training and the model is supposed to make the loss as low as possible. So, how to know the model loss value for a given set of parameters? Or is there any other metrics that we can know to understand the model itself?
Hope I have made my question clear. In a word, I don't want to evaluate the model by its outputs as in the Google test set http://word2vec.googlecode.com/svn/trunk/questions-words.txt, but I want to understand the model itself as a simple machine learning problem during its training process. Would this be possible?
Related
I tried image classification using trained model and its working well but some images could not find perfectly in that time have to get that image and label from users so my doubt is..Is it possible to add new data into already trained model?
No, during inference time you use the weights of the trained model for predictions. Which basically means that at the time your model is deployed the capabilities of your image classifier are fixed by the weights. If you wish to improve your model, you would have to retrain your model with the new - data. However, there is another paradigm of learning called "Online Learning" where the model is continuously learning and modifying the weights. In this case your weights are not fixed and your model is continuously updating its weights with each training input. However afaik this is not usually recommended for CNNs, because the backward pass of gradients is computationally intensive and your inference will be slow because of this.
No model can predict with 100% accuracy if it does it's an ideal model. And if you want to add more data to your train model you have to retrain the model with the new data. Having more data is always a good idea. It allows the “data to tell for itself,” instead of relying on assumptions and weak correlations. Presence of more data results in better and accurate models. So if you want to get better accuracy you have to train your model with more data. Without retraining, you can't add data to your trained model.
As title, I know there're some model supporting streaming learning like classification model. And the model has function partial_fit()
Now I'm studying regression model like SVR and RF regressor...etc in scikit.
But most of regression models doesn't support partial_fit .
So I want to reach the same effect in neural network. If in tensorflow, how to do like that? Is there any keyword?
There is no some special function for it in TensorFlow. You make a single training pass over a new chunk of data. And then another training pass over another new chunk of data, etc till you reach the end of the data stream (which, hopefully, will never happen).
I'm new to keras seq2seq LSTM models. I have a working machine translation model and English-to-Arabic training data. I just trained the model using google colab tool and made some predictions. As you can see in the image, when I test the model on a text from the training data, it predicts well, but when I change ONE word, the prediction goes completely wrong!
I want my model to UNDERSTAND the full meaning of the text even when adding/deleting one word. How can I solve this problem?
LSTM wrong predictions when adding/deleting one word
In the image, the first test of each section is the text from the training data, which predicts well. The second test is the same but with adding/deleting one word.
UPDATE: Whenever I add validation split, the val_loss is always increasing and the model isn't learning too much! What's going worng?
This is the classical overtraining problem. Your model only learn to translate your training data by remembering each sample instead of understanding the concept behind it.
For this reason always split your training data in training data and validation data. The validation data must not be in the training data set! This way you can check if your model is actually learning something.
There are two main solution for this:
Like m33n said more training data (there is no data like more data)
Implement more regularization techniques like Dropout
Also the problem seems very ambigous. Translating sentences is not an easy task at all and copanies like google or deepl created very complex models trained with lots and lots of data occupied over years. Are you sure you have the necessary resources to accomplish this?
Can any one please help me out?
I am working on my thesis work. Its about Predicting Parkinson disease, Since i want to build an LSTM model to adapt independent of patients. Currently i have implemented it using TensorFlow with my own loss function.
Since i am planning to introduce both labeled train and unlabeled train data in every batch of data to train the model. I want to apply my own loss function on this both labeled and unlabeled train data and also want to apply cross entropy loss only on labeled train data. Can i do this in tensorflow?
So my question is, Can i have combination of loss functions in a single model training on different set of train data?
From an implementation perspective, the short answer would be yes. However, I believe your question could be more specific, maybe what you mean is whether you could do it with tf.estimator?
I am building a tensorflow model with new estimator high-level api. My model looks like below screenshot
.
In fact, the model is more complex than that due to the model is used to simulate game operation. Classification is responsible for decide whether it is good time for action. Then the regression will give the details about the action. It contains a combination of CNN and RNN.
However, due to the complexity and memory consumption, it is impossible to train and run classification and regression as two network simultaneously. Also, when I create my estimator like:
# Create the Estimator
mnist_classifier = tf.estimator.Estimator(
model_fn=cnn_model_fn, model_dir="/tmp/mnist_convnet_model")
I can only provide one model function for the estimator. Is it possible to train and run two estimator together?
Change your loss function to be a linear combination of regression and classification losses. It will be one estimator with one loss, but multiple inferences.