Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
I am new in the NLP community and need more light on something.
I saw that Keras has an Embedding layer that is generally used before the LSTM layer. But what algorithm hides behind it? Is it Word2Vec, Glove or something else?
My task is a supervised text classification problem.
The embedding layer is a randomly initialized matrix, with the dimension of (number_of_words_in_vocab * embedding_dimension). The embedding_dimension is custom defined dimension, and an hyper-parmeter that we will have to choose.
Here, the embeddings are updated during back-propagation, and are learnt from your task and task-specific corpus.
However, pre-trained embeddings such as word2vec, glove are learnt in an unsupervised manner on huge corpus. Pre-trianed embeddings provides a good initialization for this embedding layer. Thus, you can use the pre-trained embeddings to initialize this embedding layer, and also choose if you want to freeze these emebeddings or update these embeddings during the back-propagation.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
I have a BERT multilanguage model from Google. And I have a lot of text data in my language (Korean). I want BERT to make better vectors for texts in this language. So I want to additionally train BERT on that text corpus I have. Like if I would have w2v model trained on some data and would want to continue training it. Is it possible with BERT?
There are a lot of examples of "fine-tuning" BERT on some specific tasks like even the original one from Google where you can train BERT further on your data. But as far as I understand it (I might be wrong) we do it within our task-specified model (for classification task for example). So... we do it at the same time as training our classifier (??)
What I want is to train BERT further separately and then get fixed vectors for my data. Not to build it into some task-specified model. But just get vector representation for my data (using get_features function) like they do in here. I just need to train the BERT model additionally on more data of the specific language.
Would be endlessly grateful for any suggestions/links on how to train BURT model further (preferably Tensorflow). Thank you.
Package transformers provides code for using and fine-tuning of most currently popular pre-trained Transformers including BERT, XLNet, GPT-2, ... You can easily load the model and continue training.
You can get the multilingual BERT model:
tokenizer = BertTokenizer.from_pretrained('bert-base-multiligual-cased')
model = TFBertForSequenceClassification.from_pretrained('bert-base-multiligual-cased')
The tokenizer is used both for tokenizing the input and for converting the sub-words into embedding ids. Calling the model on the subword indices will give you hidden states of the model.
Unfortunately, the package does not implement the training procedure, i.e., the masked language model and the next sentence prediction. You will need to write it yourself, but the training procedure well described in the paper and the implementation will be straightforward.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I am a total rookie in computer vision. I am looking to build a model without using pre-trained models for coco dataset or any open-source image datasets. Any articles or references to build such models would be appreciated. I would like to build this model from scratch and make no suggestions on pre-existing trained models or Api are irrelevant to this question. and thanks in advance for any suggestions. the programming language of preference for this project is python
How about this tutorial on keras blogs:
https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html
Should be pretty straightforward and it is written in a step by step manner by the author of Keras. It has these three stages, but you only need the first one:
training a small network from scratch (as a baseline)
using the bottleneck features of a pre-trained network
fine-tuning the top layers of a pre-trained network
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
I'm doing a project these days.
Goal of this project is approximately 200 symbol recognition.
Symbols are using in the navigation(turn_right, turn_left etc..)
I'm using YOLO model now
For traning this models, I thought I needed some improvement about traning speed.
This program will using when testing new navigation.
Is there any better models?
The model needs very fast traning speed, and high accuracy
Yolo is one of the best object detection for real time detection. Fast Training and high accuracy are competing goals. Did you mean test speed (with a trained model)?
Anyway, if you need fast training I highly suggest the cyclical learning rate strategy proposed by Leslie N. Smith.
Yolo has different version, so take a look at that as well.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I am planning on building a gender classifier. I know the two popular models are tf-idf and word2vec.
While tf-idf focuses on the importance of a word in a document and similarity of documents, word2vec focuses more on the relationship between words and similarity between them.
However none of theme seem to be perfect for building vector features to be used for gender classification. Is there any other alternative vectorization model that might suit this task?
Yes, there is another alternative to w2v: GloVe.
GloVe stands for Global Vector Embeddings.
As someone who has used this technique before to good effect, I would recommend GloVe.
GloVe optimally trains neural word embeddings not just by looking at local windows but considering a much larger width (30+ size), thereby embedding a much deeper level of semantics to the embedding.
With glove, it is easy to model relationships such as: X[man] - X[woman] = X[king] - X[queen], where these are all vectors.
Credits: GloVe GitHub page (linked below).
You can train your own GloVe embeddings, or you may use their retrained models available. Even for specific domains, the general models seem to work reasonably well, although you would get a lot more out of your models if you trained them yourself. Please look at the GitHub page for instructions on how to train your own models. It is very easy.
Additional reading:
GloVe: Global Vectors for Word Representation
GloVe repository
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I'm trying to create a multilabel classifier with scikit-learn to texts.
I am new to scikit learn and I do not know if it is possible to create a classifier for text.
My intention is to use SVM multilabel, but do not know if I have to adapt the texts to train the classifier or else you can work directly with texts.
Does anyone know some documentation on this subject?
You can refer to this example: Classification of text documents using sparse features
which can give you exposer to not only multiclass but also basic text mining details of:
Vectorizer and hashing
Feature selection
Handling Sparse Data
Comparing different basic models