I am new to nlp field. I have some questions about word2vec embeddings. as I know they have a fixed size dictionary of vocabs. so definitely there some words which is not in that predefined dictionary of vocab, for i.e. we may get an special name (like people's name) which doesn't exist in the vocab and they are called out-of-vocabulary (OOV) words.
So how embedding and de-embedding for a translation language model work? I want the language model of the example to include the de-embedding part; as we know in classification language models we may only concern about the embedding part as the output is not an embedding vector, and does not need to be converted to a string(word)
I don't know what ways are used to embed those OOV words? I guess one way is to use some main word2vec and another complementary fastText for OOV words, or instead some complementary generic way of embedding like
nn.embedding in pytorch. This won't make any problem in the classification models like sentiment analysis or Text classification.
Question 1: what are other methods that can be used for embedding an OOV word while we want to use word2vec?
but in the models which Their output is eventually gonna be a word (like translation or text summarization), I have some concerns as below:
Note we know the output of a translation language model is a sequence of embedding vectors to correspondent output word.
Note the way that we convert those embedding vectors to words, usually is by finding the closest embedding Vector in the embedding Matrix, either word2vec or fastText or de-embedder of
nn.embedding. Also note this way eventually result, some Vector and from there, by its index, we would get a word.
as we have two embedding models, so when we are the step which have received the language model output embedding vector, we need first distinguish, this output embedding vector Should be de-embedded from which embedding model (word2vec or fastText)? note as I said eventually the next action (finding closest embedding) would result some word.
so I guess one way maybe is find the closest vectors for 2 embedding models, also we their similarity score to output embedding vector of the language model, and know to de-embed from which one.
question 2: so how this problem to distinguish to which embedding model to de-embed from is handled? I guess there should be more sophisticated methods in Practice so if there are please tell me?
question 3: do u know a model which have implemented my solutions (one for embedding by complementary embedding model, 2nd one distinguishing by similarity score)?