3
$\begingroup$

Problem Background

I am working with a problem, which requires a character-level, deep learning model. Previously I was working with word-level deep NLP (Natural Language Processing) models, and in these models almost always embedding encoding was used to represent given word in a lower-dimensional vector form. Furthermore, such embedding encoding allowed for putting similar words near themselves in the new lower-dimensional vector representation (i.e. man and woman vectors were near themselves in the vector space) which improved learning. Nevertheless, I often see that people use embedding encoding in character level NLP models. Even if the character-level one-hot encoding vectors are quite small in comparison to word-level one-hot encoding vectors (about 36 to 32k rows). Furthermore, there is no much correlation between characters, there is no something like "similar characters" in comparison to similar words, therefore some characters in comparison to other shouldn't be put near themselves.

Question Why embedding encoding is used in the character-level NLP models?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.