People say embedding is necessary in NLP because if using just the word indices, the efficiency is not high as similar words are supposed to be related to each other. However, I still don't truly get it why.
The subword-based embedding (aka syllable-based embedding) is understandable, for example:
biology --> bio-lo-gy biologist --> bio-lo-gist
For the 2 words above, when turning them into syllable-based embeddings, it's good because the 2 words will be related to each other due to the sharing syllables:
However, it's hard to understand the
autoencoder, it turns an index value into vector, then feed these vectors to DNN. Autoencoder can turn vectors back to words too.
How does autoencoder make words related to each other?