Tag Info

Accepted

Why does the transformer do better than RNN and LSTM in long-range context dependencies?

I'll list some bullet points of the main innovations introduced by transformers , followed by bullet points of the main characteristics of the other architectures you mentioned, so we can then ...
• 3,433

• 1,210
Accepted

• 4,016

Are embeddings in multi-lingual language models comparable across languages?

Embeddings generated by transformers like Bert or XLM-R are fundamentally different from embeddings learned through language models like GloVe or Word2Vec. The latter are static, i.e. they are just ...
• 3,433