What is the current state-of-the-art in unsupervised cross-lingual representation learning?
The blog post Unsupervised Cross-lingual Representation Learning (2019), the related paper and slides by Sebastian Ruder (a researcher currently at DeepMind) summarize what you are looking for. In fact, the authors write
We will introduce researchers to state-of-the-art methods for constructing resource-light cross-lingual word representations and discuss their applicability in a broad range of downstream NLP applications, covering bilingual lexicon induction, machine translation (both neural and phrase-based), dialogue, and information retrieval tasks. We will deliver a detailed survey of the current cutting-edge methods, discuss best training and evaluation practices and usecases, and provide links to publicly available implementations, datasets, and pretrained models and word embedding collections.