It's often assumed in literature that BERT embeddings are contextual representations of the corresponding word. That is, if the 5th word is "cold", then the 5th BERT embedding is a representation of that word, using context to disambiguate the word (e.g. determine whether it's to do with the illness or temperature).
However, because of the self-attention encoder layers, this embedding can in theory incorporate information from any of the other words in the text. BERT is trained using masked language modelling (MLM), which would encourage each embedding to learn enough to predict the corresponding word. But why wouldn't it contain additional information from other words? In other words, is there any reason to believe the BERT embeddings for different words contain well-separated information?