It is clear that word positions are essential for the meaning of a sentence, and so are essential when feeding a sentence (= sequence of words) as a matrix of word embedding vectors into a transformer. I also have understood roughly how positions are encoded, but what I did not understand in the very begining is why just creating a matrix consisting of a number of word embedding vectors (as columns) with the columns in the same order as the words in the sentence does not suffice. A matrix with permutated columns obviously would "mean" something different - and sometimes nothing at all - like a pixel matrix would change its "meaning" when we permutated some pixel columns. Is there an intuitive explanation why position encoding vectors have to be added to the word embedding vectors. Why and how would the position information (which is still present in the input matrix) get lost otherwise?
(I have learned in the meanwhile that transformers are in general not permutation invariant, but that there are transformers that are: set transformers.)