What is the intuition behind position-encoding?

Question

It is clear that word positions are essential for the meaning of a sentence, and so are essential when feeding a sentence (= sequence of words) as a matrix of word embedding vectors into a transformer. I also have understood roughly how positions are encoded, but what I did not understand in the very begining is why just creating a matrix consisting of a number of word embedding vectors (as columns) with the columns in the same order as the words in the sentence does not suffice. A matrix with permutated columns obviously would "mean" something different - and sometimes nothing at all - like a pixel matrix would change its "meaning" when we permutated some pixel columns. Is there an intuitive explanation why position encoding vectors have to be added to the word embedding vectors. Why and how would the position information (which is still present in the input matrix) get lost otherwise?

(I have learned in the meanwhile that transformers are in general not permutation invariant, but that there are transformers that are: set transformers.)

Answer 1

I understand the confusion. Although transformers are autoregressive (they predict something based on past information), they are not recurrent (do not have hidden states).

In fact, think of a transformer as a multi-layer perceptron. Each input neuron is fully connected to every neuron in the first hidden layer. If you would swap the places of the first and the second input neuron (including their associated weights), all the connections/weights stay the same, but the order of the neurons is different.