I'm trying to understand the difference between the concept of self-attention and multi-head attention. The latter is not actually too clear to me.
I understand that, in the case of self-attention, we start with a feature matrix $X \in \mathbb{R}^{n \times d}$, and then we use the same linear transformation $W$ to produce
\begin{align} Q &= XW \\ K &= XW \\ V &= XW \end{align}
and then we compute the following
$$X' = \text{softmax} \left(\frac{Q\cdot K^T}{\sqrt{d}} \right)V$$
where $X' \in \mathbb{R}^{n \times d}$ is a new version of the input matrix, where the pairwise interactions between the points will be encoded.
What is multi-head attention doing, from a mathematical point of view, and what's the difference? I know we are going to use three different linear transformations in this case (so no weight-sharing), but what exactly will be encoded using three different $W$? Maybe it's more the conceptual view that it's not too clear in this case.