I have an essential regression task, where the input is of dimension $d$ and the output is a scalar. I think the transformer model is a good fit for this problem. In the vanilla multi-head-attention formulation, we essentially expect the key and query to have the same dimensionality. With this set-up, in the decoder, we would have dimensionality $N \times d$ for the query and $N \times 1$ for the Key.
What is the canonical way to handle this? Can I add a linear layer to "bloat up" the key?