# Transformers: how does the decoder final layer output the desired token?

In the paper Attention Is All You Need, this section confuses me:

In our model, we share the same weight matrix between the two embedding layers [in the encoding section] and the pre-softmax linear transformation [output of the decoding section]

Shouldn't the weights be different, and not the same? Here is my understanding:

For simplicity, let us use the English-to-French translation task where we have $$n^e$$ number of English words in our dictionary and $$n^f$$ number of French words.

• In the encoding layer, the input tokens are $$1$$ x $$n^e$$ one-hot vectors, and are embedded with a $$n^e$$ x $$d^{model}$$ learned embedding matrix.

• In the output of the decoding layer, the final step is a linear transformation with weight matrix $$d^{model}$$ x $$n^f$$, and then applying softmax to get the probability of each french word, and choosing the french word with the highest probability.

How is it that the $$n^e$$ x $$n^{model}$$ input embedding matrix share the same weights as the $$d^{model}$$ x $$n^f$$ decoding output linear matrix? To me, it seems more natural for both these matrices to be learned independently from each other via the training data, right? Or am I misinterpreting the paper?

It seems like they learned a single embedding matrix ($$n^e + n^f$$) x $$d^{model}$$ in dimension.