2
Can the decoder in a transformer model be parallelized like the
encoder?
NO:
Your understanding is completely right. In the decoder, the output of each step is fed to the bottom decoder in the next time step, just like an LSTM.
Also, like in LSTMs, the self-attention layer needs to attend to earlier positions in the output sequence in order to compute ...
1
Answer to Q1) If sampling for next token do you need to apply mask during inference?
Yes you do! The models ability to transfer information across positions was trained in this manner, and changing it up will have unpredictable consequences. Let my try to give an example:
Tokens: 1:sally, 2:sold, 3:seashells, 4:on, 5:the, 6:____
In the above you are ...
1
Basically, it means that the "localization network" should output a set of real valued parameters (typically 6 numbers). The word "regression" doesn't bear any specific meaning.
Any network that relies on the original image as input (directly or indirectly) and outputs 6 numbers, would work. And its last layer would qualify as "regression layer" as long as ...
Only top voted, non community-wiki answers of a minimum length are eligible
