Timeline for Seq2seq with RNNs, how is the training loop performed?
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 23, 2023 at 17:26 | comment | added | FluidMechanics Potential Flows | Thank you so much | |
| Mar 23, 2023 at 10:48 | comment | added | Hans-Peter Schrei | Yes, that's correct. During training with teacher forcing, you keep the same hidden state updating mechanism, but at each time step of the decoder, you input the correct (ground truth) word/token from the target sequence along with the hidden state, instead of using the predicted word/token from the previous time step. | |
| Mar 23, 2023 at 10:28 | comment | added | FluidMechanics Potential Flows | Oh my bad, you keep the same hidden state but you input the correct word + the hidden state instead of the wrong word + the hidden state? | |
| Mar 21, 2023 at 9:26 | comment | added | FluidMechanics Potential Flows | During training you input words instead of hidden states to the decoder? | |
| Mar 21, 2023 at 8:59 | comment | added | Hans-Peter Schrei | You don't need to change what the decoder passes to itself. Instead, you alter the input to the decoder during training. | |
| Mar 21, 2023 at 8:41 | comment | added | FluidMechanics Potential Flows | How can we do teacher forcing when the things decoder pass to themselves are hidden states? Do we need to change what they pass themselves (maybe just the last hidden state of the encoder and the raw words output)? | |
| Mar 21, 2023 at 8:34 | vote | accept | FluidMechanics Potential Flows | ||
| Mar 21, 2023 at 8:25 | history | answered | Hans-Peter Schrei | CC BY-SA 4.0 |