Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Questions tagged [teacher-forcing]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
29 views

Could anyone please explain this sentence about training in parallel?

One way to reduce the computational complexity of hidden state recurrences is to connect a unit's hidden state to the prior unit's output rather than its hidden state. The resulting RNN has a lower ...
user avatar
1 vote
1 answer
33 views

How to do testing for an RNN that was trained with teacher forcing only?

If an RNN is trained using only the teacher forcing, then the network takes the actual output from the previous time step as input to the hidden state the next time step. We know that the actual ...
user avatar
  • 3,059
0 votes
0 answers
22 views

Which kind of RNNs are mostly used in applications: hidden-hidden or actual output-hidden?

I came across two types of RNN while reading the chapter titled Sequence Modeling: Recurrent and Recursive Nets of the textbook named Deep Learning by Ian Goodfellow et al. First type: Recurrent ...
user avatar
  • 3,059
1 vote
0 answers
25 views

Can teacher forcing in RNN ensure Turing completeness?

RNN has the same capability as a universal Turing machine. But I am confused whether RNN holds the same capabilities if we use teacher forcing. Consider the following excerpts from paragraphs taken ...
user avatar
  • 3,059
1 vote
0 answers
31 views

Do bi-directional RNNs necessarily use 100% teacher forcing?

I typically think of teacher forcing as optional when training an RNN. We may either: use the output of time-step $t$ as the input to time-step $t+1$ use the $(t+1)$th input as the input to time-...
user avatar
3 votes
1 answer
300 views

What is teacher forcing?

In the paper Neural Programmer-Interpreters, the authors use the teacher forcing technique, but what exactly is it?
user avatar
  • 33.2k
1 vote
1 answer
81 views

Why feeding the correct output as input during training of seq2seq models?

I've read about seq2seq for time-series and it seemed really promising, but, when I went to implement it, all the tutorials I've found use the correct output as input to the decoder phase during ...
user avatar