Why feeding the correct output as input during training of seq2seq models?

Question

So, I've read about seq2seq for time-series and it seemed really promising, but when I went to implement it, all the tutorial I've found use the correct output as input to the decoder phase during training, instead of using the actual prediction made by the cell before it. Is there a reason to why not do the later?

I've been using the tutorial from here: https://weiminwang.blog/2017/09/29/multivariate-time-series-forecast-using-seq2seq-in-tensorflow/

But all the other tutorials that I've found followed the same principle.

Answer 1

The reason why you would use the ground truth as input to the decoder is to lean the testing distribution. From what I've seen so far most of the papers are using scheduled sampling (Bengio et al.). Meaning that you will introduce a new term p which will be the probability of the network to feed as input its own prediction. Initially p will be very small so that the network will use the ground truth, but later the more iterations passes by, the probability will decrease and the network will start using its own predictions.