2
$\begingroup$

I've trained a neural network that can predict the $(n+1)^{th}$ element in a sequence, given the $n^{th}$ element.

It does a pretty good job doing this, with very little error.

The problem emerges when I start using the network recursively. When I feed in the $n^{th}$ element, I get the $(n+1)^{th}$ element with very little error, as I said earlier. I feed the $(n+1)^{th}$ element (the prediction) back into the network to get the $(n+2)^{th}$ element. I do this multiple times to get whatever element I want to get.

The issue is that each time I feed in a past prediction to get a new prediction, the tiny error in the previous prediction is compounded. The error increases exponentially as I predict successive elements in the sequence.

This was a problem I anticipated before I even began training the model. I figured that I'd be able to make the model so accurate that this error buildup wouldn't be noticeable until the model got far into the predicted sequence. I severely underestimated the exponential error buildup, and I don't think it's possible for me to make my model more accurate than it already is.

I figured that this is a problem that many others would've faced in the past, but I haven't found any good solutions to the problem. Are there useful tricks/methods by which I can fix the issue?

Any help would be much appreciated.

Thank you.

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.