Why do we use a delay when feeding our input data to the echo state network?

Question

I'm new to working with neural networks and have recently began implementing neural networks for time series forecasting in some of my work. I've been particularly using Echo State Networks and have been doing some reading to understand how they work. For the most part, things seem pretty straight forward, but I'm confused as to why we use a 'delay' when feeding our input data (the delay concept mentioned in the paper Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication)?

I'm looking at some source code on Github, and they implement this delay as well (they feed two arrays inputData & targetData, into the network where one is delayed by one element relative to the other). I am noticing that the larger the delay, the worse the fit.

Why is this done? My interest is eventually to forecast past sample data.

Answer 1

It is not a 'delay' but a 'look ahead' as some people call it. This is how far in the future the model can forecast (or predict).

In reference to the GitHub source, the code uses a look ahead ('delay') of one. The echo state network learns on the 'inputData' using a future version of itself called 'targetData'.

Here is an analogy. It is early on Friday, June 5, and I have the last 51 days worth of Amazon stock prices. I can use the first 50 days of prices, which go up to and include Wednesday, June 3, to train a network to predict Thursday, June 4. If I have a lot of historical data on Amazon, I can train many 50 day windows. Each 50 day window would have as its target a stock price one day in the future.

Performance generally degrades the further into the future you try to forecast. One reason is due to the build up of error over time. This is true in life. We usually know fairly accurately what we will do tomorrow, less so next week, and it would be a wild guess to predict what we will be doing in ten years.