# What do echo state networks give us over a generic RNN resevoir?

Slightly generalizing the definition in Jaeger 2001, let's define a reservoir system to be any system of the form

$$h_{t}=f(h_{t-1}, x_t)$$ $$y_t=g(Wh_t)$$

where $$f$$ and $$g$$ are fixed and $$W$$ is a learnable weight matrix. The idea is we feed a sequence of inputs $$x_t$$ into the system, which has some fixed initial state $$h_0$$, and thereby generate the sequence of outputs $$y_t$$. $$f$$ is fixed (for example, a randomly generated RNN) we can then attempt to learn $$W$$ in some way in order to get the system to have the behavior that we want.

Now we add the echo state condition: the system has the echo state condition iff for any left-infinite sequence $$...x_{-3}, x_{-2}, x_{-1}, x_0$$, there is only one sequence of states $$h_t$$ consistent with this input sequence.

Seen from this perspective, any training procedure that could be applied to an echo state system could be applied to a generic reservoir system. So what do we get out of the echo state condition? Is there some reason to think echo state systems will generalize better, or be more quickly trainable? Jaeger does not seem to attempt to argue in this direction, he just describes how to train an ESN, but as I've said, nothing about these training methods seems to require the echo state property.