One way to reduce the computational complexity of hidden state recurrences is to connect a unit's hidden state to the prior unit's output rather than its hidden state. The resulting RNN has a lower capacity than the architecture discussed previously, but different time steps are now decoupled and can be trained in parallel.
This is from page 595 of the book Machine Learning for Algorithmic Trading. And this chapter is about RNN.
I don't quite understand why it says now the architecture can be trained in parallel, because from what I can see it still requires output from the prior unit.