Consider the following excerpt paragraph taken from the section titled "Recurrent Neural Networks" of the chapter 10: Sequence Modeling: Recurrent and Recursive Nets of the textbook named Deep Learning by Ian Goodfellow et al on the computational graph of some computations.
The recurrent neural network of ..... is universal in the sense that any function computable by a Turing machine can be computed by such a recurrent network of a finite size. The output can be read from the RNN after a number of time steps that is asymptotically linear in the number of time steps used by the Turing machine and asymptotically linear in the length of the input. The functions computable by a Turing machine are discrete, so these results regard exact implementation of the function, not approximations. The RNN, when used as a Turing machine, takes a binary sequence as input, and its outputs must be discretized to provide a binary output. It is possible to compute all functions in this setting using a single specific RNN of finite size. The “input” of the Turing machine is a specification of the function to be computed, so the same network that simulates this Turing machine is sufficient for all problems. The theoretical RNN used for the proof can simulate an unbounded stack by representing its activations and weights with rational numbers of unbounded precision.
This paragraph clearly explains that RNN is capable of computing any computable function exactly and is the same as the Turing machine in terms of capability.
Afaik, MLP is capable of approximating any continuous, bounded function.
So, it seems to me that RNN is more powerful in terms of the capability of computing functions than MLP. RNN can learn any function that MLP can learn and RNN can learn more than that can be learned by any MLP in general.
Am I correct? Or is there any issue in my interpretation or do more details need to be considered?
