I read somewhere that a Multilayer Perceptron is a recursive function in its forward propagation phase. I am not sure, what is the recursive part? For me, I would see a MLP as a chained function. So, it would nice anyone could relate a MLP to a recursive function.

  • Probably wrong... Although RNN maybe perceived as recursive to some....And NN maybe designed as a recursive function but I am not sure of anyone does that – DuttaA Aug 15 at 16:05
  • Are you sure it was only talking about the forward pass? The full training process could be modeled as a recursive process (although it usually isn't), but the forward pass by itself, not so much. – Ray Aug 15 at 21:08
  • Every iterative computing task can be realized in a recursive function (preferably tail recursive to automatic memory allocation) or using a loop, with appropriate iteration technique. That is why functional programning languages and collections libraries have iterators. It was found that the idea of tail recursion was only comprehensible to a small subset of those that program, so LISPers use recursion more frequently and with greater efficiency than Java programmers, who tend to use the Iterator interface or its sub interfaces. – FauChristian Aug 15 at 21:24
  • That applies to forward signal propagation during training or in field use, back propagation of correction during training, functions that perform training epochs, or any small function that deals with lists from within these higher calls. ~~ Iterating through horizontal pixel rows could be done recursively (although it would be obscure) and traversing a tree could be done with iteration (although recursion is much more transparent an approach). – FauChristian Aug 15 at 21:26

Inherently, no. The MLP is just a data structure. It represents a function, but a standard MLP is just representing an input-output mapping, and there's no recursive structure to it.

On the other hand, possibly your source is referring to the common algorithms that operate over MLPs, specifically forward propagation for prediction and back propagation for training. Both of these algorithms are easy to think about recursively, with each node performing a sort of recursive call with its children or parents as the target, and some useful information about activations or errors attached. I actually encourage my students to implement it recursively for this reason, even though it's probably not the most efficient solution.

Sure, you can define plenty of things we don't generally need to regard as recursive as so. An MLP is just a series of functions applied to its input. This can be loosely formulated as

$$ o_n = f(o_{n-1})$$

Where $o_n$ is the output of layer $n$.

But this clearly doesn't reveal, much does it?

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.