4
$\begingroup$

Recurrent neural networks, abbreviated as RNNs, are widely used in deep learning literature, especially for text processing.

Are they related to recursive neural networks in any way?

I am asking for the general/special relationship that enables us to view the one in terms of another if possible.

$\endgroup$
3
  • $\begingroup$ The Wikipedia articles on the two types of network look like a good place for someone to start an answer. $\endgroup$ Dec 17, 2021 at 10:53
  • $\begingroup$ @NeilSlater They seem to be different based on that, but at some places (like proofs) they are referring to the same... $\endgroup$
    – hanugm
    Dec 17, 2021 at 10:58
  • $\begingroup$ Form the articles, recurrent are a sub-type of recursvive. I hope that someone who knows more about this can answer in depth. $\endgroup$ Dec 17, 2021 at 11:16

1 Answer 1

0
$\begingroup$

Yes, recursive neural networks (recursive NN) are related to recurrent neural networks (RNNs), because they generalize the latter (at least, structurally), as stated in section 10.6 of the Deep Learning book, similar to the way that a tree generalizes a list/array.

So, in a list/array, element $e_1$ is connected to element $e_2$, which is connected to element $e_3$, and so on. In a tree, element $e_1$ can be connected to more than one other element (node), and the same applies to the other elements, provided there are no cycles, otherwise, you get a graph, which is a generalization of a tree.

If these explanations weren't yet clear, the best way to understand the difference between the two is to look at the diagrams of an RNN and a recursive NN.

Here's an RNN.

enter image description here

The caption should describe the diagram (in case it is not clear). The important thing to note is that, on the left, we have an RNN, which looks like a list.

Here's a recursive NN.

enter image description here

which looks like a tree.

There are many variations of RNNs (e.g. Bidirectional RNNs, multi-layered RNNs, LSTMs, or GRUs) and recursive NNs (e.g. that attempt to have balanced trees, similar to the way that a Red-Black Tree is a balanced binary search tree), so we cannot give all the details about the differences between these two approaches.

However, a few things should be kept in mind

  • recursive NNs might be used in the context of parse trees
  • recursive NNs have been used to process data structures as input to neural nets
  • nodes in recursive NNs don't necessarily perform a linear operation followed by a non-linearity
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .