0
$\begingroup$

I'm starting to study RNN for a project of video prediction, but I encounter these CNN-LSTM models. Initially, I thought that is another name for RNN, but I think I get it wrong. Since I'm a beginner can someone explain me the difference between them?

$\endgroup$
1
  • 2
    $\begingroup$ Why would "CNN-LSTM" be another name for RNN, when it doesn't even have RNN in it? Can you clarify this? What is your knowledge of RNNs and CNNs? Do you know what an LSTM is? $\endgroup$
    – nbro
    Commented Apr 15, 2022 at 12:30

1 Answer 1

2
$\begingroup$

An LSTM is a specific type of RNN. So let's just focus on the CNN part in CNN-LSTM.

What's the difference between a plain RNN and a CNN-RNN, (more generally called convolutional RNN or ConvRNN)?

The equations which define a vanilla RNN are (I'm omitting a bias term for clarity):

$h_t = \sigma_h(W_{hx}x_t + W_{hh}h_{t-1})$

$y_t = \sigma_y(W_{yh}h_t)$

where

  • $h_t \in \mathbb{R}^m$ is the hidden state
  • $x_t \in \mathbb{R}^n$ the input sequence (e.g. a string of characters)
  • $y_t \in \mathbb{R}^l$ is the output vector
  • $W_{hh} \in \mathbb{R}^{m\times m}$, $W_{hx} \in \mathbb{R}^{m\times n}$ and $W_{yh} \in \mathbb{R}^{l\times m}$ are matrices of parameters
  • $\sigma_h$, and $\sigma_y$ are activation functions (such as the ReLu)

Now, in an CNN-RNN, the parameter matrices $W_{hh}$ and $W_{hx}$ are convolution matrices. We use them for input sequences which are typically better handled by convolutional neural networks, such as a sequence of images.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .