Why are RNNs used in some computer vision problems?

Question

I am learning computer vision. When I was going through implementations of various computer vision projects, some OCR problems used GRU or LSTM, while some did not. I understand that RNNs are used only in problems where input data is a sequence, like audio or text.

So, in kernels of MNIST on kaggle almost no kernel has used RNNs and almost every repository for OCR on IAM dataset on GitHub has used GRU or LSTMs. Intuitively, written text in an image is a sequence, so RNNs were used. But, so is the written text in MNIST data. So, when exactly is it that RNNs(or GRUs or LSTMs) need to be used in computer vision and when don't?

Answer 1

There are tasks in computer vision where recurrent neural networks (RNNs) can be useful because there's some sequential sub-task in the main task.

For instance, in the paper Long-Term Recurrent Convolutional Networks for Visual Recognition and Description, the authors investigate the use of a neural network that is both recurrent and convolutional to solve certain computer vision tasks that also have a sequential component/part, such as video recognition tasks, image to sentence generation problems, and video narration challenges.

There are other papers that investigate the combination of convolutional and recurrent layers, such as Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition, which also has a biological motivation.