I'm trying to distinguish between the fundamental structures of the convolutional neural network and the recurrent neural network. Convolutional neural networks build a hierarchical model from the data using the convolutional layers. Recurrent neural networks build a sequential model from the data using sequential passing of information (are "recurrent layers" (to contrast with the aforementioned convolutional layers) even a thing? This website seems to have a tag for recurrent-layers, so I guess so?). The differences here aren't clear to me; specifically, it isn't clear to me how one is "hierarchical" whereas the other is "sequential", precisely how the "hierarchical" structure differs from the "sequential" structures, and how this "hierarchical" vs "sequential" distinction / inductive bias affects the suitability (effectiveness) of the models for different tasks (images vs language is the obvious case, but I'm also interested in this in a more general/abstract sense).
P.S. With regards to the "recurrent layers" question, I'm familiar with RNN architecture and memory cells (LSTM, GRU), but I've never seen the "recurrent layer" terminology, and it isn't clear to me how any part of the RNN architecture is a "recurrent layer", as opposed to the convolutional layers of CNNs, which is clearer to me as having distinct "layers" for distinct features (for instance, in an image).
P.P.S. Given the "sequential" structure, are recurrent neural networks not a hierarchical model? Do they not model things hierarchically? Or do they also model things hierarchically (although, it seems to me that this clearly isn't as clear as with convolutional neural networks)?
