# What does it mean by bottleneck and representational bottleneck in feedforward neural networks?

Consider the following paragraph from section 2: General Design Principles of the research paper titled Rethinking the Inception Architecture for Computer Vision

Avoid representational bottlenecks, especially early in the network. Feed-forward networks can be represented by an acyclic graph from the input layer(s) to the classifier or regressor. This defines a clear direction for the information flow. For any cut separating the inputs from the outputs, one can access the amount of information passing though the cut. One should avoid bottlenecks with extreme compression. In general the representation size should gently decrease from the inputs to the outputs before reaching the final representation used for the task at hand. Theoretically, information content cannot be assessed merely by the dimensionality of the representation as it discards important factors like correlation structure; the dimensionality merely provides a rough estimate of information content.

This paragraph warned us to avoid bottlenecks and also representational bottlenecks. What does it mean by a bottleneck of/in a neural network and representational bottleneck?

• I don't feel like providing a formal answer because I am not sure whether these terms "bottlenecks" are formally defined. In fact, we had similar questions in the past, here or this (actually, I think your question is a duplicate of this last one), but it seems to me that these terms have been used more loosely. Having said that, I think the authors of that quote are using the term "representational bottleneck" as a synonym for just "bottleneck".
– nbro
Sep 9, 2021 at 13:14
• Yeah, @nbro. After understanding the terminology regarding this, I also feel that this is a duplicate one. But, while asking the question, I thought that the word bottleneck may be used for different purposes and hence I asked. Jan 5, 2022 at 23:43
• what-are-bottlenecks-in-neural-networks? Oct 28, 2022 at 3:29