All CNN's can be represented as Vanilla networks on the flattened image data. Just to do so, you would need A LOT of parameters (most of which would be 0) for what CNN's do freely. You can think of a CNN as reusing a filter on a masked input (which ever receptive field its looking at whatever point during the convolution) repetitively.

In other-words Fully Connected layers use all the information, so it can still learn spatial dependence as a CNN does, while CNN's for each neuron only look at a specific receptive field and will reuse that filter for all neurons in that channel. This constraint saves computation and allows wider and deeper models under some budget.

This is nice because the hypothesis of why CNN's work are, is that at each point in the network we care about looking at localized features rather than global ones and that creating a composition of these makes it so even if each neuron only relates to a handful of neurons in the previous layer, the receptive field from the initial image can still be quite large if not the whole thing.

take away: CNNs are an efficient implementation of a Vanilla NN given the locality constraint that each neuron only looks at a small localized subset of neurons from the previous layer

All CNNs can be represented as vanilla networks on the flattened image data. Just to do so, you would need A LOT of parameters (most of which would be 0) for what CNNs do freely. You can think of a CNN as reusing a filter on a masked input (whichever receptive field it's looking at whatever point during the convolution) repetitively.

In other words, fully connected layers use all the information, so it can still learn spatial dependence as a CNN does, while CNNs for each neuron only look at a specific receptive field and will reuse that filter for all neurons in that channel. This constraint saves computation and allows wider and deeper models under some budget.

This is nice because the hypothesis of why CNN's work are, is that at each point in the network we care about looking at localized features rather than global ones and that creating a composition of these makes it so even if each neuron only relates to a handful of neurons in the previous layer, the receptive field from the initial image can still be quite large if not the whole thing.

Take away: CNNs are an efficient implementation of a vanilla NN, given the locality constraint that each neuron only looks at a small localized subset of neurons from the previous layer.

All CNN's can be represented as Vanilla networks on the flattened image data. Just to do so, you would need ALOT of parameters (most of which would be 0) for what CNN's do freely. You can think of a CNN as reusing a filter on a masked input (which ever receptive field its looking at whatever point during the convolution) repetitively.

In other-words Fully Connected layers use all the information, so it can still learn spatial dependence as a CNN does, while CNN's for each neuron only look at a specific receptive field and will reuse that filter for all neurons in that channel. This constraint saves computation and allows wider and deeper models under some budget.

This is nice because the hypothesis of why CNN's work are, is that at each point in the network we care about looking at localized features rather than global ones and that creating a composition of these makes it so even if each neuron only relates to a handful of neurons in the previous layer, the receptive field from the initial image can still be quite large if not the whole thing.

take away: CNNs are an efficient implementation of a Vanilla NN given the locality constraint that each neuron only looks at a small localized subset of neurons from the previous layer

All CNN's can be represented as Vanilla networks on the flattened image data. Just to do so, you would need A LOT of parameters (most of which would be 0) for what CNN's do freely. You can think of a CNN as reusing a filter on a masked input (which ever receptive field its looking at whatever point during the convolution) repetitively.

In other-words Fully Connected layers use all the information, so it can still learn spatial dependence as a CNN does, while CNN's for each neuron only look at a specific receptive field and will reuse that filter for all neurons in that channel. This constraint saves computation and allows wider and deeper models under some budget.

This is nice because the hypothesis of why CNN's work are, is that at each point in the network we care about looking at localized features rather than global ones and that creating a composition of these makes it so even if each neuron only relates to a handful of neurons in the previous layer, the receptive field from the initial image can still be quite large if not the whole thing.

take away: CNNs are an efficient implementation of a Vanilla NN given the locality constraint that each neuron only looks at a small localized subset of neurons from the previous layer