When I was learning about neural networks, I saw that a complex neural network can understand the MNIST dataset and a simple convolution network can also understand the same. So I would like to know if we can achieve a CNN's functionality with just using a simple neural network without the convolution layer and if we can then how to convert a CNN into an ANN.
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
The convolutional aspect of a CNN comes purely from the connections between layers. Instead of a fully-connected network, which can be difficult to train and tends to overfit more, the convolutional network utilizes hierarchical patterns in the data to limit the number of connections - a local edge detection feature in an image analysis network, for example, only needs input from a small number of local pixels, not the entire image. But in principle, you could assign weights to a fully-connected network to perfectly mimic a convolutional one - you just set the weights of the unneeded connections to zero. Because a general ANN has all the connections present in a CNN plus more, it can do anything a CNN can do plus more, although the training can be more difficult.
-
$\begingroup$ Might be worth showing with an example, just how crazy the numbers get if you try to do this with a typical small image size and multi-layer CNN. It really demonstrates how efficient CNN approach is $\endgroup$ Jun 11, 2020 at 18:45
-
$\begingroup$ Yeah I agree with @NeilSlater, I didn't quite get the "utilizing of hierarchical patterns" part of the answer. Can you please explain that part in brief with an example? $\endgroup$ Jun 12, 2020 at 2:29
$\begingroup$
$\endgroup$
It can be argued that CNN will outperform a fully connected network if they have the same structure (number of neurons).
Normal neural networks can probably learn to detect things like CNNs, but the task would be a lot more computationally expensive. In a CNN, all neurons in a feature maps share the same parameters, so if CNN learns to recognize a pattern in one location, it can detect the pattern in any other location. Furthermore, CNNs take into account the fact that pixels that are closer in proximity with each other are more heavily related than the pixels that are further apart, this information is lost in a Normal neural network.
Read More here.
