Features Map convolutional neural network

Question

I have a question about convolutional neural newtork.

Consider this image: conv example

We have a part of an input matrix and a filter. Ok, now we can do the convolution and the result is a scalar, if it is a large number the future was found otherwise no. So, the features map is a matrix where each number indicates the points where a feature was found. I understood this. The output of this convolution is a features map (after activation function). My misunderstanding start here.

The next convolution, will find another feature. I don't understand how this filter, can find a new feature from a representation of where the previous feature was found. The feature maps, output of the first convolution is a rappresentation of indicates only where the features were found.

Shouldn't be the features found instead of a number that indicates how much was found this feature?? How exactly does this work?

Answer 1

Since the kernel size is limited. A feature can encompass more than 3 pixels or say 5 pixels, for that we cannot find it in 1 layer. So by adding 2 layers and having an activation function that is not like a step function which you would prefer, we extend the feature to 5 or 9 pixels. And with n layers, a feature covering 2n+1, or 4n+1 can be seen.

PS. You have some spelling errors. Take your time to frame well. Welcome!

Answer 2

Well, let's start from the beginning, convolution is an operation that is used not only in neural networks. Actually, people were using it in signal processing long before neural networks. Convolution is also used in image processing, in photoshop for example. So let's start from image processing. Take a look at this link. Here you can see how we can use conv op to process an image. If you choose an outline kernel you can basically extract edges from that image using 3x3 kernel and applying it on the image.

So when you train your network you change parameters of that kernel in a way that minimizes your loss function. However, it is a really hard question what representations your CNN will learn. But for the sake of the example, let's assume that in the first layer your network learned to extract edges from an input image. Then you feed that feature map to the second layer and the second layer now should learn to extract new features from those edges provided by the first layer. For example, it can learn to detect horizontal edges or vertical one, so if you trying to recognize cars vs pedestrians, for example, cars will have more horizontal edges when pedestrians will have more vertical edges. Also note, that we usually make several feature maps per layer, so in this example, 2nd layer can output 16 feature maps produced by different kernels with different parameters so it could recognize vertical edges, horizontal edges, round one and so on. After that you feed the output of the 2nd layer, let's say, to the fully connected layer and the fc layer will say "ok, I see that previous layer recognized horizontal edges so I must output higher probability for the car, but if 2nd layer recognized vertical edges I would output higher probability for the pedestrian".

Of course, this is oversimplified and in reality, representations learned by the network will be way more abstract, but I think just for the sake of gaining intuition this is an acceptable example. But if you are looking to gain a deeper knowledge about what CNNs learn in their layers read this article, not an easy reading, especially if you don't know much about CNNs, but it is quite insightful. Also if you don't want to read that article here is a video presentation made by the authors of that paper on youtube.

Answer 3

Three reasons:

1. Limited field of view by one small kernel
2. Layers of abstraction and combinations of intermediate features
3. Non-linearity of decision boundary

The previous two answers by caissalover and Andrew got the first two points. Let's elaborate on the third:

Convolution is a linear transformation of the input. But not everything can be properly expressed with linear operations. This is why we apply non-linear activation functions (e.g. ReLU) in between.