# What is the reasoning behind the number of filters in the convolution layer?

Let's assume an extreme case in which the kernel of the convolution layer takes only values 0 or 1. To capture all possible patterns in input of $$C$$ number of channels, we need $$2^{C*K_H*K_W}$$ filters, where $$(K_H, K_W)$$ is the shape of a kernel. So to process a standard RGB image with 3 input channels with 3x3 kernel, we need our layer to output $$2^{27}$$ channels. Do I correctly conclude that according to this, the standard layers of 64 to 1024 filters are only able to catch a small part of (perhaps) useful patterns?

• what is the point of assuming that extreme case? Is it only for the sake of asking or you have a practical task where you are restricted to use only 0 and 1 filters? Jan 2, 2020 at 3:50
• @voo_doo Maybe instead of extreme, I should use the word edge or least complex. The more values a kernel can take, the higher the power base and the more filters are needed to capture all the patterns. So I wanted to say that the layer with the continuous kernel should use at least more filters than the layer with the binary kernel. This question came to me when I read about wide_resnet, which improved the results by increasing the number of filters in layers. Jan 2, 2020 at 7:06

Let $$n=C*K_w*K_h$$. Then you should only need $$n$$ filters. Not $$2^n$$ to keep all the information. If you just used the rows of the identity matrix as your filters than your convolution would just be making an exact copy so it definitely wouldn't be throwing away information. On the other hand, there will be a max pooling operation. To simplify the question let's suppose we have a 3 channels, and a 1 by 1 kernel. And then let's suppose it is just one convolution followed by global max pooling. Also, let's use your assumption that it's all binary. If you have $$m$$ filters then the final output will be $$m$$ dimensional no matter how many input points you have. So clearly information is being thrown away there. But that's not such a bad thing. Throwing away irrelevant information gets us closer to the features we need the problem at hand. The parts that get thrown away by max pooling correspond to features not being found in a particular part of the image.
• Keeping information is not the goal here. The layer is supposed to add new information's about the relationship between the data in the input. To find all possible relationships between just two RGB pixels using binary kernel, we need $2^{2*3}=64$ filters. Dropping some of the information's may not be the problem in later layers working on high level features, but the first layers need to find a kernel, that basically fits all cases, because this kernel has to find some useful pattern in every single slice of the image. Jan 2, 2020 at 23:56