In a convolutional neural network (CNN), since the RGB values get multiplied in the first convolutional layer, does this mean that color is essentially only extracted in the very first layer?

Snippets from Stanford CS231n Chapter on CNN:

[...] One dangerous pitfall that can be easily noticed with this visualization is that some activation maps may be all zero for many different inputs, which can indicate dead filters, and can be a symptom of high learning rates [...] Typical-looking activations on the first CONV layer (left), and the 5th CONV layer (right) of a trained AlexNet looking at a picture of a cat. Every box shows an activation map corresponding to some filter. Notice that the activations are sparse (most values are zero, in this visualization shown in black) and mostly local.

  • $\begingroup$ Welcome to ai.se...this link might help ai.stackexchange.com/questions/1479/… $\endgroup$
    – user9947
    Mar 19 '18 at 6:07
  • $\begingroup$ Thanks for the text snippet and link! I've taken the liberty of adding the "ai-basics" tag, as it seemed appropriate for this question. $\endgroup$
    – DukeZhou
    Mar 19 '18 at 20:36

Neural networks are all about taking raw input data (RGB values and pixel location) and learning useful features that are relevant to some downstream task. This process of aggregating raw inputs into higher-level features can start at the first layer past the inputs.

So yes, only the first layer of the network is using the actual raw color information from the image. Beyond that, the network has already started to put together nearby pixels and disparate color channels in order to find more complex patterns. Deeper layers in the neural network typically do further aggregation on features learned in earlier layers, rather than taking raw color information as input.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.