Below is an excerpt in an instructor's manual on ML that is explaining deep neural networks, using cat recognition (what else!) from images as example. On how DL performs this feat, the excerpt said that,
Assume that the first layer returns the number of pixels that are brown/black/blue/red, and the second layer finds the most common color, and the third layer returns “cat” if previous layer had supplied “brown”. [..] Mathematically, this model would be, for the first layer, [ sum(r = 255, g=255, b=255), ..., ..., sum(r=255, g=0, b=0)] -- this is just a set of appropriately positioned relu functions (okay, for r=234, we’d need two relu functions, so two layers, but you get the idea). The second layer would be a softmax layer. The third layer is simply an identity!
Now I worked with deep nets, but I am not sure how I can structure a DL to do this. ReLu is simply a max(0,x), so how would I filter out pixel vals for example 128,128,128 and sum them up? Wouldn't the convolution layer play a role here too? What would the layout of a simple deep net be that does what is described above?