0
$\begingroup$

I was reading Goodfellow. At the start of the text it was mentioned that there are two ways to represent depth of a deep neural network. One is using the depth of the computation graph and the other is using the depth of the chain of features/concepts that is learned by NN.

Don't every layer in NN represent a concept so the depth would be the number of the layers, but then each layer also does calculation on input from previous layer so the length of computation graph will be same as number of layers. So both representation of depth will be same as number of layers in NN.

Or is it that features can also be represented using more that one layers and not individual layers.

Maybe I have misunderstood the concept of NN.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

You cannot reason in a mathematical way over features in my opinion, as they are not defined. However, you can think of deep neurons as a hierarchy of always more high level concepts, as observed in the human brain.

For example consider the task of identifying animals: the first layer, can at most identify simple stuff like edges, then the second layer combines edges and finds corner, the third layer might combine edges to find paws/ears/mouths and so on

This is what he ideally describes as features, but there is nothing stopping the net to learn features that makes no sense for us humans, or not to learn anything on a layer and just propagate the identity

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .