In the famous Deep Learning book by Goodfellow et al., it is mentioned on page 169 in the caption of Figure 6.1 that

The linear model can now describe the function as increasing in $h_1$ and decreasing in $h_2$

I don’t understand what they mean by that, both visually and numerically. I can follow the matrix multiplications ReLU, etc. I see that given the value of the $w$ and biases they specify, the math works, i.e., the net gives the correct output for any input.

But what do they mean by "the function"? If it were a function of both $h_1$ and $h_2$ as input arguments, it would be a surface (a plane I guess), or are they talking about 2 lines? In that case which lines? What slope and y-intercept?

It is so annoying that this is not explicitly given with a lot of details. Can anyone clarify it with as many steps graphs and details as possible?

(Why do they talk separately of $x_1$ and $x_2$ also. As they do for $h_1$ and $h_2$. This sounds strange to me, since $x_1$ and $x_2$ are components of a 2D vector, which, in my view, must be considered jointly for the XOR. What use is there to try to consider them separately? For me, what happens with $x_1$ is meaningless is not considered at the same time with what happens with $x_2$. but of course, I am missing something)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.