I have the following assignment.

I can't understand the b part of this question in my assignment. I have completed the 1st part and understand the maths behind it, but the 2nd part has me stumped.

I looked up ridge functions and they basically map real vectors to a single real value, from what I understood. For that reason, I considered that the activation function has to be one that ranges over the real numbers, but that still doesn't clear my doubts.

I don't need a full answer just an explanation of the question will be very helpful, here's some text from the book I'm referring( Russel and Norvig), though I couldn't really grasp how this would help me choose an activation function.

Before delving into learning rules, let us look at the ways in which networks generate complicated functions. First, remember that each unit in a sigmoid network represents a soft threshold in its input space, as shown in Figure 18.17(c) (page 726). With one hidden layer and one output layer, as in Figure 18.20(b), each output unit computes a soft-thresholded linear combination of several such functions. For example, by adding two opposite-facing soft threshold functions and thresholding the result, we can obtain a “ridge” function as shown in Figure 18.23(a). Combining two such ridges at right angles to each other (i.e., combining the outputs from four hidden units), we obtain a “bump” as shown in Figure 18.23(b).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.