Desperate trying to understand something for couple of weeks. All those questions are actually one big question.Please help me. Time-codes and screens in my question refer to this great(IMHO) 3d explanation:
.... Here is the case: Say I have 2 inputs (lets call them X1 and X2) into my ANN. Say X1= persons age and X2=years of education.
1) First question: do I plug those numbers as is or normalize them 0-1 as a "preprocessing"?
Thus when I plug in my variables, like in regression I will get on a Z axis my output. So the second question is: am I right up to here?
-----------------From here come real important couple of questions.
3) My output (before I plug it into the activation function) is just a simple number, IT IS NOT A PLANE and NOT A SURFACE, but a simple scalar, without any sigh on it coming from 2d surface in a 3d space(though it does come from there). Thus, when I plug this number (which was Z value in a previous step) into the activation function (say sigmoid) my number enters there in to the X axis, and we get as an output some Y value. As I understand this operation was totally 2d operation, is was 2d sigmoid and not some kind of 3dsigmoidal surface.
4)Now lets say that I was right in the previous step and as an output from the activation function I do get a simple number not a 2d surface and not a 3d one. I just have some number like I had in the very beginning of the ANN as an input (age, education etc). If i want to add another layer of neurons, this very number enters there as is not telling any one the "secret" that it was created by some kind of sigmoid. In this next layer this number is about to take similar transformations as it happened to age and education in a previous layer, it is going to be Xn in just the same scenario: sigmoid(XnWn+XmWm=output) and in the end we will get once again just a number. If I am right, why in the movie they say (time-code 14:50 ) that when we add together two activation functions we get something unlinear. They show result of such "addition" first as 2d (time-code 14:50 and 14:58). So, Here comes my question: how come that they "add" two activation functions, if to the second activation function reaches just a simple number as said above he is not telling any one the "secret" that it was created by some kind of sigmoid.
5) And then again, they show this addition of 3d surfaces (time-code 19:39 )
How it is possible? I mean again there should not happen any addition of surfaces, because no surface passes to next step but a number. What do I miss?