2
$\begingroup$

In the diagram below, although the flow of information happens from the input to output layer, the labeling of weights appears reverse. Eg: For the arrow flowing from X3 to the fourth hidden layer node has the weight labeled as W(1,0) and W(4,3) instead of W(0,1) and W(3,4) which would indicate data flowing from the 3rd node of the 0'th layer to the 4th node of the 1st layer.

enter image description here

One of my neural networks teachers did not emphasize on this convention at all. Another teacher made it a point to emphasize on it.

Is there a reason there is such an un-intuitive convention and is there really a convention?

$\endgroup$
  • 1
    $\begingroup$ Very unimportant detail, may vary from person to person, layout to layout. $\endgroup$ – DuttaA Mar 6 '18 at 7:27
  • 1
    $\begingroup$ @DuttaA: it should be always welcome student who thinks about things and arise questions to himself or publicly. $\endgroup$ – pasaba por aqui Mar 6 '18 at 8:30
  • 1
    $\begingroup$ @pasabaporaqui Humans,lets get serious here.someone in Microsoft is working on ai project,however if it requires him/her to query through SE knowledge base,then we shouldn't bring here students course work but rather world real problems to solve, out of school.Humans,hope this can save your planetary civilisation. $\endgroup$ – quintumnia Mar 6 '18 at 9:24
  • 1
    $\begingroup$ @pasabaporaqui If you analyze and comprehend this question very well,then it's right fit in cross validated community for effective feedback.Try to analyze it ! $\endgroup$ – quintumnia Mar 6 '18 at 10:08
  • 1
    $\begingroup$ Guys, students are the ones going to spread false info in the professional world if concepts and standards are not clear. I am a seasoned StackExchange user and a working professional doing a part time course. A proper answer to this question will ensure that anyone Googling this question in future wont end up with a nonsensical concept. I wanted to know if it is a standard notation uniformly used and recognized across the world. $\endgroup$ – Nav Mar 6 '18 at 12:50
2
$\begingroup$

When the system grows matrix notation is used, as a=Wx, being a (input to activation function in hidden layer) and x (values from input layer) column vectors, transpose of (a1,a2,...a_m) and (x1,x2,...,x_n), and W a m-by-n matrix of dimensions m rows and n columns. The standard way to denote matrix elements is w(i,j) where "i" is the row number and "j" column number:

enter image description here (from wiki)

For this reason, the weight that applies to h4 from x3 is element in row 4 column 3 of the matrix W, that is, W(4,3) ( as your teachers advocates but with a sad lack of ability to explain ).

In your example:

enter image description here

Note: things are a few more complex when x1, x2, ... are itself vectors, but final conclusion is the same.

( PS: URGENT to allow latex notation on this stack exchange ! )

$\endgroup$
  • $\begingroup$ Indeed there seems to be a naming convention but a more elaborate and clear explanation would be appreciated. Even the author here has not explained the reason properly neuralnetworksanddeeplearning.com/chap2.html $\endgroup$ – Nav Mar 7 '18 at 15:49
  • $\begingroup$ Which part of current explanation do you think must be expanded? $\endgroup$ – pasaba por aqui Mar 7 '18 at 16:03
  • $\begingroup$ Depicting the weights of the ANN in my question as matrices could make it more intuitive. Thanks. $\endgroup$ – Nav Mar 7 '18 at 16:26
  • $\begingroup$ @Nav: answer edited $\endgroup$ – pasaba por aqui Mar 7 '18 at 16:36
  • $\begingroup$ Lovely! That's a lot clearer now. Thanks Pasaba. $\endgroup$ – Nav Mar 8 '18 at 8:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.