2
$\begingroup$

In the diagram below, although the flow of information happens from the input to output layer, the labeling of weights appears reverse. Eg: For the arrow flowing from X3 to the fourth hidden layer node has the weight labeled as W(1,0) and W(4,3) instead of W(0,1) and W(3,4) which would indicate data flowing from the 3rd node of the 0'th layer to the 4th node of the 1st layer.

enter image description here

One of my neural networks teachers did not emphasize on this convention at all. Another teacher made it a point to emphasize on it.

Is there a reason there is such an un-intuitive convention and is there really a convention?

Improve this question
$\endgroup$
11
  • 1
    $\begingroup$ Very unimportant detail, may vary from person to person, layout to layout. $\endgroup$
    – user9947
    Mar 6, 2018 at 7:27
  • 1
    $\begingroup$ @DuttaA: it should be always welcome student who thinks about things and arise questions to himself or publicly. $\endgroup$
    – pasaba por aqui
    Mar 6, 2018 at 8:30
  • 1
    $\begingroup$ @pasabaporaqui Humans,lets get serious here.someone in Microsoft is working on ai project,however if it requires him/her to query through SE knowledge base,then we shouldn't bring here students course work but rather world real problems to solve, out of school.Humans,hope this can save your planetary civilisation. $\endgroup$
    – quintumnia
    Mar 6, 2018 at 9:24
  • 1
    $\begingroup$ @pasabaporaqui If you analyze and comprehend this question very well,then it's right fit in cross validated community for effective feedback.Try to analyze it ! $\endgroup$
    – quintumnia
    Mar 6, 2018 at 10:08
  • 1
    $\begingroup$ Guys, students are the ones going to spread false info in the professional world if concepts and standards are not clear. I am a seasoned StackExchange user and a working professional doing a part time course. A proper answer to this question will ensure that anyone Googling this question in future wont end up with a nonsensical concept. I wanted to know if it is a standard notation uniformly used and recognized across the world. $\endgroup$
    – Nav
    Mar 6, 2018 at 12:50

1 Answer 1

Reset to default
2
$\begingroup$

When the system grows matrix notation is used, as a=Wx, being a (input to activation function in hidden layer) and x (values from input layer) column vectors, transpose of (a1,a2,...a_m) and (x1,x2,...,x_n), and W a m-by-n matrix of dimensions m rows and n columns. The standard way to denote matrix elements is w(i,j) where "i" is the row number and "j" column number:

enter image description here (from wiki)

For this reason, the weight that applies to h4 from x3 is element in row 4 column 3 of the matrix W, that is, W(4,3) ( as your teachers advocates but with a sad lack of ability to explain ).

In your example:

enter image description here

Note: things are a few more complex when x1, x2, ... are itself vectors, but final conclusion is the same.

( PS: URGENT to allow latex notation on this stack exchange ! )

Improve this answer
$\endgroup$
5
  • $\begingroup$ Indeed there seems to be a naming convention but a more elaborate and clear explanation would be appreciated. Even the author here has not explained the reason properly neuralnetworksanddeeplearning.com/chap2.html $\endgroup$
    – Nav
    Mar 7, 2018 at 15:49
  • $\begingroup$ Which part of current explanation do you think must be expanded? $\endgroup$
    – pasaba por aqui
    Mar 7, 2018 at 16:03
  • $\begingroup$ Depicting the weights of the ANN in my question as matrices could make it more intuitive. Thanks. $\endgroup$
    – Nav
    Mar 7, 2018 at 16:26
  • $\begingroup$ @Nav: answer edited $\endgroup$
    – pasaba por aqui
    Mar 7, 2018 at 16:36
  • $\begingroup$ Lovely! That's a lot clearer now. Thanks Pasaba. $\endgroup$
    – Nav
    Mar 8, 2018 at 8:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .