2
$\begingroup$

I used to treat back propagation as a black box but lately I want to understand more about it. I have used mattmuzr's and DuttA's explanaiton as a guide to hand compute a simple neural network. I have computed feed forward and back propagation to a network similar to this one with one input, one hidden and one output

enter image description here

Here are my computations

enter image description here

enter image description here

enter image description here

Is my computations correct?

*Full latex code

$\endgroup$
  • $\begingroup$ I think you should provide the structure of your NN for better understanding $\endgroup$ – DuttaA Mar 12 '18 at 2:48
  • $\begingroup$ My review concludes that your analysis is correct. Congratulations also for the presentation. Just a minor editorial: replace "b" by b1 and b2; must be b1 and b2 also evaluated ?; clarify that sigma function is sigmoid; and better write w_2*h that h*w_2 (in this way, most of your equations are applicable to h vector and W matrix). $\endgroup$ – pasaba por aqui Mar 12 '18 at 9:09
  • 1
    $\begingroup$ Looks perfect..can't write a full answer to that $\endgroup$ – DuttaA Mar 12 '18 at 12:24
  • 1
    $\begingroup$ I think you should really check out this course coursera.org/learn/machine-learning/home/welcome you won't have any more doubts, its simple and easy $\endgroup$ – DuttaA Mar 13 '18 at 10:19
  • 1
    $\begingroup$ I also do the stuff on python but since this course uses octave it's universally understood since octave is very readable, but there is another course frm the same teacher deeplearning.ai which uses python but he doesn't give intuitive understanding in that course...The course is good to understand conventions and intuitions $\endgroup$ – DuttaA Mar 13 '18 at 16:22
2
$\begingroup$

One important point I missed in first review: error is a summatory, its derivative is also a summatory.

About offsets "b": usually they are different in each cell (if not fixed to some value, as 0). Thus, replace them by b1 and b2. Moreover, they should be optimized in the same way that the weights.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.