1
$\begingroup$

I am implementing neural network to train hand written digits. Following is the cost function,

cost function In log(1-(h(x)), if h(x) is 1, then it would result in log(1-1) (i.e) log(0). So im getting math error.

Im initializing the weights randomly between 10-60. Im not sure where i have to change and what i have to change! Thanks for your help.

Improve this question
$\endgroup$
  • $\begingroup$ It's multiplied by 0 as 1-y $\endgroup$ – DuttaA Jan 23 '18 at 3:50
  • $\begingroup$ what if 1-y is not zero and 1-(h(x) is zero? $\endgroup$ – Gokulakannan Jan 23 '18 at 3:53
  • $\begingroup$ This particular cost function is for sigmoid cost function...And sigmoids do 1 or 0 as per convention ....It is based on convention that y must be 1-0 and anything can't be put $\endgroup$ – DuttaA Jan 23 '18 at 5:40
1
$\begingroup$

So, firstly, for h _{Theta}(x) to be 1, the weighted sum of x (after you dot product it with Theta) would have to be literally infinity, if you're using the sigmoid function. Doesn't happen in practice, even with the rounding computers do, as we don't use big numbers to initialize our Theta matrices.

Intuitively, that'd mean you're basically more certain than one can possibly be in this universe that the label of this example should be '1'.

So if (1 - h_{Theta}(x)) = 0, 'y' is certainly 1, and so 1-y will be zero.

Secondly, as the comments note, the convention is to drop the entire right-hand-side term when y^(i) is 1. This will not cause problems when programming, due to the first point I made above.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.