2
$\begingroup$

I am building a supervised learning model and I wish to compute the log-likelihood for the training set at the point of the minimum validation error.

Initially, I was computing the sum of all the probabilities with maximum value obtained after applying softmax for each example in the training set at the point of minimum validation error but that doesn't look correct.

What is the correct formula for the log-likelihood?

$\endgroup$
1
$\begingroup$

The log-likelihood function for the training set (in general, not for deep learning in particular) will depend on your choice of loss function.

I'm guessing you're using something like a quadratic loss function for a binary classification problem, since this is a common approach. In that case, the log-likelihood is the sum of logs of the squared differences between the target label and the softmax value produced by your model. If you want to compute the log likelihood under a particular set of parameters (say, those that minimize validation error), then you just use those parameters in the model when generating softmax values.

$\endgroup$
  • $\begingroup$ I too thought on the similar lines. I thought of using negative loss likelihood loss as given here. But this gives us negative log-likelihood loss whereas we are interested in calculating only log-likelihood. Could you please clarify my doubt? $\endgroup$ – Akhilesh Pandey Aug 9 '18 at 6:16
  • $\begingroup$ I suspect that multiplying by -1 will give you the log likelihood then. Or you could minimize instead of maximizing. $\endgroup$ – John Doucette Aug 9 '18 at 13:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.