1
$\begingroup$

After each epoch, Keras provides the following evaluations (depending on how the model is compiled):

  1. train_accuracy
  2. train_loss
  3. validation_loss
  4. validation_accuracy

Keras evaluates the performance of the model using the validation set at the end of each epoch. But how does Keras do this? Assume that we are performing binary classification and using a binary loss function. Assume that there are 100 images in the validation set.

QUESTION: Can the accuracy can be found by simply calculating accuracy for each item and then average them (/100)?

CONTEXT: I want to implement a custom loss function that calculates the loss towards reducing validation loss (increasing validation accuracy). Therefore, I need to understand how the accuracy is calculated for the validation set.

$\endgroup$
2
  • $\begingroup$ Keras is a powerful tool that does many things. Some more automated than others. It would be best to read and understand the code, referring to the documentation if necessary (I presume you're using someone else's code?). In any case, we would need more information about what functions are being used before we can tell. $\endgroup$
    – Martino
    Commented Aug 16, 2022 at 9:25
  • $\begingroup$ From what I understand, the reason for your question is that are trying to calculate a loss from the validation set. Please note that if you were to do such a thing, that is no longer a validation set, but a second train set. A validation set is specifically used to monitor the performance without learning from that set of data. $\endgroup$
    – Kroshtan
    Commented Nov 8, 2022 at 8:03

1 Answer 1

2
$\begingroup$

After each epoch, the model is applied to the features of the validation set (x_val) to predict the probability of the labels. Then, the probabilities are thresholded to generate the predicted label (y_pred). Finally, the accuracy is calculated as follows:

$$ Accuracy\ =\ \frac{Concordant\ samples}{Total\ samples} $$

, where concordant samples are those for which the predicted (y_pred) and true labels (y_val) agree.

Assume we have a binary classification task and 5 validation samples. If y_pred = [0, 1, 1, 0, 1] and y_val = [0, 1, 1, 1, 0], then there are 3 concordant samples, giving an accuracy of 3/5 = 60%.

$\endgroup$
2
  • $\begingroup$ Thanks for your reply. But the thing is that I need a more deep explanation as I am writing a custom loss function that requires to optimized toward a value in each epoch. Do you have any idea? $\endgroup$
    – David
    Commented Nov 7, 2022 at 18:28
  • 1
    $\begingroup$ Based on the information that you have provided, this is as deep an explanation as anyone would be able to provide you. $\endgroup$ Commented Nov 7, 2022 at 20:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .