Usually, when I evaluate() a model, I would get a single loss that is already averaged over all samples. How do I get the loss per each sample and return all of them?

E.g. if my dataset has 100 samples, I want to get 100 losses, for each of the sample instead of 1 averaged loss.

  • $\begingroup$ This is the type of question that is perfect for Stack Overflow, given that this is just a programming issue. Please, take a look at ai.stackexchange.com/help/on-topic to know more about our scope. $\endgroup$
    – nbro
    Jan 12 at 10:16

You cannot achieve that from a single model.evaluate(...) call. But you could always do evals = [model.eval(X[i:i+1], Y[i:i+1]) for i in range(len(X))].

Note if you get the tensors yourself, you can get the backend session from keras.backend.get_session() and get anything you want by making your own sess.run() calls

  • $\begingroup$ That should be a very slow way of evaluating because you don't compute things in parallel. I want to evaluate with batches. $\endgroup$
    – off99555
    Dec 19 '19 at 8:43
  • $\begingroup$ @off99555 using the .evaluate method sure, but thats because of how it's interfaced to return aggregates. On the other hand, using the sess method or using .predict and computing the metrics on your own will still let you batchify it (in other words parallelize), you just can't if your forcing .evaluate as OP asked about. $\endgroup$
    – mshlis
    Dec 19 '19 at 8:47

Not the answer you're looking for? Browse other questions tagged or ask your own question.