I've a prediction matrix(P) of dimension 3x3 and one-hot encoded label matrix(L) of dimension 3x3 as shown below.

    |0.5 0.3 0.1|      |1 0 0|
P = |0.3 0.2 0.1|  L = |0 1 0|
    |0.2 0.5 0.8|      |0 0 1|

each column in 'P' corresponds to prediction of a label in 'L'

  1. How is the BCELoss calculated using pytorch?, my experimentation by giving these two matrices as parameters to loss function yielded me poor results and pytorch's loss calculation function doesn't disclose on how loss calculation is done for this case.

  2. How is the loss averaged for each instance and across the a batch?

  3. if loss is calculated column wise and averaged for each instance and across the batch, then how can loss be backprop'd in pytorch?

Thanks in advance.


1 Answer 1

  1. BCELoss ( Binary Cross Entropy Loss) is used for binary classifier, which is a neural network that have a binary output, 0 or 1. It is not used for multi-output neural network like your case. For that kind of networks, you can use MSELoss or CrossEntropyLoss as your loss for the network.
  2. For the calculation of BCE, it is shown on pytorch documentation. https://pytorch.org/docs/stable/nn.html#BCELoss For across batch, it either sum the loss or take the mean. you can set that through the reduction="" argument.
  3. As i said, the loss is either summed or taken the mean. Then there will be a single value so pytorch can do backdrop.

Hope I can help you


Not the answer you're looking for? Browse other questions tagged .