I've a prediction matrix(P) of dimension 3x3 and one-hot encoded label matrix(L) of dimension 3x3 as shown below.

    |0.5 0.3 0.1|      |1 0 0|
P = |0.3 0.2 0.1|  L = |0 1 0|
    |0.2 0.5 0.8|      |0 0 1|

each column in 'P' corresponds to prediction of a label in 'L'

  1. How is the BCELoss calculated using pytorch?, my experimentation by giving these two matrices as parameters to loss function yielded me poor results and pytorch's loss calculation function doesn't disclose on how loss calculation is done for this case.

  2. How is the loss averaged for each instance and across the a batch?

  3. if loss is calculated column wise and averaged for each instance and across the batch, then how can loss be backprop'd in pytorch?

Thanks in advance.

  • 2
    $\begingroup$ Hi! There is a site called PyTorch forums where users/developers actively participate in solving PyTorch related questions. I suggest you head over there for a much in depth and overall better answer. discuss.pytorch.org $\endgroup$
    – user9947
    Oct 10 '19 at 18:05
  1. BCELoss ( Binary Cross Entropy Loss) is used for binary classifier, which is a neural network that have a binary output, 0 or 1. It is not used for multi-output neural network like your case. For that kind of networks, you can use MSELoss or CrossEntropyLoss as your loss for the network.
  2. For the calculation of BCE, it is shown on pytorch documentation. https://pytorch.org/docs/stable/nn.html#BCELoss For across batch, it either sum the loss or take the mean. you can set that through the reduction="" argument.
  3. As i said, the loss is either summed or taken the mean. Then there will be a single value so pytorch can do backdrop.

Hope I can help you


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.