I'm having a problem understanding how the MSE should be used when working with a multidimensional target, e.g 3 dimensiones. (My outputs are continuois values, not categorical)

Let us say I have a batch size of 2, to make it simple; I pass my input in the network and my y_pred would be a 2x3 tensor. The same happens for y_true, 2x3 itself.

Now the thing I'm not sure of: I take first the difference, diff = y_true - y_pred; this maintains the dimension. Now, for MSE, I square diff, obtaining again a 2x3 tensor, is that right ?

Now the tricky part (for me): I have to Mean. Which one should I consider:

Mean all the (six) values obtaining thus a scalar ? But in this case I do not understand how the backpropagation would make better on specific targets

Mean by rows, i.e, obtaining a 2x1 tensor, so that I have a mean for each example ? Also here I cannot get how the optimization would work

Mean by columns, i.e, obtaining a 1x3 tensors, where I obtain thus an "error" for each target? This seems the more logical to me, but I'm not so sure.

Hope this is clear


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.