1
$\begingroup$

I'm having a problem understanding how the MSE should be used when working with a multidimensional target, e.g 3 dimensiones. (My outputs are continuois values, not categorical)

Let us say I have a batch size of 2, to make it simple; I pass my input in the network and my y_pred would be a 2x3 tensor. The same happens for y_true, 2x3 itself.

Now the thing I'm not sure of: I take first the difference, diff = y_true - y_pred; this maintains the dimension. Now, for MSE, I square diff, obtaining again a 2x3 tensor, is that right ?

Now the tricky part (for me): I have to Mean. Which one should I consider:

Mean all the (six) values obtaining thus a scalar ? But in this case I do not understand how the backpropagation would make better on specific targets

Mean by rows, i.e, obtaining a 2x1 tensor, so that I have a mean for each example ? Also here I cannot get how the optimization would work

Mean by columns, i.e, obtaining a 1x3 tensors, where I obtain thus an "error" for each target? This seems the more logical to me, but I'm not so sure.

Hope this is clear

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.