I'm having a problem understanding how the MSE should be used when working with a multidimensional target, e.g 3 dimensiones. (My outputs are continuois values, not categorical)
Let us say I have a batch size of 2, to make it simple; I pass my input in the network and my y_pred would be a 2x3 tensor. The same happens for y_true, 2x3 itself.
Now the thing I'm not sure of: I take first the difference, diff = y_true - y_pred; this maintains the dimension.
Now, for MSE, I square diff, obtaining again a 2x3 tensor, is that right ?
Now the tricky part (for me): I have to Mean. Which one should I consider:
Mean all the (six) values obtaining thus a scalar ? But in this case I do not understand how the backpropagation would make better on specific targets
Mean by rows, i.e, obtaining a 2x1 tensor, so that I have a mean for each example ? Also here I cannot get how the optimization would work
Mean by columns, i.e, obtaining a 1x3 tensors, where I obtain thus an "error" for each target? This seems the more logical to me, but I'm not so sure.
Hope this is clear
