What I want to achieve is this: If my desired outputs are [1, 2, 3, 4] I would rather have my network produce this output:
[0.99, 2.01, 999, 4.01]
than say this:
[0.94, 1.88, 3.12, 4.1]
So I'd rather have a few very accurate outputs and the rest completely off, than have them all be decent but no more than that. My question is, is there a known way to do this? If not, would it make sense to remove the inputs that produce poor outputs, and redo the learning phase?