2
$\begingroup$

I have learned so far how to linear regression with one or multiple features. So far, so good, everything seems to work fine, at least for my first simple examples.

However, I now need to normalise my features for training. I'm doing this by calculating the mean and the standard deviation per feature, and then calculate the normalised feature by subtracting the mean, taking the absolute value, and dividing by the standard deviation. Again, so far, so good, the results of my tensors which I use for training look good.

I understand why I need to normalise input data, and I also understand why one can do it like this (I know that there are other ways as well, e.g. to map values to a 0-1 interval).

Now I was wondering about two things:

  • First, after having trained my network, when I want to make a prediction for a specific input – do I need to normalise this as well, or do I use the un-normalised data? Does it make a difference? My gut feeling says, I should normalise it, as it should make a difference, but I'm not sure. What should I do here, and why?
  • Second, either way, I get a result. Now I was wondering whether I need to denormalise this? I mean, it should make a difference, shouldn't it? If so, how? How do I get from the normalised result value to a denormalised one? Do I just need to reverse the calculation with mean and standard deviation, to get the actual value?

It would be great if someone could shed some light on this.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Okay, I figured it out by myself, simply by trial and error:

When you normalize your training and test data, you also need to normalize the input you want to have a prediction for. You will also need to denormalize the result, to get a reasonable prediction.

Specifically, in my case, this means:

  • First I calculated the mean and the standard deviation, and then subtracted the mean and divided by standard deviation, to normalize my training and test data.
  • So to normalize the input I want to have a prediction for, I also subtract the mean and divide by the standard deviation.
  • Finally, when I get a result, I multiple by the standard deviation, and add the mean.

This way, I get reasonable results 😊

Improve this answer
$\endgroup$
5
  • $\begingroup$ What? How did you assume the multiple non-linearities and even linearities preserve the statistical point estimates i.e mean and std dev of your data? By which I mean the output has a different mean and std dev than input (should be in most cases) $\endgroup$
    – user9947
    Apr 7, 2020 at 2:23
  • $\begingroup$ Maybe it was just accidentally that the results after reverting the transformation made sense? What should I do instead? $\endgroup$
    – Golo Roden
    Apr 7, 2020 at 5:59
  • $\begingroup$ You need to specify the architecture/model you are using. $\endgroup$
    – user9947
    Apr 7, 2020 at 6:05
  • $\begingroup$ Neural network with a single dense layer, with a single node using MSE and SGD. Just a really, really simple example. $\endgroup$
    – Golo Roden
    Apr 7, 2020 at 9:15
  • $\begingroup$ A liitle more detailed maybe? Add it in your question body. I don't think the explanation you have given is correct (I maybe wrong though) $\endgroup$
    – user9947
    Apr 12, 2020 at 3:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .