# Which generalization of standard deviation to use for multidimensional input normalization

For machine learning tasks, it's common to normalize input data by subtracting the mean $$\mu$$ and dividing by the standard deviation $$\sigma$$ of the dataset:

$$\hat{x_i} = \frac{x_i - \mu}{\sigma}$$

—where the standard deviation is computed independently for each feature.

Suppose instead we compute another multidimensional generalization of the standard deviation—the matrix square root $$\Sigma$$ of the covariance matrix:

$$\Sigma\Sigma^T = \frac{1}{n}\sum_i^n (x_i - \mu)(x_i - \mu)^T$$

(Since covariance matrix $$\Sigma\Sigma^T$$ is positive semidefinite, it has a unique positive semidefinite square root $$\Sigma$$.)

And then normalize the samples by

$$\hat{x_i} = \Sigma^{-1}(x_i - \mu)$$

In a sense this is a "better" normalization—it will not only map the values to nice ranges, but also decorrelate different features.

However, I have never seen this done. Is there some non-obvious drawback to this approach I'm not seeing, aside from being more computationally intensive? Is there any research on what effect this would have on model training?  