# TensorFlow 2.0 - Normalizing input to DNN (on structured data) [closed]

I have a structured dataset of around 100 gigs, and I am using DNN for classification in TF 2.0. Because of this huge dataset, I cannot load entire data in memory for training. So, I'll be reading data in batches to train the model.

Now, the input to the network should be normalized and for that, I need training dataset mean and SD. I have been reading TensorFlow docs to get info on how to normalize features when reading data in batches. But, couldn't find one. though I found this article, it is only for the case where entire data can be loaded in memory.

So, If any of you have worked on creating such a TensorFlow data pipeline for normalizing input features while loading data in batches and training model, It would be helpful.

The central limit theorem tells us that the error in an estimate of the mean or standard deviation of a dataset will decline as $$\frac{1}{\sqrt{n}}$$, where $$n$$ is the number of samples taken at random from the set, and combined together to compute the mean and deviation.
If you select at random, for example, $$10^6$$ examples (probably a few megabytes), then the mean and standard deviation you compute from those examples will be within $$\frac{1}{sqrt(10^6)} = \frac{1}{10^3}$$ of the "true" answer. That's one part in 1,000, which is certainly accurate enough to use for re-scaling the dataset.