I have been searching this but did not find the answer, so sorry if this is a duplicated question.
I was working with cross-validation, where some doubts came to my mind, and I am not sure about which is the correct answer.
Lets say I have a mixed dataset, with numerical and categorical features. I want to perform a K-Fold Cross-Validation with it, with a K=10. Some of this numerical features are missing, so I decided that I will replace those NaNs with the average of that feature.
My steps are the following ones:
- Read entire dataset
- Perform One Hot Encoding to categorical features.
- Divide my data into different folds. Lets say that I will use 90% for training, 10% for validating.
- For every different combination of folds, I replace the missing values from the traininig and validating sets separately. This means, in one hand I get the average of the missing values of the training part, and on the other hand the average of the missing values of the validating part.
- Normalize the data of the training and validating sets between [0, 1] separately, like I did before.
- Train the correspondant model.
So lets put a simple example of a dataset of 20 rows with N columns. Once I do steps 1 and 2, in the first iteration I will select the 18 first rows as a training set, and the last two rows as validating set. I fill the missing values of the 9 first rows with the average of those 18 rows. Then the same for the 2 last rows. Then, again, normalize in the same way, separately. And do this for every combination of folds.
I am doing it like this, because otherwise, from my understanding, is that you are training your model with a biased data. You should not have access to the validation data, thus you should not be able to do the average with those numbers. Hence I am using only the numbers of the training part. If I do the average with the entire dataset, this will make my model to overfitting.
I am not so sure about the normalization step, as I do not really think this will have the same impact. But here I do not really know...
Is this aproach correct? Or should I do the average and normalization with the entire dataset? Why?