I'm training a convolutional neural network for image classification,and i want to preprocess the images, for example with the CLAHE method. I'm not sure if this preprocessing has to be used on the test and the validation datasets, or just on the training dataset? Because when I try to train with the preprocessed training dataset and evaluate on the original,not preprocessed test set, the network doesn't learn anything, the training accuracy decreases and the test accuracy gets stuck at 49%. But on the other hand, if the test and validation sets are preprocessed as well, everything is fine, accuracies are going up. What am I missing?
I'm wondering if there is a competition (like on Kaggle),where you train your network on preprocessed images, and then your network has to give good performance on a publicly not available, consequently not preprocessed test set. But from my experience described above, it won't give good results on that test set, as opposed to the phenomenon that people win Kaggle competitions with networks trained on preprocessed images. So, what am I missing? Or maybe I just have to optimize the hiperparameters better, to get good results on the not preprocessed test set?