I'm currently coding a GAN on the dataset MNIST. I'm using the following code to transform my data:
# MNIST Dataset transform = transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))]) # the output of torchvision datasets are PILImage images of range [0, 1] and we # want data that is centered around 0 with a std of 1 (0.1307 and 0.3081 are the estimated values of the MNIST mean & std)
I will have data centered around 0 with a standard deviation of 1 ((0.1307,), (0.3081,) are the estimated mean & standard deviation on the training dataset)). So that means that there will occasionnaly be values above 1 and below -1 in the real data.
Now, my generator ends up with a tanh activation function:
return torch.tanh(self.fc4(x)) # outputs in[-1; 1]
That means there will never be values above 1 and below -1 in the faked data.
Is it possible that the discriminator picks on this phenomenon? This seems to be the case as its loss goes to 0 really quickly. However this also could also be the case that the discriminator is just "too strong" as I've seen numerous times on stackexchange posts. I however never seen nobody talking about the fact that it could pick on the fact that there are outliers in the "real data" and only pixels between -1 and 1 in the "fake data".
EDIT: the entirety of my code can be found here: https://github.com/JQuentinMendoza2008/PyTorch_GAN_for_MNIST_Dataset
Any suggestion is welcomed.