# Does the reduction of the dimensions over multiple layers allow more details to be stored within the final representation?

From : https://debuggercafe.com/implementing-deep-autoencoder-in-pytorch/ the following autoencoder is defined

class Autoencoder(nn.Module):
def __init__(self):
super(Autoencoder, self).__init__()

# encoder
self.enc1 = nn.Linear(in_features=784, out_features=256)
self.enc2 = nn.Linear(in_features=256, out_features=128)
self.enc3 = nn.Linear(in_features=128, out_features=64)
self.enc4 = nn.Linear(in_features=64, out_features=32)
self.enc5 = nn.Linear(in_features=32, out_features=16)

# decoder
self.dec1 = nn.Linear(in_features=16, out_features=32)
self.dec2 = nn.Linear(in_features=32, out_features=64)
self.dec3 = nn.Linear(in_features=64, out_features=128)
self.dec4 = nn.Linear(in_features=128, out_features=256)
self.dec5 = nn.Linear(in_features=256, out_features=784)

def forward(self, x):
x = F.relu(self.enc1(x))
x = F.relu(self.enc2(x))
x = F.relu(self.enc3(x))
x = F.relu(self.enc4(x))
x = F.relu(self.enc5(x))

x = F.relu(self.dec1(x))
x = F.relu(self.dec2(x))
x = F.relu(self.dec3(x))
x = F.relu(self.dec4(x))
x = F.relu(self.dec5(x))
return x

net = Autoencoder()


From the Autoencoder class, we can see that 784 features are passed through a series of transformations and are converted to 16 features.

The transformations (in_features to out_features) for each layer are:

784 to 256
256 to 128
128 to 64
64 to 32
32 to 16


Why do we perform this sequence of operations? For example, why don't we perform the following sequence of operations instead?

784 to 256
256 to 128


Or maybe

784 to 512
512 to 256
256 to 128


Or maybe just encode in two layers:

784 to 16


Does the reduction of the dimensions over multiple layers (instead of a single layer) allow more details to be stored within the final representation? For example, if we used only the transformation $$784 \rightarrow 16$$, may this cause some detail not to be encoded? If so, why is this the case?