Should a CNN generalize to arbitrary positions in the data?

Question

I have trained a CNN on one dimensional data that is the power spectral density (PSD) of a $N$ different classes of signals ($N=4$). Each of the $N$ signals has a different spectral shape (not shown here). For illustrative purposes, the plots shown below are from the same signal class. The idea is to treat this as an image classification problem.

The model performs exceptionally well when the training data has all the examples centered around the same frequency (i.e., within a few hundred Hertz):

The CNN fails to properly classify new examples that are outside the frequency range of the training data:

Model Details and Assumptions:

The CNN model is implemented in PyTorch using the following layers:

    model = nn.Sequential(
            nn.Conv1d(in_channels=1, out_channels=64, kernel_size=128, stride=1, padding=1),
            nn.ReLU(),
            nn.MaxPool1d(kernel_size=16, stride=2),
            nn.Dropout(p=0.25),
            nn.Flatten(),
            nn.Linear(257664, n_classes)
    )

The optimizer is torch.optim.Adam. Batch size has been varied from 8 to 128, epochs varied from 10 to 50. The input data is normalized to $[0,1]$. Training examples varied from [2000, 8000], where 20% are used for validation and 20% are used for test. I have also tried adding additional convolutional layers, varying the kernal sizes, neurons, layers, etc.

Questions:

1. Shouldn't the CNN model generalize such that new examples that are not within the same frequency range (i.e., centered around the middle) should be identified as the correct class?
2. Are there other steps I need to take, whether in the model or training data?

Answer 1

You would expect the model to extrapolate due to the 1D-Convolutions and the pooling, because they are translation equivariant (with local invariance due to the pooling). Thus, these layers produce encodings that are shifted just like the input timeseries. In your implementation, these operations are currently doing the following: The convolutions + pooling layers have a receptive field of $144$ frequencies

$$ \text{Convolution}: \;\;\text{kernel-size} = 128 ; \; \; \text{stride} = 1\\ \text{Max-Pooling}: \;\;\text{kernel-size} = 16 ; \; \; \text{stride} = 2\\ \Rightarrow \text{Receptive Field} \approx 2 \cdot \frac{128}{2} + 16 = 144 $$

and conceptually, this means that you encode segments with 144 steps into a single step. You then feed the resulting sequence of 4026 steps ($257664 \;/\; 64 = 4026$) to a dense layer. This dense layer is translation sensitive and therefore doesn't naturally generalize to shifted timeseries.

One possible way to approach this is to make the model more translation equivariant. For example by stacking more convolutions/pooling layers to increase the receptive field of your convolutional encoder part. As a side effect, this will probably also benefit training, as the dense layer will get a much smaller input vector.

Another/Additional option is to replace the flatten operator (translation sensitive) with a global pooling operation (translation invariant). But this option will only work if the receptive field of your encoder is wide enough to capture the pattern that identified either class.

EDIT Overall I would look to change the setup of the convolutional encoder: The 1D filters of the convolution have a large overlap (due to the stride of 1 and the kernel size of 128). This results in unnecessarily large feature maps, so you can probably increase the strides a lot (even as large as 32-96 should not be a problem). Also to make the model generalize better to the patterns in your data, I'd try stacking more than one convolution.

I hope this was helpful.