The problem

I have a multi-channel 1D signal I want to auto-encode.
I am unable to resonstruct the input when the number of channels increases.


I am using a convolutional encoder, and a convolutional decoder:

latent_dim: 512, frames_per_sample: 128

    self._encoder = nn.Sequential(
        nn.Conv1d(in_channels=self._n_in_features, out_channels=50, kernel_size=15, stride=1, padding=7),
        nn.Conv1d(in_channels=50, out_channels=50, kernel_size=7, stride=1, padding=3),
        nn.Conv1d(in_channels=50, out_channels=50, kernel_size=3, stride=1, padding=1),
        # nn.Flatten(start_dim=1, end_dim=-1)
        nn.Conv1d(in_channels=50, out_channels=1, kernel_size=1, stride=1, padding=0),
        nn.Flatten(start_dim=1, end_dim=-1),
        nn.Linear(frames_per_sample, self._config.case.latent_dim)


    start_channels = 256
    start_dim = frames_per_sample // (2 ** 4)

    start_volume = start_dim * start_channels
    self._decoder = nn.Sequential(
            nn.Linear(self._config.case.latent_dim, start_volume),
            # b, latent
            nn.Unflatten(dim=1, unflattened_size=(start_channels, start_dim)),
            # b, start_channels, start_dim
            nn.Upsample(scale_factor=2, mode='linear', align_corners=False),
            # b, start_channels, start_dim*2
            nn.Conv1d(in_channels=start_channels, out_channels=128, kernel_size=3, stride=1, padding=1),
            # b, 128, start_dim*2
            # b, 128, start_dim*2
            nn.Upsample(scale_factor=2, mode='linear', align_corners=False),
            # b, 128, start_dim*4
            nn.Conv1d(in_channels=128, out_channels=64, kernel_size=7, stride=1, padding=3),
            # b, 64, start_dim*4
            # b,64, start_dim*4
            nn.Upsample(scale_factor=2, mode='linear', align_corners=False),
            # b, 64, start_dim*8
            nn.Conv1d(in_channels=64, out_channels=32, kernel_size=11, stride=1, padding=5),
            # b, 32, start_dim*8
            # b, 32, start_dim*8
            nn.Upsample(scale_factor=2, mode='linear', align_corners=False),
            # b, 32, start_dim*16
            nn.Conv1d(in_channels=32, out_channels=16, kernel_size=21, stride=1, padding=10),
            # b, 16, start_dim*16
            # b, 16, start_dim*16
            nn.Conv1d(in_channels=16, out_channels=self._n_features, kernel_size=3, stride=1, padding=1),

I am not putting the entire code/data here because this is a theoretical question, and I don't expect anyone to go and run this.


The result (orange) has artifacts on the edges, relative to the input data (blue):

This is easy to see on training examples:

enter image description here

Worse - for unseen examples (validation), reconstruction misses on the bias

enter image description here


The above only starts to happen when adding more channels, which have different biases.

I am normalizing the entire dataset to sit between -1 and 1, but still each channel has its own typical boundary.

Here is a (nice) result, for a single channel:

enter image description here

What I think

My guess - Multiple channels force filters to have a single bias, which doesn't fit all of them.
The edges problems are due to bias + zero padding, and the validation data is due to bias that doesn't agree with all channels.


  1. Does my analysis make sense?
  2. What is a possible way to solve this?

My thoughts:

  1. A distinct bias per channel on at least the last layer. How to do this in Pytorch?
  2. Normalizing per-sample (and not per channel) just before passing it to the model, then de normalizing the reconstructed sample.

I don't know how to correctly implement either of those, nor if they make sense, or how to check.

Also posted here, but I think this also belongs on ai.stackexchange


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.