1
$\begingroup$

The problem

I have a multi-channel 1D signal I want to auto-encode.
I am unable to resonstruct the input when the number of channels increases.


Code

I am using a convolutional encoder, and a convolutional decoder:

latent_dim: 512, frames_per_sample: 128

    self._encoder = nn.Sequential(
        nn.Conv1d(in_channels=self._n_in_features, out_channels=50, kernel_size=15, stride=1, padding=7),
        nn.LeakyReLU(inplace=True),
        nn.Conv1d(in_channels=50, out_channels=50, kernel_size=7, stride=1, padding=3),
        nn.LeakyReLU(inplace=True),
        nn.Conv1d(in_channels=50, out_channels=50, kernel_size=3, stride=1, padding=1),
        nn.LeakyReLU(inplace=True),
        # nn.Flatten(start_dim=1, end_dim=-1)
        nn.Conv1d(in_channels=50, out_channels=1, kernel_size=1, stride=1, padding=0),
        nn.Flatten(start_dim=1, end_dim=-1),
        nn.Linear(frames_per_sample, self._config.case.latent_dim)
    )

and

    start_channels = 256
    start_dim = frames_per_sample // (2 ** 4)

    start_volume = start_dim * start_channels
    self._decoder = nn.Sequential(
            nn.Linear(self._config.case.latent_dim, start_volume),
            nn.LeakyReLU(inplace=True),
            # b, latent
            nn.Unflatten(dim=1, unflattened_size=(start_channels, start_dim)),
            # b, start_channels, start_dim
            nn.Upsample(scale_factor=2, mode='linear', align_corners=False),
            # b, start_channels, start_dim*2
            nn.Conv1d(in_channels=start_channels, out_channels=128, kernel_size=3, stride=1, padding=1),
            # b, 128, start_dim*2
            nn.LeakyReLU(inplace=True),
            # b, 128, start_dim*2
            nn.Upsample(scale_factor=2, mode='linear', align_corners=False),
            # b, 128, start_dim*4
            nn.Conv1d(in_channels=128, out_channels=64, kernel_size=7, stride=1, padding=3),
            # b, 64, start_dim*4
            nn.LeakyReLU(inplace=True),
            # b,64, start_dim*4
            nn.Upsample(scale_factor=2, mode='linear', align_corners=False),
            # b, 64, start_dim*8
            nn.Conv1d(in_channels=64, out_channels=32, kernel_size=11, stride=1, padding=5),
            # b, 32, start_dim*8
            nn.LeakyReLU(inplace=True),
            # b, 32, start_dim*8
            nn.Upsample(scale_factor=2, mode='linear', align_corners=False),
            # b, 32, start_dim*16
            nn.Conv1d(in_channels=32, out_channels=16, kernel_size=21, stride=1, padding=10),
            # b, 16, start_dim*16
            nn.LeakyReLU(inplace=True),
            # b, 16, start_dim*16
            nn.Conv1d(in_channels=16, out_channels=self._n_features, kernel_size=3, stride=1, padding=1),
    )

I am not putting the entire code/data here because this is a theoretical question, and I don't expect anyone to go and run this.


Results

The result (orange) has artifacts on the edges, relative to the input data (blue):

This is easy to see on training examples:

enter image description here

Worse - for unseen examples (validation), reconstruction misses on the bias

enter image description here


Observation

The above only starts to happen when adding more channels, which have different biases.

I am normalizing the entire dataset to sit between -1 and 1, but still each channel has its own typical boundary.

Here is a (nice) result, for a single channel:

enter image description here


What I think

My guess - Multiple channels force filters to have a single bias, which doesn't fit all of them.
The edges problems are due to bias + zero padding, and the validation data is due to bias that doesn't agree with all channels.


Questions:

  1. Does my analysis make sense?
  2. What is a possible way to solve this?

My thoughts:

  1. A distinct bias per channel on at least the last layer. How to do this in Pytorch?
  2. Normalizing per-sample (and not per channel) just before passing it to the model, then de normalizing the reconstructed sample.

I don't know how to correctly implement either of those, nor if they make sense, or how to check.

Also posted here, but I think this also belongs on ai.stackexchange

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.