# Dealing with bias in multi-channel auto encoders

## The problem

I have a multi-channel 1D signal I want to auto-encode.
I am unable to resonstruct the input when the number of channels increases.

## Code

I am using a convolutional encoder, and a convolutional decoder:

latent_dim: 512, frames_per_sample: 128

    self._encoder = nn.Sequential(
nn.LeakyReLU(inplace=True),
nn.LeakyReLU(inplace=True),
nn.LeakyReLU(inplace=True),
# nn.Flatten(start_dim=1, end_dim=-1)
nn.Flatten(start_dim=1, end_dim=-1),
nn.Linear(frames_per_sample, self._config.case.latent_dim)
)


and

    start_channels = 256
start_dim = frames_per_sample // (2 ** 4)

start_volume = start_dim * start_channels
self._decoder = nn.Sequential(
nn.Linear(self._config.case.latent_dim, start_volume),
nn.LeakyReLU(inplace=True),
# b, latent
nn.Unflatten(dim=1, unflattened_size=(start_channels, start_dim)),
# b, start_channels, start_dim
nn.Upsample(scale_factor=2, mode='linear', align_corners=False),
# b, start_channels, start_dim*2
# b, 128, start_dim*2
nn.LeakyReLU(inplace=True),
# b, 128, start_dim*2
nn.Upsample(scale_factor=2, mode='linear', align_corners=False),
# b, 128, start_dim*4
# b, 64, start_dim*4
nn.LeakyReLU(inplace=True),
# b,64, start_dim*4
nn.Upsample(scale_factor=2, mode='linear', align_corners=False),
# b, 64, start_dim*8
# b, 32, start_dim*8
nn.LeakyReLU(inplace=True),
# b, 32, start_dim*8
nn.Upsample(scale_factor=2, mode='linear', align_corners=False),
# b, 32, start_dim*16
# b, 16, start_dim*16
nn.LeakyReLU(inplace=True),
# b, 16, start_dim*16
)


I am not putting the entire code/data here because this is a theoretical question, and I don't expect anyone to go and run this.

## Results

The result (orange) has artifacts on the edges, relative to the input data (blue):

This is easy to see on training examples:

Worse - for unseen examples (validation), reconstruction misses on the bias

## Observation

The above only starts to happen when adding more channels, which have different biases.

I am normalizing the entire dataset to sit between -1 and 1, but still each channel has its own typical boundary.

Here is a (nice) result, for a single channel:

## What I think

My guess - Multiple channels force filters to have a single bias, which doesn't fit all of them.
The edges problems are due to bias + zero padding, and the validation data is due to bias that doesn't agree with all channels.

## Questions:

1. Does my analysis make sense?
2. What is a possible way to solve this?

## My thoughts:

1. A distinct bias per channel on at least the last layer. How to do this in Pytorch?
2. Normalizing per-sample (and not per channel) just before passing it to the model, then de normalizing the reconstructed sample.

I don't know how to correctly implement either of those, nor if they make sense, or how to check.

Also posted here, but I think this also belongs on ai.stackexchange