In general, how does one balance the two opposing forces of allowing a layer of a neural network to adapt/learn to its training data, while also forcing the neural network to represent some known structure (e.g., representing a family of functions)? The neural network might find the optimal weights, but those weights might no longer make the layer represent the function I originally intended.
For example, suppose I want to create a convolutional layer in the middle of my neural network that is a low-pass filter. In the context of the entire network, however, the layer might cease to be a low-pass filter at the end of training because the backpropagation algorithm found a better optimum. How do I allow the weights to be as optimal as possible, while still maintaining the low-pass characteristics I originally wanted?
Thanks in advance for any help and suggestions. General tips or pointing to specific literature would be much appreciated.