In general, how does one make a neural network learn the training data while also forcing it to represent some known structure (e.g., representing a family of functions)?
The neural network might find the optimal weights, but those weights might no longer make the layer represent the function I originally intended.
For example, suppose I want to create a convolutional layer in the middle of my neural network that is a low-pass filter. In the context of the entire network, however, the layer might cease to be a low-pass filter at the end of training because the backpropagation algorithm found a better optimum.
How do I allow the weights to be as optimal as possible, while still maintaining the low-pass characteristics I originally wanted?
General tips or pointing to specific literature would be much appreciated.