# How does one make a neural network learn the training data while also forcing it to represent some known structure?

In general, how does one make a neural network learn the training data while also forcing it to represent some known structure (e.g., representing a family of functions)?

The neural network might find the optimal weights, but those weights might no longer make the layer represent the function I originally intended.

For example, suppose I want to create a convolutional layer in the middle of my neural network that is a low-pass filter. In the context of the entire network, however, the layer might cease to be a low-pass filter at the end of training because the backpropagation algorithm found a better optimum.

How do I allow the weights to be as optimal as possible, while still maintaining the low-pass characteristics I originally wanted?

General tips or pointing to specific literature would be much appreciated.

• Two ways to do it - freeze above mentioned layer, don't update it. Another - add regularizer to loss wich force that layer to retain it's properties. Latter could be very non-trivial Jun 11, 2019 at 5:34