Is it possible to exclude specific layers from the optimization?
For example, let's say I have an input layer, 2 hidden layers, and the output layer. I know there is a perfect solution for my problem with this setup and I already know the perfect weights between the first and the second hidden layer.
Can I have the weights between the first and the second hidden layer be fixed during the training phase?
I understand that I could just not update these specific weights after I computed the backpropagation for the entire network. But if I throw away those specific weights, will this affect the optimization of the rest of my weights?