Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
Transfer learning consists of taking features learned on one problem and leveraging them on a new, similar problem.
In the Transfer Learning, we take layers from a previously trained model and freeze them.
Why is this layer freezing required and what are the effects of layer freezing?
Why is this layer freezing required?
It's not.
What are the effects of layer freezing? The consequences are:
(1) Should be faster to train (the gradient will have far less components)
(2) Should require less data to train on
If you do unfreeze the weights, I'd think your performance would be better because you are adjusting (i.e., fine-tuning) the parameters to your specific problem at hand. I am not sure what the marginal improvements are in practice, as I have not experiemented much with fine-tuning (like are the improvements typically a 0.01% reduction in error rate? Not sure.)
Layer freezing means that the layer weights of the trained model do not change when reused on a subsequent downstream mission, they remain frozen. Basically, when backpropagation is performed during training, these layer weights aren't compromised.
