3
$\begingroup$

Transfer learning consists of taking features learned on one problem and leveraging them on a new, similar problem.

In the Transfer Learning, we take layers from a previously trained model and freeze them.

Why is this layer freezing required and what are the effects of layer freezing?

$\endgroup$
1
$\begingroup$

Why is this layer freezing required?

It's not.

What are the effects of layer freezing? The consequences are:

(1) Should be faster to train (the gradient will have far less components)

(2) Should require less data to train on

If you do unfreeze the weights, I'd think your performance would be better because you are adjusting (i.e., fine-tuning) the parameters to your specific problem at hand. I am not sure what the marginal improvements are in practice, as I have not experiemented much with fine-tuning (like are the improvements typically a 0.01% reduction in error rate? Not sure.)

| improve this answer | |
$\endgroup$
0
$\begingroup$

Layer freezing means that the layer weights of the trained model do not change when reused on a subsequent downstream mission, they remain frozen. Basically, when backpropagation is performed during training, these layer weights aren't compromised.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.