I have a base model $M$ trained on a data say type 1 for task $T$. Now, I want to update $M$ by applying transfer learning for it to work on data type 2 for the same task $T$. I am very new to AI/ML field. One common way I found for applying transfer learning is to add a new layer at the end or replace the last layer of the base model with a new layer, and then retrain the model on new data (type 2 here). Depending upon the size of type 2, we may decide whether we retrain the whole model or only the new layer.
However, my question is that how do we decide following:
- What should be the new layer(s)?
- Should the objective function while retraining be the same as the one used for the base model, or it can be different? If different, then any insights on how to figure out a new objective function?
P.S. Data of type 1 and type 2 are of the same category (like both are logs or both are images), however are significantly different.