# How to choose the new layer and objective function for transfer learning on a neural network?

I have a base model $$M$$ trained on a data say type 1 for task $$T$$. Now, I want to update $$M$$ by applying transfer learning for it to work on data type 2 for the same task $$T$$. I am very new to AI/ML field. One common way I found for applying transfer learning is to add a new layer at the end or replace the last layer of the base model with a new layer, and then retrain the model on new data (type 2 here). Depending upon the size of type 2, we may decide whether we retrain the whole model or only the new layer.

However, my question is that how do we decide following:

1. What should be the new layer(s)?
2. Should the objective function while retraining be the same as the one used for the base model, or it can be different? If different, then any insights on how to figure out a new objective function?

P.S. Data of type 1 and type 2 are of the same category (like both are logs or both are images), however are significantly different.

• What do you mean by "same task $T$"? Do you mean one of the two general tasks of classifying or regressing? If you change the dataset, then you may also be changing the task, that's why I am asking this question.
– nbro
Nov 3 at 13:00
• By same task I mean that they are doing the exact same thing. However the data is changing considerably in my case. Hence, I am trying to fine tune the model for the new data (very small size available) using the transfer learning techniques. Hope that answers yours question @nbro Nov 5 at 17:21
• Can you be more concrete? For example, the task is classifying dogs vs cats, but the images are different (e.g. they have higher resolution in the second dataset?)
– nbro
Nov 5 at 21:33
• Lets say the task is about compressing some data using autoencoders. The AE is trained on one type of log data now a variation of data has come up and need to update the AE accordingly. @nbro Nov 8 at 18:55