I have a combined network consisting of two parts: one is for images and the other is for numerical data. Each sample is matched with a numerical case by an ID. For this combined network, a
lr of 0.01 was found to be best working via hyperparameter tunings
However, when I trained them as a separate task (binary classification), a
lr of 0.001 for images and 0.01 for numerical data were best.
As for an AUC metric, the combined network (0.818) is performing on average of image and numerical networks (0.799 and 0.821, respectively).
Here I thought maybe the combined network's
lr is too high for image part and should apply lower
lr for that part. But, I don't know if it is possible.
If anyone has any idea of what is what, let me know