I’m in this problem and haven’t found a sound solution to it. Been like 20 days now. I have a dataset that looks like this:
X=image Y1= current_zoom (0,25,50,75) Y2= predicted_zoom (0,25,50,75)
y1 will have equal images for all classes. Also, I will know X and y1 when I test the model.
y2 will have variation with it because it has the predicted zoom level.
I tried to train on MTL model, I used two outputs from the model - y1 and y2. Now, y2 overfits (still not sure why, but my best guess is class imbalance). And y1 accuracy is around 0.99 in validation. Now the thing is when I deploy this on production, I’ll always have the current zoom (y1) with the image. So, I wanted to incorporate this to my model. First, I tried with two inputs and one output model, but loss was too much. Here, I concatenated the y1 to the output of last conv layer before it flattens and goes to dense. And second I tried was to concatenate y1 after flattening the output from last conv. Both didn’t work.
Any ideas on how can I with data like this.
Models used: resnet-18, vgg, alexnet
size of data: approx 7000 images in total.