I have a dataset with the following info
Image1 x1 x2 x3 y
Image2 x1 x2 x3 y ...
Where x1, x2 & x3 are categorical features. My goal is to extract features from the images and use those features combined with x1 x2 and x3 as an input to another model to classify y.
My question is the following. Assuming I use a pre-trained resnet model, should I retrain the last layer using my images and y's to fit the resnet better to my problem or should I directly use the features extracted from the resnet as an input to my other model without retraining resnet?