I'm not sure how to describe this in the most accurate way but I'll give it a shot.
I've developed a Inception-Resnet V2 model for detecting audio signals via spectrogram. It does a pretty good job but is not exactly the way I'd like it to be.
Some details: I use 5 sets of data to evaluate my model during training. They are all similar but slightly different. Once I get to a certain threshold of F1 Scores for each training set I stop training. My overall threshold is pretty hard to get to. Every time training develops a model that produces a "best yet" of one of these data sets I save the model.
What I've noticed is that, during training, some round will produce a high F1 Score for one particular set while the other sets languish as mediocre. Then, several dozen rounds later, another data set will peak while the others are mediocre. Overall the entire model gets better but there are always some models that work better for some data sets.
What I would like to know is, given I might have 5 different models that each work better for a particular subset of data, is there a way that I can combine these models (either as a whole or better yet their particular layers) to produce a single model that works the best for all my data validation subsets?
Thank you. Mecho