I'm building an object detection model with convolutional neural networks (CNN) and I started to wonder when should one use either multi-class CNN or a single-class CNN. That is, if I'm making e.g. a human detector from image data and a cat detector also from image data, then when should I have a specific model for each task, and when should I just combine all the data into one and use just one general multi-class CNN?
I've understood from the No-Free-Lunch-Theorem and generally from estimation theory, that there there does not, in theory, exist a model which is simultaneously optimal for every problem. In other words, case specific models should, in general, beat the "all-purpose"-models in the same task.
I have a difference in opinion with a colleague of mine whether to use one-class of a multi-class CNN and I would like to hear the communities opinion on this.