Suppose one trains a CNN to determine if something was either a cat/dog or neither (2 classes), would it be a good idea to assign all cats and dogs to one class and everything else to another? Or would it be better to have a class for cats, a class for dogs, and a class for everything else (3 classes)? My colleague argues for 3 classes because dogs and cats have different features, but I wonder if he's right.
The best approach may be to have a cat, dog, and neither class (3 classes total) and go with a regression approach — specifically, outputting the probabilities of each class for any given input. From there, you can always take the probabilities of each output and derive the probability of a cat and dog class or neither class. Also, make sure you use the right activation on the output layer and cost function so that you can interpret the outputs as probabilities (e.g. softmax activation and cross-entropy loss).
If you want to determine if something is either a
cat/dog or neither
you need 2 classes:
- one for dog or cat, and
- one for anything else.
However, if you assign all cats and dogs to the same class $A$, if an input is classified as $A$, then you won't be able to know whether it is a dog or a cat, you will just know that it is either a dog or a cat.
In case you wanted to distinguish between cats and dogs too (apart from neither of them), then you'll need $3$ classes.
Finally, if you specify only 2 classes:
- dog, and
then your CNN will try to classify any new input as either a dog or a cat, even though it is neither a dog nor a cat (e.g. maybe it is a horse).
As far as generalization error is concerned, you are better off by learning the data distribution of (A and B) classes using unsupervised criterion.
If you capture the underlying factors that explain most of the variations belong to A and B classes, after that, fine-tune it using a supervised criterion. in this way if you used two classes one for (A or B) and the other for neither (A or B), you will not force the model to learn features don't belong to (A or B), because the model just checks if a new data point is probably likely drawn from the data distribution that resembles (A or B).
Side note: you will never have the data necessary to explore the internal structure of the otherwise class (neither A nor B).