We have an image classifier that was built using CNN with faster R-CNN and Yolov5.
It is designated to run on 3D objects. All of those objects have similar "features" structure, but the actual features of each object class are somewhat different one from another. Therefore, we strive to detect the classes based on those differences in features.
In theory there are thousands of different classes, but for now we have trained the model to detect 4 types of classes, by training it on data sets that includes many images from different angles for each of those 4 classes (1,000 images each).
The main problem we face is that whenever the model runs on an "unknown" object, it may still classify it as one of our 4 classes, and sometimes it will do it with a high probability score (0.95), which hinders the whole credibility of our model results.
We think it might be since we are using SoftMax, which seems to force the model to assign an unknown object to one of the 4 classes.
We want to know what will the best way to overcome this issue.
We tried adding a new, fifth "trash" class, with 1,000 images of "other" objects that do not belong to our four classes, but it significantly reduced the confidence level for our test images, so we are not sure if this is at all a progress.
