It seems to me that in context of object detection True Negative "Detections" amount is an infinite number since counting an absence of detected bounding boxes can be described by an infinite number of not existing boxes of different sizes and locations.
If I am correct, than such metric as Accuracy has no meaning since it includes TN in both - the numerator and the denominator.
Please help me pinpoint my mistake if there is.
All metrics are assessed against test data and/or estimated against a theoretical population. This population is often assumed to be a "target" population, i.e. it contains relative numbers of the values of interest that are likely to occur when a model is used for real.
So, if in real use for a classifier, your positive class detections are a tiny percentage of observations, then yes this may mean accuracy is impractical. However, you should not base this on some theoretical but non-existing population such as calculating the negative class for all possible images. If your model is being used to count parked cars in a car park, then the positive/negative balance in practice doesn't need to take account of what the model would do with random artworks. If the model would not be used overnight, it may not need to be assessed against empty parking lots in the dark.
There is a related issue for using accuracy as a reporting metric, which is a real concern. If a model would perform "well" with zero processing by e.g. always predicting the negative case, then a measure of accuracy is not very useful when reporting results. It's hard to show a useful linear difference between a 99.9% accuracy from a fixed value prediction, and a 99.9% accuracy from a smarter model that actually predicts things from related variables (but that could get a few false positives as well as false negatives). In those cases, you should search for some other metric that helps assess how useful a model would be, perhaps based on some feature that relates to the impact of basing decisions on the prediction.
