Comparing a large/general CNN to a smaller more specialized one?

Question

I am still somewhat a novice in the ML world, but I had a strange idea about CNNs and wanted to ask if this would be a valid way to check the robustness of a general CNN that classifies certain images.

Let's say that I make a CNN that takes in many different images of sports players performing a certain action (basketball shot, football kick, freestyle in swimming, flip in gymnastics, etc). Firstly, would it be possible for such a CNN to distinguish between such varied images and classify them accurately? And if so, can it be a good idea to compare this "larger" CNN to multiple "smaller" more specialized ones that take in images from one particular sport?

In other words, I want to know that if I have a "larger" CNN that gives me an output like "football being kicked", is there a way to then double-check that output with a smaller CNN that only focuses on football moves? In essence, could we create a system where once you obtain an output from a general CNN, it automatically classifies the same image through a more specialized CNN, and then if the results are of similar accuracy, you know for sure that CNN works?

Kind of like having a smaller CNN as a "ground-truth" for the bigger one? In my head it kind of goes like this:

large_net_output = 'Football kick identified with 95.56% confidence' 

for sport in large_net:
    if sport == 'football':
        access = small_net_for_football
        return small_net_for_football_output

    elif sport == 'swimming':
        access = small_net_for_swimming
        return small_net_for_swimming_output

    elif sport == 'baseball':
        access = small_net_for_baseball
        return small_net_for_baseball_output

# and so on....
>>> small_net_for_football_output = 'Football kick identified with 97.32% confidence'

robustness_check = large_net_output - small_net_for_football_output
print(robustness_check)

>>> 'Your system is accurate within a good range of 1.76%'
     

I hope this makes sense, and that this question does not cause any of your to cringe. Would appreciate any feedback on this!

Answer 1

After reading your question I can relate it to the Representation Learning papers such as SimCLR and SwAV. These models use a "Big Task agnostic CNN" to obtain smaller representations of the images and then they train another CNN for classification. I suggest you read Big Self-Supervised Models are Strong Semi-Supervised Learners by Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi and Geoffrey Hinton. The code for the following can be found here. But I feel that training such a model would take up a lot of computational resources.