Is size of trained model on disk a good measure of model complexity?

Question

I am writing a research paper on my own custom CNN model for image classification. I am comparing my model architecture with pre-trained architectures, like DenseNet121 and InceptionV3. I want to compare the size of my model with the pre-trained models. I know the following metrics for judging the size of a model:

1. Number of Parameters
2. Number of Layers

However, I was also thinking about comparing the trained model size on disk. As trained DenseNet121 is taking 200MBs and my model is taking less than 20MBs. Therefore, my model is better for embedding in a smartphone application. Is that a good comparison, or am I missing some important point?

Answer 1

The title of your question asks about model complexity. Yet the body of your question talks about this metric as being useful for embedded systems like a smartphone, which have more limited memory.

I think the answer depends on which of those two questions you actually mean:

1. As a measurement of how useful the model may be on a limited system: Yes, this is very useful knowledge! However, I caution you to look at floating point precision of both models, and note that in your comparison if they are different.

2. As a measurement of model complexity: No, this is not a good metric. Smaller disk space does not imply that the model is less complex. For example, one could imagine a KNN taking up enormous disk space, but we would likely not consider that as complex a model as a large (but not as large!) neural network.