0
$\begingroup$

I notice that I create a model using tensorflow.keras.Sequential(), save it and the file size is around 5 MiB, but after I call model.fit(..), the file size is increased to 17 MiB. I copied the model to reduce the filesize and see that the accuracy is the same.

My question is, what exactly is the content of extra 12 MiB that fit() produces? How can I access such content? If I remove those extra 12 MiB, could it affect prediction accuracy or any weird side-effect?

See my experiment code here: https://nbviewer.jupyter.org/github/off99555/TensorFlowExperiments/blob/master/test-save-keras-model.ipynb

$\endgroup$
1
$\begingroup$

The answer is that it's the size of the Adam optimizer state. When I change the optimizer to SGD (the vanilla optimizer), the size is not big anymore. As far as I know, the Adam optimizer maintains gradients information of previous training iterations. And the gradient size can be as big as the model size. That's why it causes the file size to be so big.

With this in mind, when you save your model please make sure to set include_optimizer=False if you seem to use an optimizer that maintains big state similar to Adam.

Beware though, it means that you cannot load the model and continue training it again, it should only be used for inference.

|improve this answer|||||
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.