I am aware of several ways to optimize a convolutional (or any) model after training to make inferencing quicker. I am currently implementing BatchNormalization Folding and removing Dropout layers from the network. I am also aware of post training quantization (specifically 16-bit quantization for use on GPU).
Are there other layer optimization techniques that I can use other than quantization?
My current model uses, Conv2D, Activation(relu), BatchNormalization, Dropout, Dense layers.
Basic mnist Convnet metrics for 10K images, batch size of 1: (All have 98.92% accuracy)
- Original Network: 49.3s
- Folded Network: 33.37s
- Quantized Original: 7.449s