1
$\begingroup$

I am aware of several ways to optimize a convolutional (or any) model after training to make inferencing quicker. I am currently implementing BatchNormalization Folding and removing Dropout layers from the network. I am also aware of post training quantization (specifically 16-bit quantization for use on GPU).

Are there other layer optimization techniques that I can use other than quantization?

My current model uses, Conv2D, Activation(relu), BatchNormalization, Dropout, Dense layers.

Basic mnist Convnet metrics for 10K images, batch size of 1: (All have 98.92% accuracy)

  • Original Network: 49.3s
  • Folded Network: 33.37s
  • Quantized Original: 7.449s
Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

You could look into models pruning. There are several techniques out here, and all of them aim to reduce the amount of parameters of a model without affecting its performance metrics. Of course less parameters means less calculation and therefore faster inference time.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Thank you. I haven't looked into pruning much because it's not as straightforward as the others I've done. I'll look more into it now. $\endgroup$
    – theastronomist
    Commented Feb 12, 2022 at 22:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .