1
$\begingroup$

I have been exploring edge computation for AI, and I came across multiple libraries or frameworks, which can help to convert the model into a lite format, which is suitable for edge devices.

  • TensorFlow Lite will help us to convert the TensorFlow model into TensorFlow lite.
  • OpenVino will optimise the model for edge devices.

Questions

  1. If we have a library to optimise the model for edge devices (e.g. TensorFlow Lite), after conversion, could it make the accuracy decrease?

  2. If not, then why do people prefer don't always use e.g. TensorFlow Lite?

$\endgroup$
1
  • $\begingroup$ Note that the comparison of software is off-topic here now. However, I tried to reformulate your post so that it's more acceptable and that the question is more focused on the conceptual issue (although I'm still not fully sure it's on-topic). $\endgroup$ – nbro Feb 17 at 22:42
3
$\begingroup$

This partly answer to question 1. There is no general rule concerning accuracy or size of the model. It depends on the training data and the processed data. The lightest is your model compared to the full accuracy model the less accurate it will be. I would run the lite model on test data and compare to the accuracy of the full model to get an exact measure of the difference.

Tensor flow has different options to save the "lite" model (optimized in size, latency, none and default).

The following mostly answer question 2.

Tensor flow lite is intented to provide the ability to use the model to on line predict only and load the model not to train the model.

On the other hand Tensor flow is used to build (train) the model off line.

If your edge platform support any of the binding language provided for TensorFlow (javascript, java/kotlin, C++, python) you can use Tensorflow for prediction. The accuracy or speed options you might have selected to create the model will not be affected whether you use Tensor Flow or Tensor Flow Lite. Typically Tensor flow lite can be used on mobile devices (Ios, Android). There are other supported target, see this link

$\endgroup$
0
0
$\begingroup$

I have explored the AI for edge devices. My findings for tflite model.

  1. TFLite is just the tool suite to convert TFModel into TFLite.
  2. TFLite optimizes the model for edge or embedded devices using quantization techniques.

Quantization dramatically reduces both the memory requirement and computational cost of using neural networks.

Answer in brief:

  1. When we optimize the model definitely they are faster to take inference but it will impact the accuracy. Quantization leads to bit accuracy loss in smaller networks.
  2. TFLite is increasing the model performance but we have to pay a cost in terms of accuracy that's why we have both TFLite for edge computation and TFModel as well.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.