0
$\begingroup$

I have explored edge computation for AI and I came across multiple library or framework which can help to convert model into lite format which is suitable for edge devices.

Tensorflow Lite will help us to convert Tensorflow Model into Tensorflow lite. Or Intel OpenVino will optimise the model for edge devices.

Questions

  1. If we have library to optimise the model for edge devices, after conversion will it cause any accuracy issue. performance will get increase but accuracy of that model will get decrease.
  2. If we have such technology to optimise model performance and wont get accuracy decrease then why people wont prefer tensorflow Lite model over tensorflow model.
|improve this question
$\endgroup$
2
$\begingroup$

This partly answer to question 1. There is no general rule concerning accuracy or size of the model. It depends on the training data and the processed data. The lightest is your model compared to the full accuracy model the less accurate it will be. I would run the lite model on test data and compare to the accuracy of the full model to get an exact measure of the difference.

Tensor flow has different options to save the "lite" model (optimized in size, latency, none and default).

The following mostly answer question 2.

Tensor flow lite is intented to provide the ability to use the model to on line predict only and load the model not to train the model.

On the other hand Tensor flow is used to build (train) the model off line.

If your edge platform support any of the binding language provided for TensorFlow (javascript, java/kotlin, C++, python) you can use Tensorflow for prediction. The accuracy or speed options you might have selected to create the model will not be affected whether you use Tensor Flow or Tensor Flow Lite. Typically Tensor flow lite can be used on mobile devices (Ios, Android). There are other supported target, see this link

|improve this answer
$\endgroup$
  • $\begingroup$ Please read my question carefully. $\endgroup$ – Maheshwar Ligade Dec 19 '19 at 5:27
  • $\begingroup$ can you be more specific about your comment or rephase your question please I might be able to reformulate my answer $\endgroup$ – pascal sautot Dec 19 '19 at 9:26
  • $\begingroup$ I have two questions which are interrelated point 1 and point 2. $\endgroup$ – Maheshwar Ligade Dec 19 '19 at 10:47
  • $\begingroup$ By any chance could you check if that answered your question. $\endgroup$ – pascal sautot Jan 31 at 14:05
  • $\begingroup$ I upvoted for your answer. Thank you for giving me a brief overview. $\endgroup$ – Maheshwar Ligade Feb 3 at 5:21
0
$\begingroup$

I have explored the AI for edge devices. My findings for tflite model.

  1. TFLite is just the tool suite to convert TFModel into TFLite.
  2. TFLite optimizes the model for edge or embedded devices using quantization techniques.

Quantization dramatically reduces both the memory requirement and computational cost of using neural networks.

Answer in brief:

  1. When we optimize the model definitely they are faster to take inference but it will impact the accuracy. Quantization leads to bit accuracy loss in smaller networks.
  2. TFLite is increasing the model performance but we have to pay a cost in terms of accuracy that's why we have both TFLite for edge computation and TFModel as well.
|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.