I was reading about the possibility of using Google's Coral for deep learning-based object detection and image classification. I heard it has a good speed in terms of frames/sec.

I also read that Google's Coral is only compatible with quantized models. What does this mean? How will this affect the performance of object detection or classification in terms of accuracy and speed?

What is the advantage of using Google's Coral over Nvidia's Xavier?


Quantization is a technique used to make deep learning models smaller and faster to run.

Deep learning models are essentially collections of real-valued numbers. Because there are infinitely many real numbers, computers represent them using a format call 'floating point' numbers, which are not completely accurate. For example, a 32-bit floating point number can only represent at most $2^{32}$ distinct values. In contrast, a 64-bit floating point number can represent $2^{64}$ distinct values.

Most CPUs and GPUs cannot operate directly on large floating point numbers. This means that to do something like multiply two floating point numbers, the CPU might have to work on half of each number at a time, and do some tricky work to combine the results. Some CPUs and GPUs can operate directly on large floating point numbers, but only by using more than one core to work on a single number.

To get around this, you might chose to take a model you have that was trained with high-precision floating point weights, and reduce it to lower precision. The weights won't be exactly the same, but they'll be very close. Doing this will make the models run much faster, but you might lose some accuracy.

So the advantage of using a tool that only supports Quantized models is that models will run faster, but might be slightly less accurate.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.