0
$\begingroup$

While reading about Module in PyTorch, I came across a new data type called half datatype.

half() method when calls on a Module casts all floating-point parameters and buffers to half datatype.

It is a 16-bit floating-point number as mentioned here.

It is mentioned in Wikipedia that

It is intended for storage of floating-point values in applications where higher precision is not essential for performing arithmetic computations.

It implies that the precision of parameters (say, weights for a neural network) is not important in certain applications and hence one can use half datatype while implementing a neural network.

Did any research support the statement that precision, that is the range of values it takes, of weights, is unimportant for certain applications?

$\endgroup$
0

2 Answers 2

1
$\begingroup$

Yes, research into ultra-low precision neural network is generally referred to as network quantization. For example, the weights and actications of an artificial neural network can be quantized down to 4-bit, or in extreme cases even 2-bit and 1-bit (binary neural networks).

This is a good introductory article to start with: https://arxiv.org/abs/2106.08295, which goes into detail how network quantization can be done to the maximum extent of preserving the original network accuracy.

The applications of network quantization is immediate. A network with lower precisions would use much less memory and compute power, which is particularly important on edge devices.

However, I do not know whether there are certain ML tasks that are more amenable to quantizatios then others. Generally speaking, we talk about quantization on a network basis. For example, if you quantization ResNet-50, you can generally use this network to run multiple CV tasks like classification, detection, etc.

$\endgroup$
0
$\begingroup$

It’s a tradeoff allowing you to fit a larger model into a fixed RAM budget (ie the size of your GPU). Whether this is a good tradeoff is model- and data-specific, but anecdotally I’ve had good luck with it and usually use half precision to good effect (NLP, mostly).

$\endgroup$
1
  • $\begingroup$ Please make a note of the change in the question. $\endgroup$
    – hanugm
    Jan 17, 2022 at 8:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .