While reading about Module in PyTorch, I came across a new data type called
half() method when calls on a Module casts all floating point parameters and buffers to half datatype.
Is it a 16-bit floating point number as mentioned here?
It is mentioned in Wikipedia that
It is intended for storage of floating-point values in applications where higher precision is not essential for performing arithmetic computations.
It implies that the precision of parameters (say, weights for a neural network) is not important and hence one can use half datatype while implementing neural network.
Did any research support the statement that precision, that is range of values it takes, of weights is unimportant?