Questions tagged [distillation]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
0answers
20 views

What is the purpose of hard distillation?

In order to get a smaller model, one often uses larger model, that performs reasonably well on the data as a teacher, and uses the information from large model to train the smaller one. There are ...