What is early stopping in machine learning and, in general, artificial intelligence? What are the advantages of using this method? How does it help exactly?
I'd be interested in perspectives and links to recent research.
$\begingroup$
$\endgroup$
0
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
In some iterative learning methods the more iterations you apply the more specific your model becomes about the training set. If there are too many iterations, your model will become too specifically trained for the training samples and will score less on other samples that are not seen during the training phase. This is called over-fitting, though over-fitting is not specific to iterative learning methods.
One solution to prevent over-fitting in these iterative learning algorithms is early stopping. Normally a control group of samples called validation samples (validation set) are used to validate the model and notify when it starts to over-fit. The validation set is not used by the training algorithm, however, its corresponding outputs are known and after each iteration, its samples are employed to measure how well the model currently works. As soon as the performance on the validation set stops growing and starts to drop we stop iterating the training algorithm. This is called early stopping which can help to maximize the generalization power of our learned model.
Note that if we use the training set itself for validation the performance will always increase because that is what the learning algorithm is designed to do. However, the learning algorithm does not know how specifically it should learn the training set and that is why we need methods like early stopping.
