I am learning about incremental learning and read that rehearsal learning is retraining with old data. In essence, isn't this the exact same thing as batch learning (with stochastic gradient descent)? You train a model by passing in batches of data and redo this with a set number of epochs.

If I'm understanding rehearsal learning correctly, you do the exact same thing but with "new" data. Thus, the only difference is inconsistencies in the epoch number across data batches.


1 Answer 1


In rehearsal, you do not necessarily train with all old training data, but you can just use some of it [1], which you add to your current (or new) training data.

In batch learning, at every epoch, you typically train with all training data, every step with a different batch (or subset) of the training data; so, if you have $N$ training examples and your batch size is $M$, then you will have $\lfloor N/M \rfloor $ gradient descent steps.

There is also pseudo-rehearsal, which is an alternative to rehearsal that is useful when you may not have access to the previously used training data [1] (for example, because it is expensive to store).

  • $\begingroup$ TODO: I really need to explain what "pseudo-rehearsal" is, which I don't do it now. $\endgroup$
    – nbro
    Commented Nov 11, 2020 at 12:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .