I am self-learning incremental learning about incremental learning and read that rehearsalrehearsal learning is retraining with old data. In essence, isn't this the exact same thing as normal batch/stochastic learning (with stochastic gradient descent)? You train a model by passing in batches of data and redo this with a set number of epochs.
If I'm understanding rehearsal learning correctly, you do the exact same thing but with "new" data. Thus, the only difference is inconsistencies in the epoch number across data batches.