How can I train a neural network for another input set, without losing the learning of the previous input set?

Question

I read this tutorial about backpropagation.

So using this backpropagation we are training the neural network repeatedly for one input set, say [2,4], until we reach 100% accuracy of getting 1 as output. And the neural network is adjusting its weight values accordingly. So once after the neural network is trained this way, suppose we are giving another input set, say [6,8], also then will the neural network update its weight values (overwriting previous values), right? This will result in losing the previous learning, right?

Answer 1

Yes, this is actually a limitation known as catastrophic forgetting. A proposed way to deal with this is elastic weight consolidation that "remembers old tasks by selectively slowing down learning on the weights important for those tasks". See Overcoming catastrophic forgetting in neural networks for details. Another approach is Learning without forgetting.

If the tasks are different, the approach you are talking about is called transfer learning. You might want to have a look at multi-task learning as well

If the tasks are the same, you could try creating a join of both datasets and training on that.