What do you call a machine learning system that keeps on learning?

Question

As I understand it from this video lecture, there are three types of deep learning:

• Supervised
• Unsupervised
• Reinforcement

All these can serve to train a neural network either only prior to its deployment or during its operating.

For the latter case, it is referred to as continuous learning here and here and as dynamic learning here and here.

Which term should I use to refer to a machine learning algorithm that keeps on learning (even after deployment)? If it's "continuous", is there an opposing term (such as "static" for "dynamic") for those systems that stop learning before being deployed?

Answer 1

There are several terms or expressions related to such systems, such as online learning, incremental learning, continuous learning, continual learning, and lifelong learning. They are sometimes used interchangeably, but some of them have slightly different meanings. For example, online learning does not need to be incremental, which refers to algorithms that attempt not to forget previously learned information.

The opposite of online is offline. However, the expression batch learning is sometimes used as an antonym for online learning.

Answer 2

In such a thriving field like AI, many terms are not fully established and in some areas it will take more time until the community agrees on specific terms for specific technologies.

Therefore, it might not be possible to give an definitive answer to your question yet. But it is important to note, that the references you gave do not all speak about the same type of learning during operation. And this gives as cues to dig deeper into the topic.

The easiest way to learn during operation is simple reinforcement learning. DQN of Deepmind works exactly that way. It tries out different moves and learns what works by receiving feedback through the reward function. There is not necessarily a separate learning and working phase. The AI just gets better over time. This means, that the NN keeps adapting to the challenge at hand and if that challenge changes, it adopts to the new challenge over time and "forgets" what worked before, if this old strategy is no longer efficient.

Your first reference to Deepmind addresses the aspect of forgetting. When talking about continuous learning, it could also mean, that a system does not forget previous skills but can utilize them later after learning something totally different. This requires more advanced architectures of AI. And this approach doesn't mean, that the NN must necessarily keep learning during operation. It could use this technique during training and stop learning once it is productive.

And there is a third field, recursive networks, which do not pass their information from neuron to neuron in a straight fashion but can contain loops or other types of gates, that control the flow of information. Such architectures are usually more complex than normal NNs, but resemble the working of animal brains a little more. In this field the term dynamic is used frequently.

Therefore, I would say that the term you are looking for is continuous learning - I have also read online learning for this type of NN - when you want to say that the NN keeps learning during production. The term dynamic is more common for recursive or other more complex types of NNs and fits there better in my opinion.

To conclude my answer, I need to jump back to the beginning. Many terms in the field of AI are not finally settled. Therefore my analysis is just a summary of the trends I have seen in different publications and lectures and must not hold true in the years to come. Hope it helps nevertheless.

Answer 3

ML, being a relatively young and fast-developing field, has numerous (near-)synonyms for many concepts.

One paradigm difference is whether a model is learned from a static, pre-defined set of data, or whether it adapts as new data is presented to it over time.

Some of the terms used to describe these two paradigms respectively (with subtle differences in meaning between authors/terms) are:

• Offline / batch / isolated learning
• Online / continual / continuous / incremental / lifelong learning

Further, some familiar branches of ML (like Transfer Learning and Multi-task learning) have a lot of intersection with Continual Learning.

---

Related q's:

• What is the difference between active learning and online learning?

• https://stats.stackexchange.com/questions/897/online-vs-offline-learning