7
$\begingroup$

As I understand it from this video lecture, there are three types of deep learning:

  • Supervised
  • Unsupervised
  • Reinforcement

All these can serve to train a neural network either only prior to its deployment or during its operating.

For the latter case, it is referred to as continuous learning here and here and as dynamic learning here and here.

Which term should I use to refer to a machine learning algorithm that keeps on learning (even after deployment)? If it's "continuous", is there an opposing term (such as "static" for "dynamic") for those systems that stop learning before being deployed?

$\endgroup$
2
$\begingroup$

Reliability of Definitions

The terms Supervised, Unsupervised, and Reinforcement are explained reasonably in the video lecture referenced in the question, however Pascal Vincent from the University of Montreal did not annotate the lecture, show proofs, or demonstrate a comparison using working systems for each term. It is just one faculty member's view. In the search for symmetry and precision in the definitions of those three terms, along with Continuous and Dynamic, ambiguity may be found.

There is no standards body to define terms across the artificial intelligence research and development community. Lack of standardization is a big problem in the software industry in general, but software engineering is still young compared to other engineering realms.

In mechanical engineering, one can design a system and call for six bolt circle of 1/2-20 stainless steel 2" long bolts. Manufacturers can order the bolts and procure the circle with ease because of standards and conventions. Civil engineers can unambiguously tell what is and what is not a buttress in a highway bridge design. In electrical engineering, an instrument amplifier is a class of circuits about which there is little ambiguity.

In software engineering, terms like Component or Framework possess remarkable ambiguity. In contrast, the main features of a RestFUL API have little ambiguity. Similarly, subsets of software engineering, artificial intelligence and machine learning, have their ambiguous terms and their relatively well defined ones.

Learning Follows Deployment

The term in the middle of this question is interesting: "Only prior to its deployment."

There is actually no learning software that learns prior to deployment. The software must be deployed somewhere to learn. If the domain knowledge is in the source code, then the person who wrote the program did the learning, not the software system.

There may be one component that both learns and uses what was learned to perform. Another arrangement involves two components, one that learns but is unable to perform and a separate component that is unable to learn but performs based on what the former component learned. In the integrated case, the one component must be deployed to begin learning. In the two component case, the learning component must be deployed to begin learning. This is true for neural networks and other types of learning systems.

The question would make more sense if it read thus: "Is there [a] term for those systems that stop learning before being used to perform their function?"

There is no single term that is defined exactly like that. One way to describe those systems with no concurrence between learning and performing function using what is learned is Temporally Disjunct, temporally meaning in the time domain, disjunct meaning without intersection. Calling a system a Temporally Disjunct, of course, makes no statement as to whether learning can later be resumed.

Relatively Unambiguous, Common Sense Definitions

Here's how I would define the terms, but this is just my view, based on my research experience and academic background. I unfortunately have less time than university faculty members to provide annotations, proofs, or comparative pseudo-code.

  • Supervised — The learning process is controlled or assisted by an external factor other than merely the signal containing the pattern to be learned. In common practice, the supervision is actually a set of expected values. The expected values are often oriented in the input data alongside the feature values of the training data. (Another way to look at this is that the independent and dependent data are contained in a single point structure.)
  • Unsupervised — Not supervised. Neither expected values nor other data to assist in training is provided beyond the feature data. (This is the case in naive pattern recognition algorithms.)
  • Reinforcement — The provision of a correctness signal indicates to the learning process whether the most recent behavior stemming from previous learning is desirable or undesirable. In some systems, the degree of desirability or undesirability is also provided. The signal may be a vector or matrix in more sophisticated designs. Psychology minded researchers sometimes call the correctness signal a representation of reward or punishment, but such is the anthropomorphism of something more basic than the coercive aspects of parenting, animal training, economic incentivization, or law enforcement.
  • Re-entrant — The learning process is re-entrant, meaning that it can stop, the system can perform a function using what it learned, and learning can resume. Note that the temporal granularity is not implied; no standard defines whether switching between learning and performing based on what was learned occurs every nanosecond or every year.
  • Continuous — Learning does not stop, implying that learning occurs concurrently with the use of what has been learned. (Continuity is actual in the human brain or with massively parallel computing equipment with multiple bus architectures. It is merely simulated in a single CPU system, since even in the most integrated of learning systems, the instructions delegated to acquisition of a pattern must be distinct from the instructions that use what is learned for performing a useful function. In practice, therefore, continuous approaches usually employ re-entrant learning.)

Dynamicity

There is little to add regarding the terms Static and Dynamic beyond the dictionary definition of those terms. Any number of features or organs of learning systems can be either Dynamic or Static. These are just a few example contexts in which dynamicity can be relevant within the domain of machine learning.

  • Values of edges between vertices
  • Numbers of vertices in some place in the structure
  • Overarching network parameters or signals
  • Number of dimensions in signals along edges
  • Ranges of each dimension
  • Number of layers
  • Topology of layers

In Summary

If you are looking for an antonym for Continuous Learning, you may wish to use Discontinuous Learning. As disappointing as this may seem, it is the antonym with less ambiguity. Terms like Re-entrant Training Algorithms may be more technically correct but is somewhat obscure.

Static and Dynamic should always remain antonyms of one another. The many kinds of dynamic features (listed above) that a neural net can have renders the relationship between dynamicity and continuity too convoluted to briefly summarize.

Acknowledgement

Thanks to Douglas Daseeco, our Lab Director, for encouraging comprehensive responses to high level questions (instead of dismissing them as too wide or answering them insufficiently) and for correcting my initial explanation of reinforcement. I had incorrectly entangled re-entrant concepts with the reinforcement and failed to address the existence of a correctness signal.

$\endgroup$
1
$\begingroup$

In such a thriving field like AI, many terms are not fully established and in some areas it will take more time until the community agrees on specific terms for specific technologies.

Therefore, it might not be possible to give an definitive answer to your question yet. But it is important to note, that the references you gave do not all speak about the same type of learning during operation. And this gives as cues to dig deeper into the topic.

The easiest way to learn during operation is simple reinforcement learning. DQN of Deepmind works exactly that way. It tries out different moves and learns what works by receiving feedback through the reward function. There is not necessarily a separate learning and working phase. The AI just gets better over time. This means, that the NN keeps adapting to the challenge at hand and if that challenge changes, it adopts to the new challenge over time and "forgets" what worked before, if this old strategy is no longer efficient.

Your first reference to Deepmind addresses the aspect of forgetting. When talking about continuous learning, it could also mean, that a system does not forget previous skills but can utilize them later after learning something totally different. This requires more advanced architectures of AI. And this approach doesn't mean, that the NN must necessarily keep learning during operation. It could use this technique during training and stop learning once it is productive.

And there is a third field, recursive networks, which do not pass their information from neuron to neuron in a straight fashion but can contain loops or other types of gates, that control the flow of information. Such architectures are usually more complex than normal NNs, but resemble the working of animal brains a little more. In this field the term dynamic is used frequently.

Therefore, I would say that the term you are looking for is continuous learning - I have also read online learning for this type of NN - when you want to say that the NN keeps learning during production. The term dynamic is more common for recursive or other more complex types of NNs and fits there better in my opinion.

To conclude my answer, I need to jump back to the beginning. Many terms in the field of AI are not finally settled. Therefore my analysis is just a summary of the trends I have seen in different publications and lectures and must not hold true in the years to come. Hope it helps nevertheless.

$\endgroup$
  • $\begingroup$ The third question, the term for the converse of continuous learning, could perhaps be added. ... The comment that the term dynamic is used frequently is correct, but the most pertinent point about the term, that it is used in multiple contexts and for a great many software features is not mentioned. It is an important fact that led to the questioner's confusion, so it may be useful to add. ... Although recursion may be used in dynamic features, it is a feature of an algorithm, and is tangential to dynamicity. ... Reinforcement may be easy for a user, but not the AI programmer. $\endgroup$ – FauChristian Aug 30 '17 at 16:31
0
$\begingroup$

There are several terms or expressions related to such systems, such as online learning, incremental learning, continuous learning, continual learning, and lifelong learning. They are sometimes used interchangeably, but some of them have slightly different meanings. For example, online learning does not need to be incremental, which refers to algorithms that attempt not to forget previously learned information.

The opposite of online is offline. However, the expression batch learning is sometimes used as an antonym for online learning.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.