12

Active learning (AL) is a weakly supervised learning (WSL) technique where you can have both labelled and unlabelled data [1]. The main idea behind AL is that the learner (or learning algorithm) can query an "oracle" (e.g. a human) to label some unlabelled instances. AL is similar to semi-supervised learning (SSL), which is also a WSL technique, ...


12

The concepts of on-policy vs off-policy and online vs offline are separate, but do interact to make certain combinations more feasible. When looking at this, it is worth also considering the difference between prediction and control in Reinforcement Learning (RL). Online vs Offline These concepts are not specific to RL, many learning systems can be ...


4

As it is referred in the survey paper "Active Learning Literature Survey": The key idea behind active learning is that a machine learning algorithm can achieve greater accuracy with fewer training labels if it is allowed to choose the data from which it learns. An active learner may pose queries, usually in the form of unlabeled data instances to ...


4

You are looking for incremental (or online) learning. A CNN can be trained incrementally. For example, in the paper Incremental Learning of Convolutional Neural Networks, the authors propose an incremental learning algorithm (inspired by AdaBoost and Learn++, which is another incremental learning algorithm for supervised learning of neural networks) for ...


4

This article on Dynamically Expandable Neural Networks (DEN) (by Harshvardhan Gupta) is based on this paper Lifelong Learning with Dynamically Expandable Networks (by Jeongtae Lee, Jaehong Yoon, Eunho Yang, Sung Ju Hwang) This presents 3 solutions to increase the capacity of the network if needed retaining whatever useful information from the old model and ...


3

A first question that I think is important to consider is: do you expect the data that you're dealing with to be changing over time (i.e. do you expect there to be concept drift)? This could be any kind of change. Simply changes in how frequent certain inputs are, changes in how frequent positives/negatives are, or even changes in relations between inputs ...


3

I mostly studied HMMs and such models are called Infinite HMMs in that specific domain. I believe that what you are looking for is called Infinite Neural Networks. Not having access to scientific publications, I cannot really refer any work here. However, I found this GitHub repository: https://github.com/kutoga/going_deeper that provides some ...


3

Do you know which are the state-of-the-art approaches on this topic, and could you point me to some literature on them? This answer already mentions some of the approaches. More concretely, currently, the most common approaches to continual learning (i.e. learning with progressively more data while attempting to address the catastrophic forgetting problem) ...


2

What I understand from your questions is that you are trying to avoid catastrophic forgetting while applying online learning. This problem should be addressed by implementing methods that reduce catastrophic forgetting for different tasks. At first glance it might seem that they don't apply because it's data that change and not a particular task but ...


2

There are lots of different approaches that try to avoid catastrophic forgetting in neural networks. It is impossible to summarize all contributions here. However, in addition to the already mentioned techniques, there are sparsity approaches that try to disentangle internal representations of the network on different tasks or learning steps. Sparsity ...


2

In general, is continuous learning possible with a deep convolutional neural network, without changing its topology? Your intuition that it is possible to perform incremental (aka continual, continuous or lifelong) learning by changing the NN's topology is correct. However, dynamically adapting the NN's topology is just one approach to continual learning (a ...


1

You are right. If you don't continuously train the neural network after you have deployed it, there is no way it can continuously learn or be updated with more information. You need to program the neural network to learn even after it has been deployed. There is no such thing as a neural network that decides what it does without a human deciding first what ...


1

You are probably looking for incremental learning (sometimes known as lifelong learning) techniques, i.e. machine learning techniques that attempt to address the catastrophic forgetting effect of neural networks when trained incrementally, i.e. as new classes or data are added to the original training data. There are different techniques and some of them ...


1

Yes, you're interpreting the $\max$ there wrongly. In your second formula $$ \operatorname{Regret}_{T}(\mathcal{H})=\max _{h^{\star} \in \mathcal{H}} \operatorname{Regret}_{T}\left(h^{\star}\right) \label{1}\tag{1} $$ The sign $=$ means "is defined as", so maybe the following notation is less confusing $$ \operatorname{Regret}_{T}(\mathcal{H}) \...


Only top voted, non community-wiki answers of a minimum length are eligible