Skip to main content
21 votes

What is the relation between online (or offline) learning and on-policy (or off-policy) algorithms?

The concepts of on-policy vs off-policy and online vs offline are separate, but do interact to make certain combinations more feasible. When looking at this, it is worth also considering the ...
Neil Slater's user avatar
  • 32.7k
15 votes
Accepted

What is the difference between active learning and online learning?

Active learning (AL) is a weakly supervised learning (WSL) technique where you can have both labelled and unlabelled data [1]. The main idea behind AL is that the learner (or learning algorithm) can ...
nbro's user avatar
  • 40.9k
5 votes

What is the difference between active learning and online learning?

As it is referred in the survey paper "Active Learning Literature Survey": The key idea behind active learning is that a machine learning algorithm can achieve greater accuracy with fewer ...
ddaedalus's user avatar
  • 919
4 votes

Can a CNN be trained incrementally?

You are looking for incremental (or online) learning. A CNN can be trained incrementally. For example, in the paper Incremental Learning of Convolutional Neural Networks, the authors propose an ...
nbro's user avatar
  • 40.9k
4 votes
Accepted

What are the state-of-the-art approaches for continual learning with neural networks?

What I understand from your questions is that you are trying to avoid catastrophic forgetting while applying online learning. This problem should be addressed by implementing methods that reduce ...
SandMan's user avatar
  • 253
4 votes

Are there dynamic neural networks?

This article on Dynamically Expandable Neural Networks (DEN) (by Harshvardhan Gupta) is based on this paper Lifelong Learning with Dynamically Expandable Networks (by Jeongtae Lee, Jaehong Yoon, Eunho ...
luvzfootball's user avatar
3 votes
Accepted

Is continuous learning possible with a deep convolutional neural network, without changing its topology?

In general, is continuous learning possible with a deep convolutional neural network, without changing its topology? Your intuition that it is possible to perform incremental (aka continual, ...
nbro's user avatar
  • 40.9k
3 votes

Are there dynamic neural networks?

I mostly studied HMMs and such models are called Infinite HMMs in that specific domain. I believe that what you are looking for is called Infinite Neural Networks. Not having access to scientific ...
Eskapp's user avatar
  • 260
3 votes

What are the state-of-the-art approaches for continual learning with neural networks?

Do you know which are the state-of-the-art approaches on this topic, and could you point me to some literature on them? This answer already mentions some of the approaches. More concretely, currently,...
nbro's user avatar
  • 40.9k
3 votes

How do I keep my system (online) learning if I can get ground truth labels only for examples flagged positive?

A first question that I think is important to consider is: do you expect the data that you're dealing with to be changing over time (i.e. do you expect there to be concept drift)? This could be any ...
Dennis Soemers's user avatar
  • 10.3k
2 votes

What are the state-of-the-art approaches for continual learning with neural networks?

There are lots of different approaches that try to avoid catastrophic forgetting in neural networks. It is impossible to summarize all contributions here. However, in addition to the already mentioned ...
andcos's user avatar
  • 21
2 votes
Accepted

Are there any resources that introduce the basics of online machine learning?

Although you don't seem to want to read papers, you should be able to follow the first pages of the following two papers, if you are familiar with the basics of machine learning (ML). Online learning:...
nbro's user avatar
  • 40.9k
2 votes
Accepted

How does a neural network that has been trained keep learning while in a real world scenario

You are right. If you don't continuously train the neural network after you have deployed it, there is no way it can continuously learn or be updated with more information. You need to program the ...
nbro's user avatar
  • 40.9k
1 vote

The complexity order of regret (especially in online reinforcement learning)?

I assume your algorithm to loop over $K$ policies (or episodes), for $H$ steps, on each state and action pairs (where $X=|\mathcal S|$ and $A=|\mathcal A|$ denote the size of the state and action ...
Luca Anzalone's user avatar
1 vote

Are there any resources that introduce the basics of online machine learning?

Duplicate of answer previously posted on DataScience.SE: On-line learning algorithms trains new data as it arrives. It is often referred to as incremental learning or continuous learning as it trains ...
Archana David's user avatar
1 vote

Is there any real-time computer vision system that can learn to detect new objects of new classes?

You are probably looking for incremental learning (sometimes known as lifelong learning) techniques, i.e. machine learning techniques that attempt to address the catastrophic forgetting effect of ...
nbro's user avatar
  • 40.9k
1 vote
Accepted

Does this $\max$ mean that we need to maximize the regret in this regret formula?

Yes, you're interpreting the $\max$ there wrongly. In your second formula $$ \operatorname{Regret}_{T}(\mathcal{H})=\max _{h^{\star} \in \mathcal{H}} \operatorname{Regret}_{T}\left(h^{\star}\right) \...
nbro's user avatar
  • 40.9k
1 vote

How does FastText support online learning?

The pull request #1327 (https://github.com/facebookresearch/fastText/pull/1327) Allows for: test after each epoch checkpointing training on large data which does not fit into memory (largest I tested ...
Sergei Alonichau's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible