Questions tagged [online-learning]
For questions related to online learning algorithms, that is, algorithms that learn while e.g. the associated agent interacts with an environment.
24
questions
0
votes
0
answers
26
views
Is there a way to adapt Particle Swarm Optimization to an incremental/online learning setting?
As stated in the title, is there a way to adapt PSO to an online scenario where new data samples arrive continuously?
In more detail: suppose that I have a classifier with several parameters for which ...
- 1
2
votes
0
answers
44
views
How to fix high variance of the returns on a 2d env?
I'm trying to train an agent on a self-written 2d env, and it just doesn't converge to the solution.
It is basically a 2d game where you have to move a small circle around the screen and try to avoid ...
- 21
1
vote
2
answers
189
views
Are there any resources that introduce the basics of online machine learning?
Are there any resources (either books, articles, or tutorials) that introduce the basics of online machine learning?
For example, this website has nice lecture notes (from lec16) on some of the ...
- 113
1
vote
1
answer
85
views
Do other online/incremental algorithms not suffer from catastrophic forgetting?
All the literature I read seems to indicate catastrophic forgetting affects only neural networks. Do other online/incremental algorithms not suffer from catastrophic forgetting (for example, ...
- 13
2
votes
0
answers
65
views
Why isn't RL considered a continual learning strategy itself?
I have read about methods that apply continual learning strategies to reinforcement learning.
Since reinforcement learning also learns step by step (i.e., task by task, in a sense) during the training ...
- 121
1
vote
0
answers
144
views
UCB-like algorithms: how do you compute the exploration bonus?
My question concerns Stochastic Combinatorial Multiarmed Bandits. More specifically, the algorithm called CombUCB1 presented in this paper. It is a UCB-like algorithm.
Essentially, in each round of ...
- 11
2
votes
1
answer
353
views
How does a neural network that has been trained keep learning while in a real world scenario
Say I trained a Neural Network (not RNN or CNN) to classify a particular data set.
So I train using a specific data set & then I test using another and get an accuracy of 95% which is good enough.
...
- 131
4
votes
1
answer
715
views
Is continuous learning possible with a deep convolutional neural network, without changing its topology?
In general, is continuous learning possible with a deep convolutional neural network, without changing its topology?
In my case, I want to use a convolutional neural network as a classifier of ...
- 65
14
votes
2
answers
5k
views
What is the difference between active learning and online learning?
The definitions for these two appear to be very similar, and frankly, I've been only using the term "active learning" the past couple of years. What is the actual difference between the two? ...
- 283
1
vote
1
answer
72
views
Is there any real-time computer vision system that can learn to detect new objects of new classes?
Suppose you have a ground plane and can use a stereo vision system to detect things that are possibly separate objects.
Suppose also your robot or agent can attempt to pick up and move these objects ...
- 763
1
vote
1
answer
84
views
Does this $\max$ mean that we need to maximize the regret in this regret formula?
I found that the regret in Online Machine Learning is stated as:
$$\operatorname{Regret}_{T}(h)=\sum_{t=1}^{T} l\left(p_{t}, y_{t}\right)-\sum_{t=1}^{T} l\left(h(x), y_{t}\right),$$
where $p_t$ is the ...
- 179
2
votes
0
answers
26
views
Deriving hyperparameter updates in Online Interactive Collaborative Filtering
I've been going through "Online Interactive Collaborative Filtering Using Multi-Armed Bandit with Dependent Arms" by Wang et al. and am unable to understand how the update equations for the ...
3
votes
1
answer
156
views
How do I keep my system (online) learning if I can get ground truth labels only for examples flagged positive?
I have a binary classifier (think of it as a content moderation system) that is deployed after having being trained via batch learning.
Once deployed, humans review and check for correctness only ...
2
votes
0
answers
40
views
Is there an online RL algorithm that receives as input a camera frame and produces an action as output?
I want to build a reinforcement learning model, which takes a camera picture as input, that learns online (in terms of machine learning). Based on the position of an object on the camera, I want the ...
- 21
7
votes
0
answers
99
views
Normalizing Normal Distributions in Thompson Sampling for online Reinforcement Learning
In my implementation of Thompson Sampling (TS) for online Reinforcement Learning, my distribution for selecting $a$ is $\mathcal{N}(Q(s, a), \frac{1}{C(s,a)+1})$, where $C(s,a)$ is the number of times ...
- 71
2
votes
0
answers
40
views
Best approach for online Machine Translation with few hundred of samples?
I want to implement a model that improves itself with the passage of time.
My main task is to build a machine translator (from English to Urdu).. The problem I am facing is that I have very little ...
- 121
4
votes
3
answers
776
views
What are the state-of-the-art approaches for continual learning with neural networks?
There seems to be a lot of literature and research on the problems of stochastic gradient descent and catastrophic forgetting, but I can't find much on solutions to perform continual learning with ...
- 346
2
votes
1
answer
324
views
How does FastText support online learning?
I'm using FastText pre-trained-embedding for tackling a classification task, but I saw it supports also online training (incremental training) for adding domain-specific corpus.
How does it work?
...
- 65
3
votes
2
answers
193
views
Which online machine learning technique to use for multi-class classification problem with multiple inputs?
I have the following problem. We have $4$ separate discrete inputs, which can take any integer value between $-63$ and $63$. The output is also supposed to be a discrete value between $-63$ and $63$. ...
- 31
1
vote
0
answers
147
views
Online normalization of database for DQN
I have an issue with the normalization of the database (a large time series) for my DQN. I obtained optimal results and saved the NN (5 LSTM layers) weights training on a database normalized as such: ...
- 145
4
votes
1
answer
968
views
Can a CNN be trained incrementally?
Like our human brain, we can first learn (train) the handwriting 0 and 1. After the traing (and test) accuray is good enough, we only need to study (traing) the hardwriting 2, Instead of cleaning all ...
- 41
4
votes
0
answers
128
views
What are stable ways of doing online machine learning?
I am trying to deploy a machine learning solution online into an application for a client. One thing they requested is that the solution must be able to learn online because the problem may be non-...
- 423
14
votes
1
answer
6k
views
What is the relation between online (or offline) learning and on-policy (or off-policy) algorithms?
In the context of RL, there is the notion of on-policy and off-policy algorithms. I understand the difference between on-policy and off-policy algorithms. Moreover, in RL, there's also the notion of ...
- 37k
9
votes
2
answers
991
views
Are there dynamic neural networks?
Are there neural networks that can decide to add/delete neurons (or change the neuron models/activation functions or change the assigned meaning for neurons), links or even complete layers during ...
- 803