Questions tagged [online-learning]
For questions related to online learning algorithms, that is, algorithms that learn while e.g. the associated agent interacts with an environment.
27
questions
0
votes
0
answers
17
views
model versioning and validation of online models
While training an online model, usually we use progressive validation https://maxhalford.github.io/blog/online-learning-evaluation/. My question is what we should do when we detected performance ...
- 101
0
votes
1
answer
46
views
The complexity order of regret (especially in online reinforcement learning)?
In online reinforcement learning theory, how to judge the complexity order of regret, if there are two or more terms in there?
For example, the state space is $X$, the action space is $A$, the episode ...
- 1
0
votes
0
answers
197
views
Has anyone tried decision transformers with PPO?
I am trying to implement a PPO agent to try and solve (or at least get a good solution) for eternity 2 a tile matching game where each tile has 4 colored size you have to minimize the number of ...
0
votes
0
answers
30
views
Is there a way to adapt Particle Swarm Optimization to an incremental/online learning setting?
As stated in the title, is there a way to adapt PSO to an online scenario where new data samples arrive continuously?
In more detail: suppose that I have a classifier with several parameters for which ...
- 1
3
votes
0
answers
54
views
How to fix high variance of the returns on a 2d env?
I'm trying to train an agent on a self-written 2d env, and it just doesn't converge to the solution.
It is basically a 2d game where you have to move a small circle around the screen and try to avoid ...
- 31
0
votes
2
answers
250
views
Are there any resources that introduce the basics of online machine learning?
Are there any resources (either books, articles, or tutorials) that introduce the basics of online machine learning?
For example, this website has nice lecture notes (from lec16) on some of the ...
- 111
1
vote
1
answer
104
views
Do other online/incremental algorithms not suffer from catastrophic forgetting?
All the literature I read seems to indicate catastrophic forgetting affects only neural networks. Do other online/incremental algorithms not suffer from catastrophic forgetting (for example, ...
- 13
2
votes
0
answers
78
views
Why isn't RL considered a continual learning strategy itself?
I have read about methods that apply continual learning strategies to reinforcement learning.
Since reinforcement learning also learns step by step (i.e., task by task, in a sense) during the training ...
- 121
1
vote
0
answers
201
views
UCB-like algorithms: how do you compute the exploration bonus?
My question concerns Stochastic Combinatorial Multiarmed Bandits. More specifically, the algorithm called CombUCB1 presented in this paper. It is a UCB-like algorithm.
Essentially, in each round of ...
- 11
2
votes
1
answer
699
views
How does a neural network that has been trained keep learning while in a real world scenario
Say I trained a Neural Network (not RNN or CNN) to classify a particular data set.
So I train using a specific data set & then I test using another and get an accuracy of 95% which is good enough.
...
- 131
4
votes
1
answer
1k
views
Is continuous learning possible with a deep convolutional neural network, without changing its topology?
In general, is continuous learning possible with a deep convolutional neural network, without changing its topology?
In my case, I want to use a convolutional neural network as a classifier of ...
- 65
16
votes
2
answers
6k
views
What is the difference between active learning and online learning?
The definitions for these two appear to be very similar, and frankly, I've been only using the term "active learning" the past couple of years. What is the actual difference between the two? ...
- 303
1
vote
1
answer
73
views
Is there any real-time computer vision system that can learn to detect new objects of new classes?
Suppose you have a ground plane and can use a stereo vision system to detect things that are possibly separate objects.
Suppose also your robot or agent can attempt to pick up and move these objects ...
- 793
1
vote
1
answer
87
views
Does this $\max$ mean that we need to maximize the regret in this regret formula?
I found that the regret in Online Machine Learning is stated as:
$$\operatorname{Regret}_{T}(h)=\sum_{t=1}^{T} l\left(p_{t}, y_{t}\right)-\sum_{t=1}^{T} l\left(h(x), y_{t}\right),$$
where $p_t$ is the ...
- 199
2
votes
0
answers
26
views
Deriving hyperparameter updates in Online Interactive Collaborative Filtering
I've been going through "Online Interactive Collaborative Filtering Using Multi-Armed Bandit with Dependent Arms" by Wang et al. and am unable to understand how the update equations for the ...
3
votes
1
answer
162
views
How do I keep my system (online) learning if I can get ground truth labels only for examples flagged positive?
I have a binary classifier (think of it as a content moderation system) that is deployed after having being trained via batch learning.
Once deployed, humans review and check for correctness only ...
2
votes
0
answers
41
views
Is there an online RL algorithm that receives as input a camera frame and produces an action as output?
I want to build a reinforcement learning model, which takes a camera picture as input, that learns online (in terms of machine learning). Based on the position of an object on the camera, I want the ...
- 21
8
votes
0
answers
118
views
Normalizing Normal Distributions in Thompson Sampling for online Reinforcement Learning
In my implementation of Thompson Sampling (TS) for online Reinforcement Learning, my distribution for selecting $a$ is $\mathcal{N}(Q(s, a), \frac{1}{C(s,a)+1})$, where $C(s,a)$ is the number of times ...
- 81
2
votes
0
answers
46
views
Best approach for online Machine Translation with few hundred of samples?
I want to implement a model that improves itself with the passage of time.
My main task is to build a machine translator (from English to Urdu).. The problem I am facing is that I have very little ...
- 121
5
votes
3
answers
921
views
What are the state-of-the-art approaches for continual learning with neural networks?
There seems to be a lot of literature and research on the problems of stochastic gradient descent and catastrophic forgetting, but I can't find much on solutions to perform continual learning with ...
- 356
2
votes
1
answer
373
views
How does FastText support online learning?
I'm using FastText pre-trained-embedding for tackling a classification task, but I saw it supports also online training (incremental training) for adding domain-specific corpus.
How does it work?
...
- 65
3
votes
2
answers
237
views
Which online machine learning technique to use for multi-class classification problem with multiple inputs?
I have the following problem. We have $4$ separate discrete inputs, which can take any integer value between $-63$ and $63$. The output is also supposed to be a discrete value between $-63$ and $63$. ...
- 31
1
vote
0
answers
155
views
Online normalization of database for DQN
I have an issue with the normalization of the database (a large time series) for my DQN. I obtained optimal results and saved the NN (5 LSTM layers) weights training on a database normalized as such: ...
- 145
4
votes
1
answer
1k
views
Can a CNN be trained incrementally?
Like our human brain, we can first learn (train) the handwriting 0 and 1. After the traing (and test) accuray is good enough, we only need to study (traing) the hardwriting 2, Instead of cleaning all ...
- 41
4
votes
0
answers
140
views
What are stable ways of doing online machine learning?
I am trying to deploy a machine learning solution online into an application for a client. One thing they requested is that the solution must be able to learn online because the problem may be non-...
- 433
14
votes
1
answer
8k
views
What is the relation between online (or offline) learning and on-policy (or off-policy) algorithms?
In the context of RL, there is the notion of on-policy and off-policy algorithms. I understand the difference between on-policy and off-policy algorithms. Moreover, in RL, there's also the notion of ...
- 39.6k
9
votes
2
answers
1k
views
Are there dynamic neural networks?
Are there neural networks that can decide to add/delete neurons (or change the neuron models/activation functions or change the assigned meaning for neurons), links or even complete layers during ...
- 833