Questions tagged [online-learning]
For questions related to online learning algorithms, that is, algorithms that learn while e.g. the associated agent interacts with an environment.
21
questions
1
vote
1answer
44 views
Do other online/incremental algorithms not suffer from catastrophic forgetting?
All the literature I read seems to indicate catastrophic forgetting affects only neural networks. Do other online/incremental algorithms not suffer from catastrophic forgetting (for example, ...
2
votes
0answers
45 views
Why isn't RL considered a continual learning strategy itself?
I have read about methods that apply continual learning strategies to reinforcement learning.
Since reinforcement learning also learns step by step (i.e., task by task, in a sense) during the training ...
1
vote
0answers
48 views
UCB-like algorithms: how do you compute the exploration bonus?
My question concerns Stochastic Combinatorial Multiarmed Bandits. More specifically, the algorithm called CombUCB1 presented in this paper. It is a UCB-like algorithm.
Essentially, in each round of ...
1
vote
1answer
83 views
How does a neural network that has been trained keep learning while in a real world scenario
Say I trained a Neural Network (not RNN or CNN) to classify a particular data set.
So I train using a specific data set & then I test using another and get an accuracy of 95% which is good enough.
...
3
votes
1answer
258 views
Is continuous learning possible with a deep convolutional neural network, without changing its topology?
In general, is continuous learning possible with a deep convolutional neural network, without changing its topology?
In my case, I want to use a convolutional neural network as a classifier of ...
12
votes
2answers
2k views
What is the difference between active learning and online learning?
The definitions for these two appear to be very similar, and frankly, I've been only using the term "active learning" the past couple of years. What is the actual difference between the two? ...
1
vote
1answer
69 views
Is there any real-time computer vision system that can learn to detect new objects of new classes?
Suppose you have a ground plane and can use a stereo vision system to detect things that are possibly separate objects.
Suppose also your robot or agent can attempt to pick up and move these objects ...
1
vote
1answer
77 views
Does this $\max$ mean that we need to maximize the regret in this regret formula?
I found that the regret in Online Machine Learning is stated as:
$$\operatorname{Regret}_{T}(h)=\sum_{t=1}^{T} l\left(p_{t}, y_{t}\right)-\sum_{t=1}^{T} l\left(h(x), y_{t}\right),$$
where $p_t$ is the ...
2
votes
0answers
25 views
Deriving hyperparameter updates in Online Interactive Collaborative Filtering
I've been going through "Online Interactive Collaborative Filtering Using Multi-Armed Bandit with Dependent Arms" by Wang et al. and am unable to understand how the update equations for the ...
3
votes
1answer
131 views
How do I keep my system (online) learning if I can get ground truth labels only for examples flagged positive?
I have a binary classifier (think of it as a content moderation system) that is deployed after having being trained via batch learning.
Once deployed, humans review and check for correctness only ...
2
votes
0answers
33 views
Is there an online RL algorithm that receives as input a camera frame and produces an action as output?
I want to build a reinforcement learning model, which takes a camera picture as input, that learns online (in terms of machine learning). Based on the position of an object on the camera, I want the ...
6
votes
0answers
60 views
Normalizing Normal Distributions in Thompson Sampling for online Reinforcement Learning
In my implementation of Thompson sampling (TS) for online Reinforcement Learning, my distribution for selecting $a$ is $\mathcal{N}(Q(s, a), \frac{1}{C(s,a)+1})$ where $C(s,a)$ is the number of times $...
2
votes
0answers
34 views
Best approach for online Machine Translation with few hundred of samples?
I want to implement a model that improves itself with the passage of time.
My main task is to build a machine translator (from English to Urdu).. The problem I am facing is that I have very little ...
3
votes
3answers
430 views
What are the state-of-the-art approaches for continual learning with neural networks?
There seems to be a lot of literature and research on the problems of stochastic gradient descent and catastrophic forgetting, but I can't find much on solutions to perform continual learning with ...
2
votes
0answers
235 views
How does FastText support online learning?
I'm using FastText pre-trained-embedding for tackling a classification task, but I saw it supports also online training (incremental training) for adding domain-specific corpus.
How does it work?
...
3
votes
2answers
124 views
Which online machine learning technique to use for multi-class classification problem with multiple inputs?
I have the following problem. We have $4$ separate discrete inputs, which can take any integer value between $-63$ and $63$. The output is also supposed to be a discrete value between $-63$ and $63$. ...
1
vote
0answers
123 views
Online normalization of database for DQN
I have an issue with the normalization of the database (a large time series) for my DQN. I obtained optimal results and saved the NN (5 LSTM layers) weights training on a database normalized as such: ...
4
votes
1answer
560 views
Can a CNN be trained incrementally?
Like our human brain, we can first learn (train) the handwriting 0 and 1. After the traing (and test) accuray is good enough, we only need to study (traing) the hardwriting 2, Instead of cleaning all ...
3
votes
0answers
83 views
What are stable ways of doing online machine learning?
I am trying to deploy a machine learning solution online into an application for a client. One thing they requested is that the solution must be able to learn online because the problem may be non-...
8
votes
1answer
3k views
What is the relation between online (or offline) learning and on-policy (or off-policy) algorithms?
In the context of RL, there is the notion of on-policy and off-policy algorithms. I understand the difference between on-policy and off-policy algorithms. Moreover, in RL, there's also the notion of ...
9
votes
2answers
802 views
Are there dynamic neural networks?
Are there neural networks that can decide to add/delete neurons (or change the neuron models/activation functions or change the assigned meaning for neurons), links or even complete layers during ...
