Questions tagged [online-learning]

For questions related to online learning algorithms, that is, algorithms that learn while e.g. the associated agent interacts with an environment.

Filter by
Sorted by
Tagged with
0 votes
0 answers

model versioning and validation of online models

While training an online model, usually we use progressive validation My question is what we should do when we detected performance ...
koch's user avatar
  • 101
0 votes
1 answer

The complexity order of regret (especially in online reinforcement learning)?

In online reinforcement learning theory, how to judge the complexity order of regret, if there are two or more terms in there? For example, the state space is $X$, the action space is $A$, the episode ...
white's user avatar
  • 1
0 votes
0 answers

Has anyone tried decision transformers with PPO?

I am trying to implement a PPO agent to try and solve (or at least get a good solution) for eternity 2 a tile matching game where each tile has 4 colored size you have to minimize the number of ...
mt-clemente's user avatar
0 votes
0 answers

Is there a way to adapt Particle Swarm Optimization to an incremental/online learning setting?

As stated in the title, is there a way to adapt PSO to an online scenario where new data samples arrive continuously? In more detail: suppose that I have a classifier with several parameters for which ...
Elise Le's user avatar
3 votes
0 answers

How to fix high variance of the returns on a 2d env?

I'm trying to train an agent on a self-written 2d env, and it just doesn't converge to the solution. It is basically a 2d game where you have to move a small circle around the screen and try to avoid ...
debrises's user avatar
0 votes
2 answers

Are there any resources that introduce the basics of online machine learning?

Are there any resources (either books, articles, or tutorials) that introduce the basics of online machine learning? For example, this website has nice lecture notes (from lec16) on some of the ...
Slim Shady's user avatar
1 vote
1 answer

Do other online/incremental algorithms not suffer from catastrophic forgetting?

All the literature I read seems to indicate catastrophic forgetting affects only neural networks. Do other online/incremental algorithms not suffer from catastrophic forgetting (for example, ...
user3631804's user avatar
2 votes
0 answers

Why isn't RL considered a continual learning strategy itself?

I have read about methods that apply continual learning strategies to reinforcement learning. Since reinforcement learning also learns step by step (i.e., task by task, in a sense) during the training ...
convaldo's user avatar
  • 121
1 vote
0 answers

UCB-like algorithms: how do you compute the exploration bonus?

My question concerns Stochastic Combinatorial Multiarmed Bandits. More specifically, the algorithm called CombUCB1 presented in this paper. It is a UCB-like algorithm. Essentially, in each round of ...
Adam's user avatar
  • 11
2 votes
1 answer

How does a neural network that has been trained keep learning while in a real world scenario

Say I trained a Neural Network (not RNN or CNN) to classify a particular data set. So I train using a specific data set & then I test using another and get an accuracy of 95% which is good enough. ...
user3536523's user avatar
4 votes
1 answer

Is continuous learning possible with a deep convolutional neural network, without changing its topology?

In general, is continuous learning possible with a deep convolutional neural network, without changing its topology? In my case, I want to use a convolutional neural network as a classifier of ...
Dominiksr's user avatar
16 votes
2 answers

What is the difference between active learning and online learning?

The definitions for these two appear to be very similar, and frankly, I've been only using the term "active learning" the past couple of years. What is the actual difference between the two? ...
David's user avatar
  • 303
1 vote
1 answer

Is there any real-time computer vision system that can learn to detect new objects of new classes?

Suppose you have a ground plane and can use a stereo vision system to detect things that are possibly separate objects. Suppose also your robot or agent can attempt to pick up and move these objects ...
FourierFlux's user avatar
1 vote
1 answer

Does this $\max$ mean that we need to maximize the regret in this regret formula?

I found that the regret in Online Machine Learning is stated as: $$\operatorname{Regret}_{T}(h)=\sum_{t=1}^{T} l\left(p_{t}, y_{t}\right)-\sum_{t=1}^{T} l\left(h(x), y_{t}\right),$$ where $p_t$ is the ...
FraMan's user avatar
  • 199
2 votes
0 answers

Deriving hyperparameter updates in Online Interactive Collaborative Filtering

I've been going through "Online Interactive Collaborative Filtering Using Multi-Armed Bandit with Dependent Arms" by Wang et al. and am unable to understand how the update equations for the ...
Shashank Gupta's user avatar
3 votes
1 answer

How do I keep my system (online) learning if I can get ground truth labels only for examples flagged positive?

I have a binary classifier (think of it as a content moderation system) that is deployed after having being trained via batch learning. Once deployed, humans review and check for correctness only ...
Davide Fiocco's user avatar
2 votes
0 answers

Is there an online RL algorithm that receives as input a camera frame and produces an action as output?

I want to build a reinforcement learning model, which takes a camera picture as input, that learns online (in terms of machine learning). Based on the position of an object on the camera, I want the ...
Wuuhi's user avatar
  • 21
8 votes
0 answers

Normalizing Normal Distributions in Thompson Sampling for online Reinforcement Learning

In my implementation of Thompson Sampling (TS) for online Reinforcement Learning, my distribution for selecting $a$ is $\mathcal{N}(Q(s, a), \frac{1}{C(s,a)+1})$, where $C(s,a)$ is the number of times ...
Kevin's user avatar
  • 81
2 votes
0 answers

Best approach for online Machine Translation with few hundred of samples?

I want to implement a model that improves itself with the passage of time. My main task is to build a machine translator (from English to Urdu).. The problem I am facing is that I have very little ...
Abdul Rehman's user avatar
5 votes
3 answers

What are the state-of-the-art approaches for continual learning with neural networks?

There seems to be a lot of literature and research on the problems of stochastic gradient descent and catastrophic forgetting, but I can't find much on solutions to perform continual learning with ...
gcorso's user avatar
  • 356
2 votes
1 answer

How does FastText support online learning?

I'm using FastText pre-trained-embedding for tackling a classification task, but I saw it supports also online training (incremental training) for adding domain-specific corpus. How does it work? ...
Alfonso's user avatar
  • 65
3 votes
2 answers

Which online machine learning technique to use for multi-class classification problem with multiple inputs?

I have the following problem. We have $4$ separate discrete inputs, which can take any integer value between $-63$ and $63$. The output is also supposed to be a discrete value between $-63$ and $63$. ...
John Mayer's user avatar
1 vote
0 answers

Online normalization of database for DQN

I have an issue with the normalization of the database (a large time series) for my DQN. I obtained optimal results and saved the NN (5 LSTM layers) weights training on a database normalized as such: ...
FS93's user avatar
  • 145
4 votes
1 answer

Can a CNN be trained incrementally?

Like our human brain, we can first learn (train) the handwriting 0 and 1. After the traing (and test) accuray is good enough, we only need to study (traing) the hardwriting 2, Instead of cleaning all ...
Sunson29's user avatar
4 votes
0 answers

What are stable ways of doing online machine learning?

I am trying to deploy a machine learning solution online into an application for a client. One thing they requested is that the solution must be able to learn online because the problem may be non-...
Rui Nian's user avatar
  • 433
14 votes
1 answer

What is the relation between online (or offline) learning and on-policy (or off-policy) algorithms?

In the context of RL, there is the notion of on-policy and off-policy algorithms. I understand the difference between on-policy and off-policy algorithms. Moreover, in RL, there's also the notion of ...
nbro's user avatar
  • 39.6k
9 votes
2 answers

Are there dynamic neural networks?

Are there neural networks that can decide to add/delete neurons (or change the neuron models/activation functions or change the assigned meaning for neurons), links or even complete layers during ...
TomR's user avatar
  • 833