Linked Questions

14 votes
2 answers

How large should the replay buffer be?

I'm learning DDPG algorithm by following the following link: Open AI Spinning Up document on DDPG, where it is written In order for the algorithm to have stable behavior, the replay buffer should ...
7 votes
2 answers

How can I handle overfitting in reinforcement learning problems?

So this is my current result (loss and score per episode) of my RL model in a simple two players game: I use DQN with CNN as a policy and target networks. I train my model using Adam optimizer and ...
6 votes
3 answers

What are the state-of-the-art approaches for continual learning with neural networks?

There seems to be a lot of literature and research on the problems of stochastic gradient descent and catastrophic forgetting, but I can't find much on solutions to perform continual learning with ...
31 votes
5 answers

Is it possible to train a neural network as new classes are given?

I would like to train a neural network (NN) where the output classes are not (all) defined from the start. More and more classes will be introduced later based on incoming data. This means that, every ...
56 votes
11 answers

What are some well-known problems where neural networks don't do very well?

Background: It's well-known that neural networks offer great performance across a large number of tasks, and this is largely a consequence of their universal approximation capabilities. However, in ...
3 votes
1 answer

Could new training pictures destroy the trained weights of the neural network?

Let's say an image has 28*28 pixels, which leads to 784 input nodes in a feed-forward neural network. If an image can be classified into 1 of 10 numbers (e.g. MNIST), there are 10 output nodes. We ...
6 votes
2 answers

What is the difference between learning without forgetting and transfer learning?

I would like to incrementally train my model with my current dataset and I asked this question on Github, which is what I'm using SSD MobileNet v1. Someone there told me about learning without ...
4 votes
1 answer

Is continuous learning possible with a deep convolutional neural network, without changing its topology?

In general, is continuous learning possible with a deep convolutional neural network, without changing its topology? In my case, I want to use a convolutional neural network as a classifier of ...
17 votes
2 answers

What is the difference between active learning and online learning?

The definitions for these two appear to be very similar, and frankly, I've been only using the term "active learning" the past couple of years. What is the actual difference between the two? ...
1 vote
1 answer

Is there any real-time computer vision system that can learn to detect new objects of new classes?

Suppose you have a ground plane and can use a stereo vision system to detect things that are possibly separate objects. Suppose also your robot or agent can attempt to pick up and move these objects ...
5 votes
1 answer

Why do DQNs tend to forget?

Why do DQNs tend to forget? Is it because when you feed highly correlated samples, your model (function approximation) doesn't give a general solution? For example: I use level 1 experiences, my ...
7 votes
2 answers

Was the corruption of Microsoft's "Tay" chatbot an example of catastrophic forgetting?

Tay was a chatbot, who learned from Twitter users. Microsoft's AI fam from the internet that's got zero chill. The more you talk the smarter Tay gets. — Twitter tagline. Microsoft trained the AI ...