Questions tagged [sample-efficiency]

For questions about the sample efficiency (or inefficiency) of learning algorithms, which is the amount of experience (or data) that the learning algorithm needs in order to reach a certain level of performance. In the case of reinforcement learning, this experience is represented by (transition) tuples of the form $(s, a, r, s')$.

Filter by
Sorted by
Tagged with
1
vote
0answers
31 views

How can I estimate the minimum number of training samples needed to get interesting results with WGAN?

Let's say we have a WGAN where the generator and critic have 8 layers and 5 million parameters each. I know that the greater the number of training samples the better, but is there a way to know the ...
7
votes
1answer
360 views

How to measure sample efficiency of a reinforcement learning algorithm?

I want to know if there is any metric to use for measuring sample-efficiency of a reinforcement learning algorithm? From reading research papers, I see claims that proposed models are more sample ...
2
votes
0answers
36 views

Can you find another reason for sample inefficiency of model-free on-policy Deep Reinforcement Learning?

The following mindmap gives an overview of multiple reasons for sample inefficiency. The list is definitely not complete. Can you see another reason not mentioned so far? Some related links: ...
1
vote
0answers
33 views

Should we start with a small batch-size and increase during training to improve sample efficiency?

Just made an interesting observation playing around with the stable-baseline's implementation of PPO and the BipedalWalker environment from OpenAI's Gym. But I believe this should be a general ...
6
votes
2answers
238 views

Why are reinforcement learning methods sample inefficient?

Reinforcement learning methods are considered to be extremely sample inefficient. For example, in a recent DeepMind paper by Hessel et al., they showed that in order to reach human-level performance ...
3
votes
1answer
520 views

Why are model-based methods more sample efficient than model-free methods?

Why do model-based methods use fewer samples than model-free methods? Here, I'm specifically referring to model-based methods in which we have to learn a policy and model. I can only think of two ...
20
votes
2answers
8k views

What is sample efficiency, and how can importance sampling be used to achieve it?

For instance, the title of this paper reads: "Sample Efficient Actor-Critic with Experience Replay". What is sample efficiency, and how can importance sampling be used to achieve it?