Questions tagged [sample-efficiency]

For questions about the sample efficiency (or inefficiency) of learning algorithms, which is the amount of experience (or data) that the learning algorithm needs in order to reach a certain level of performance. In the case of reinforcement learning, this experience is represented by (transition) tuples of the form $(s, a, r, s')$.

Filter by
Sorted by
Tagged with
1
vote
1answer
44 views

How to use a heuristic policy to increase sample efficiency of a deep reinforcement learning agent?

I have a heuristic solution to a problem which works quite well when certain environmental parameters are known and unchanging. However, in a real world setting these parameters will not be known and ...
0
votes
0answers
45 views

Does randomly adding hand-engineered features increase the CNN's sample efficiency/performance?

It is a known fact that preprocessing images using CV techniques will improve CNN performance (see this answer). But what happens when you feed in the entire image and the filtered image randomly to ...
2
votes
1answer
39 views

How do I represent sample efficiency of RL rewards in mathematical notation?

I define sample efficiency as the area under the curve/graph, where $x$-axis is the number of episodes while y-axis is the cumulative reward for that episode. I would like to formally define it with a ...
1
vote
0answers
33 views

How can I estimate the minimum number of training samples needed to get interesting results with WGAN?

Let's say we have a WGAN where the generator and critic have 8 layers and 5 million parameters each. I know that the greater the number of training samples the better, but is there a way to know the ...
7
votes
1answer
485 views

How to measure sample efficiency of a reinforcement learning algorithm?

I want to know if there is any metric to use for measuring sample-efficiency of a reinforcement learning algorithm? From reading research papers, I see claims that proposed models are more sample ...
2
votes
0answers
42 views

Can you find another reason for sample inefficiency of model-free on-policy Deep Reinforcement Learning?

The following mindmap gives an overview of multiple reasons for sample inefficiency. The list is definitely not complete. Can you see another reason not mentioned so far? Some related links: ...
1
vote
0answers
35 views

Should we start with a small batch-size and increase during training to improve sample efficiency?

Just made an interesting observation playing around with the stable-baseline's implementation of PPO and the BipedalWalker environment from OpenAI's Gym. But I believe this should be a general ...
6
votes
2answers
333 views

Why are reinforcement learning methods sample inefficient?

Reinforcement learning methods are considered to be extremely sample inefficient. For example, in a recent DeepMind paper by Hessel et al., they showed that in order to reach human-level performance ...
4
votes
1answer
662 views

Why are model-based methods more sample efficient than model-free methods?

Why do model-based methods use fewer samples than model-free methods? Here, I'm specifically referring to model-based methods in which we have to learn a policy and model. I can only think of two ...
20
votes
2answers
10k views

What is sample efficiency, and how can importance sampling be used to achieve it?

For instance, the title of this paper reads: "Sample Efficient Actor-Critic with Experience Replay". What is sample efficiency, and how can importance sampling be used to achieve it?