Questions tagged [sample-efficiency]

For questions about the sample efficiency (or inefficiency) of learning algorithms, which is the amount of experience (or data) that the learning algorithm needs in order to reach a certain level of performance. In the case of reinforcement learning, this experience is represented by (transition) tuples of the form $(s, a, r, s')$.

Filter by
Sorted by
Tagged with
2
votes
1answer
77 views

Reinforcement Learning method suitable for a large discrete action space with high sample efficiency

Consider the following problem. We have a process, that generates $N$ stones (e.g. 2000) in one batch $b$. Every pebble has state $s_{i}^b$ and reward $s_i^b$. After choosing one pebble $i$ from the $...
1
vote
1answer
54 views

How to use a heuristic policy to increase sample efficiency of a deep reinforcement learning agent?

I have a heuristic solution to a problem which works quite well when certain environmental parameters are known and unchanging. However, in a real world setting these parameters will not be known and ...
0
votes
0answers
46 views

Does randomly adding hand-engineered features increase the CNN's sample efficiency/performance?

It is a known fact that preprocessing images using CV techniques will improve CNN performance (see this answer). But what happens when you feed in the entire image and the filtered image randomly to ...
2
votes
1answer
58 views

How do I represent sample efficiency of RL rewards in mathematical notation?

I define sample efficiency as the area under the curve/graph, where $x$-axis is the number of episodes while y-axis is the cumulative reward for that episode. I would like to formally define it with a ...
1
vote
0answers
34 views

How can I estimate the minimum number of training samples needed to get interesting results with WGAN?

Let's say we have a WGAN where the generator and critic have 8 layers and 5 million parameters each. I know that the greater the number of training samples the better, but is there a way to know the ...
7
votes
1answer
537 views

How to measure sample efficiency of a reinforcement learning algorithm?

I want to know if there is any metric to use for measuring sample-efficiency of a reinforcement learning algorithm? From reading research papers, I see claims that proposed models are more sample ...
2
votes
0answers
43 views

Can you find another reason for sample inefficiency of model-free on-policy Deep Reinforcement Learning?

The following mindmap gives an overview of multiple reasons for sample inefficiency. The list is definitely not complete. Can you see another reason not mentioned so far? Some related links: ...
1
vote
0answers
35 views

Should we start with a small batch-size and increase during training to improve sample efficiency?

Just made an interesting observation playing around with the stable-baseline's implementation of PPO and the BipedalWalker environment from OpenAI's Gym. But I believe this should be a general ...
6
votes
2answers
415 views

Why are reinforcement learning methods sample inefficient?

Reinforcement learning methods are considered to be extremely sample inefficient. For example, in a recent DeepMind paper by Hessel et al., they showed that in order to reach human-level performance ...
4
votes
1answer
739 views

Why are model-based methods more sample efficient than model-free methods?

Why do model-based methods use fewer samples than model-free methods? Here, I'm specifically referring to model-based methods in which we have to learn a policy and model. I can only think of two ...
20
votes
2answers
10k views

What is sample efficiency, and how can importance sampling be used to achieve it?

For instance, the title of this paper reads: "Sample Efficient Actor-Critic with Experience Replay". What is sample efficiency, and how can importance sampling be used to achieve it?