Newest 'exploration-exploitation-tradeoff' Questions

5

votes

3answers

455 views

In Q-learning, wouldn't it be better to simply iterate through all possible states?

In Q-learning, all resources I've found seem to say that the algorithm to update the Q-table should start at some initial state, and pick actions (which are sometimes random) to explore the state ...

asked Feb 1 at 8:12

Kricket

1654 bronze badges

0

votes

0answers

22 views

PPO: sampling next action vs picking the most probable action

According to the original Proximal Policy Optimization paper (PPO paper), we always sample an action from the actor distribution. According to the link The overall loss is calculated as $\text{loss} =...

reinforcement-learning papers proximal-policy-optimization exploration-exploitation-tradeoff

asked Jan 28 at 13:58

FQT

235 bronze badges

1

vote

0answers

34 views

Why is the ideal exploration parameter in the UCT algorithm $\sqrt{2}$?

From Wikipedia, in the Monte-Carlo Tree Search algorithm, you should choose the node that maximizes the value: $${\displaystyle {\frac {w_{i}}{n_{i}}}+c{\sqrt {\frac {\ln N_{i}}{n_{i}}}}},$$ where ${...

reinforcement-learning monte-carlo-tree-search exploration-exploitation-tradeoff upper-confidence-bound

asked Dec 10 '20 at 16:25

Gilad Felsen

111 bronze badge

1

vote

0answers

44 views

What is the difference between exploitation and exploration in the context of optimization?

In the paper Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm (2015, published in Knowledge-Based Systems) The test functions are divided to three groups: unimodal, multi-...

comparison optimization exploration-exploitation-tradeoff meta-heuristics moth-flame-optimization

asked Nov 7 '20 at 18:52

user2752471

1392 bronze badges

1

vote

0answers

28 views

Should the exploration rate be updated at the end of the episode or at every step?

My agent uses an $\epsilon$-greedy strategy to learn. The exploration rate (i.e. $\epsilon$) decays throughout the training. I've seen examples where people update $\epsilon$ every time an action is ...

reinforcement-learning dqn exploration-exploitation-tradeoff epsilon-greedy-policy

asked Oct 30 '20 at 19:40

mark mark

6031 silver badge18 bronze badges

3

votes

1answer

58 views

Can we stop training as soon as epsilon is small?

I'm new to reinforcement learning. As it is common in RL, $\epsilon$-greedy search for the behavior/exploration is used. So, at the beginning of the training, $\epsilon$ is high, and therefore a lot ...

reinforcement-learning q-learning value-functions exploration-exploitation-tradeoff epsilon-greedy-policy

asked Oct 1 '20 at 8:32

Micha Christ

311 bronze badge

4

votes

2answers

232 views

What is the meaning of “exploration” in reinforcement and supervised learning?

While exploration is an integral part of reinforcement learning (RL), it does not pertain to supervised learning (SL) since the latter is already provided with the data set from the start. That said, ...

reinforcement-learning terminology supervised-learning hyperparameter-optimization exploration-exploitation-tradeoff

asked Sep 21 '20 at 7:38

Tfovid

1674 bronze badges

1

vote

1answer

49 views

What is the optimal exploration-exploitation trade-off in Q*bert?

I am training an RL agent with Deep Q-learning + Experience Replay on the Q*bert Atari environment. After 400,000 frames, my agent appears to have learned strategic information about the game, but ...

reinforcement-learning q-learning dqn deep-rl exploration-exploitation-tradeoff

asked Aug 27 '20 at 19:39

Ryan Rudes

1655 bronze badges

0

votes

0answers

56 views

How to modify the Actor-Critic policy gradient algorithm to perform Safe exploration in Reinforcement Learning

I am trying to implement safe exploration technique in [Ref.1]. I am using Soft Actor-Critic algorithm to teach an agent to introduce a bias between 0 and 1 to a specific state of interest in my ...

actor-critic-methods exploration-exploitation-tradeoff

asked Aug 19 '20 at 10:59

Ayomi Al-noor

11 bronze badge

2

votes

1answer

54 views

Why do some DQN implementations not require random exploration but instead emulate all actions?

I've found online some DQN algorithms that (in a problem with a continuous state space and few actions, let's say 2 or 3), at each time step, compute and store (in the memory used for updating) all ...

reinforcement-learning dqn deep-rl exploration-exploitation-tradeoff

asked Jul 5 '20 at 9:25

aandre_90

2915 bronze badges

2

votes

1answer

75 views

Why is 100% exploration bad during the learning stage in reinforcement learning?

Why can't we during the first 1000 episodes allow our agent to perform only exploration? This will give a better chance of covering the entire space state. Then, after the number of episodes, we can ...

reinforcement-learning exploration-exploitation-tradeoff

asked Jun 29 '20 at 2:07

Chukwudi

2295 bronze badges

3

votes

2answers

102 views

Why is it not advisable to have a 100 percent exploration rate? [duplicate]

During the learning phase, why don't we have a 100% exploration rate, to allow our agent to fully explore our environment and update the Q values, then during testing we bring in exploitation? Does ...

reinforcement-learning q-learning exploration-exploitation-tradeoff

asked Jun 26 '20 at 16:58

Chukwudi

2295 bronze badges

1

vote

1answer

36 views

How to deal with the addition of a new state to the environment during training?

Let's say we have a dynamic environment: a new state gets added after 2000 episodes have been done. So, we leave room for exploration, so that it can discover the new state. When it gets to that new ...

reinforcement-learning environment exploration-exploitation-tradeoff

asked Jun 25 '20 at 13:30

Chukwudi

2295 bronze badges

1

vote

2answers

57 views

Why do we explore after we have an accurate estimate of the value function?

Suppose we have a small space state and that, after about 2000 episodes, we've accurately explored the environment and known the accurate $Q$ values. In that case, why do we still leave a small ...

reinforcement-learning q-learning exploration-exploitation-tradeoff

asked Jun 24 '20 at 13:25

Chukwudi

2295 bronze badges

1

vote

2answers

74 views

Why can't we fully exploit the environment after the first episode in Q-learning?

During the first episode, it's 100% exploration, because all our Q values are 0. Suppose we have 1000 time steps, and it's terminated by meeting a reward. So, after the first episode, why can't we ...

reinforcement-learning q-learning exploration-exploitation-tradeoff

asked Jun 16 '20 at 14:39

Chukwudi

2295 bronze badges

1

vote

2answers

90 views

Should I use exploration strategy in Policy Gradient algorithms?

In policy gradient algorithms the output is a stochastic policy - a probability for each action. I believe that if I follow the policy (sample an action from the policy) I make use of exploration ...

reinforcement-learning policy-gradients exploration-exploitation-tradeoff upper-confidence-bound thompson-sampling

asked Jun 6 '20 at 21:38

gnikol

1254 bronze badges

1

vote

1answer

97 views

Can tabular Q-learning converge even if it doesn't explore all state-action pairs?

My understanding of tabular Q-learning is that it essentially builds a dictionary of state-action pairs, so as to maximize the Markovian (i.e., step-wise, history-agnostic?) reward. This incremental ...

machine-learning reinforcement-learning q-learning exploration-exploitation-tradeoff

asked Jun 1 '20 at 8:40

Tfovid

1674 bronze badges

1

vote

0answers

35 views

How can I increase the exploration in the Proximal Policy Optimation algorithm?

How can I increase the exploration in the Proximal Policy Optimation reinforcement learning algorithm? Is there a variable assigned for this purpose? I'm using the stable-baseline implementation: ...

reinforcement-learning proximal-policy-optimization exploration-exploitation-tradeoff

asked May 21 '20 at 9:14

Saeid Ghafouri

1134 bronze badges

1

vote

1answer

37 views

Should I just use exploitation after I have trained the Q agent?

When using a trained Q-learning algorithm in an actual game, would I just use exploitation and no longer use exploration? Should I use exploration only during the training phase?

reinforcement-learning training q-learning exploration-exploitation-tradeoff

asked May 1 '20 at 23:16

mason7663

5411 silver badge7 bronze badges

1

vote

0answers

270 views

Why is the $\epsilon$ hyper-parameter (in the $\epsilon$-greedy policy) annealed smoothly?

As far as I understand, RL is a process that can be divided into 2 stages: Exploring a wide range of paths (acting randomly) Refining the current optimal paths (revolving around actions with a so-...

reinforcement-learning dqn hyper-parameters exploration-exploitation-tradeoff epsilon-greedy-policy

asked Jul 10 '19 at 22:23

Kari

1506 bronze badges

7

votes

1answer

202 views

What is the purpose of the actor in actor-critic algorithms?

For discrete action spaces, what is the purpose of the actor in actor-critic algorithms? My current understanding is that the critic estimates the future reward given an action, so why not just take ...

reinforcement-learning actor-critic-methods exploration-exploitation-tradeoff

asked May 18 '19 at 23:07

David Rein

836 bronze badges

2

votes

1answer

252 views

Does eligibility traces and epsilon-greedy do the same task in different ways?

I understand that, in Reinforcement Learning algorithms, such as Q-learning, to prevent selecting the actions with greatest q-values too fast and allow for exploration, we use eligibility traces. Here ...

reinforcement-learning comparison exploration-exploitation-tradeoff epsilon-greedy-policy eligibility-traces

asked Oct 21 '17 at 7:39

Abhishek Dhyani

313 bronze badges

Stack Exchange Network

current community

your communities

more stack exchange communities

Questions tagged [exploration-exploitation-tradeoff]

In Q-learning, wouldn't it be better to simply iterate through all possible states?

PPO: sampling next action vs picking the most probable action

Why is the ideal exploration parameter in the UCT algorithm $\sqrt{2}$?

What is the difference between exploitation and exploration in the context of optimization?

Should the exploration rate be updated at the end of the episode or at every step?

Can we stop training as soon as epsilon is small?

What is the meaning of “exploration” in reinforcement and supervised learning?

What is the optimal exploration-exploitation trade-off in Q*bert?

How to modify the Actor-Critic policy gradient algorithm to perform Safe exploration in Reinforcement Learning

Why do some DQN implementations not require random exploration but instead emulate all actions?

Why is 100% exploration bad during the learning stage in reinforcement learning?

Why is it not advisable to have a 100 percent exploration rate? [duplicate]

How to deal with the addition of a new state to the environment during training?

Why do we explore after we have an accurate estimate of the value function?

Why can't we fully exploit the environment after the first episode in Q-learning?

Should I use exploration strategy in Policy Gradient algorithms?

Can tabular Q-learning converge even if it doesn't explore all state-action pairs?

How can I increase the exploration in the Proximal Policy Optimation algorithm?

Should I just use exploitation after I have trained the Q agent?

Why is the $\epsilon$ hyper-parameter (in the $\epsilon$-greedy policy) annealed smoothly?

What is the purpose of the actor in actor-critic algorithms?

Does eligibility traces and epsilon-greedy do the same task in different ways?

Hot Network Questions

Questions tagged [exploration-exploitation-tradeoff]

Related Tags