Questions tagged [exploration-exploitation-tradeoff]
For questions related to the exploration-exploitation trade-off (or dilemma) in the context of reinforcement learning.
13
questions
1
vote
1answer
39 views
What is the optimal exploration-exploitation trade-off in Q*bert?
I am training an RL agent with Deep Q-learning + Experience Replay on the Q*bert Atari environment. After 400,000 frames, my agent appears to have learned strategic information about the game, but ...
0
votes
0answers
50 views
How to modify the Actor-Critic policy gradient algorithm to perform Safe exploration in Reinforcement Learning
I am trying to implement safe exploration technique in [Ref.1]. I am using Soft Actor-Critic algorithm to teach an agent to introduce a bias between 0 and 1 to a specific state of interest in my ...
2
votes
1answer
52 views
Why do some DQN implementations not require random exploration but instead emulate all actions?
I've found online some DQN algorithms that (in a problem with a continuous state space and few actions, let's say 2 or 3), at each time step, compute and store (in the memory used for updating) all ...
2
votes
1answer
70 views
Why is 100% exploration bad during the learning stage in reinforcement learning?
Why can't we during the first 1000 episodes allow our agent to perform only exploration?
This will give a better chance of covering the entire space state. Then, after the number of episodes, we can ...
3
votes
2answers
90 views
Why is it not advisable to have a 100 percent exploration rate? [duplicate]
During the learning phase, why don't we have a 100% exploration rate, to allow our agent to fully explore our environment and update the Q values, then during testing we bring in exploitation? Does ...
1
vote
1answer
35 views
How to deal with the addition of a new state to the environment during training?
Let's say we have a dynamic environment: a new state gets added after 2000 episodes have been done. So, we leave room for exploration, so that it can discover the new state.
When it gets to that new ...
1
vote
2answers
57 views
Why do we explore after we have an accurate estimate of the value function?
Suppose we have a small space state and that, after about 2000 episodes, we've accurately explored the environment and known the accurate $Q$ values. In that case, why do we still leave a small ...
1
vote
2answers
67 views
Why can't we fully exploit the environment after the first episode in Q-learning?
During the first episode, it's 100% exploration, because all our Q values are 0. Suppose we have 1000 time steps, and it's terminated by meeting a reward. So, after the first episode, why can't we ...
1
vote
2answers
76 views
Should I use exploration strategy in Policy Gradient algorithms?
In policy gradient algorithms the output is a stochastic policy - a probability for each action.
I believe that if I follow the policy (sample an action from the policy) I make use of exploration ...
1
vote
1answer
88 views
Can tabular Q-learning converge even if it doesn't explore all state-action pairs?
My understanding of tabular Q-learning is that it essentially builds a dictionary of state-action pairs, so as to maximize the Markovian (i.e., step-wise, history-agnostic?) reward. This incremental ...
1
vote
0answers
32 views
How can I increase the exploration in the Proximal Policy Optimation algorithm?
How can I increase the exploration in the Proximal Policy Optimation reinforcement learning algorithm? Is there a variable assigned for this purpose? I'm using the stable-baseline implementation: ...
1
vote
1answer
33 views
Should I just use exploitation after I have trained the Q agent?
When using a trained Q-learning algorithm in an actual game, would I just use exploitation and no longer use exploration? Should I use exploration only during the training phase?
7
votes
1answer
150 views
What is the purpose of the actor in actor-critic algorithms?
For discrete action spaces, what is the purpose of the actor in actor-critic algorithms?
My current understanding is that the critic estimates the future reward given an action, so why not just take ...
