Questions tagged [stochastic-policy]
For questions related to the concept of a stochastic policy (as defined in reinforcement learning), which is a function from a state to a probability distribution over actions (from that state).
12
questions
0
votes
1
answer
53
views
Consequence of Dvoretzky Stochastic Approximation Theorem
I am trying to understand all the steps to prove the TD0 algorithm, and I am following a proof which uses a theorem of Tommi Jaakkola, Michael I. Jordan and Satinder P. Singh, in the paper: On the ...
- 19
0
votes
0
answers
36
views
What is an example of an *optimal* stochastic policy that assigns a nonzero probability to an action with a lower expected value?
A stochastic policy means that an agent has probabilities of choosing their available actions, given a state: $\pi(a|s)$.
However in an optimal stochastic policy for a given state, you would assume ...
- 133
3
votes
2
answers
256
views
How do we estimate the value of a stochastic policy?
I'm learning about reinforcement learning, particularly policy gradient methods and actor-critic methods. I've noticed that many algortihms use stochastic policies during training (i.e. they select ...
3
votes
1
answer
107
views
How is $v_*(s) = \max_{\pi} v_\pi(s)$ also applicable in the case of stochastic policies?
I am reading Sutton & Bartos's Book "Introduction to reinforcement learning". In this book, the defined the optimal value function as:
$$v_*(s) = \max_{\pi} v_\pi(s),$$ for all $s \in \...
- 33
2
votes
1
answer
103
views
Is a learned policy, for a deterministic problem, trained in a supervised process, a stochastic policy?
If I trained a neural network with 4 outputs (one for each action: move down, up, left, and right) to move an agent through a grid (deterministic problem). The output of the neural network is a ...
- 21
2
votes
1
answer
489
views
Did Alphago zero actually beat Alphago 100 games to 0?
tl;dr
Did AlphaGo and AlphaGo play 100 repetitions of the same sequence of boards, or were there 100 different games?
Background:
Alphago was the first superhuman go player, but it had human tuning ...
- 361
3
votes
1
answer
135
views
In the policy gradient equation, is $\pi(a_{t} | s_{t}, \theta)$ a distribution or a function?
I am learning about policy gradient methods from the Deep RL Bootcamp by Peter Abbeel and I am a bit stumbled by the math presented. In the lecture, he derives the gradient logarithm likelihood of a ...
- 1,251
3
votes
1
answer
145
views
What's the value of making the RL agent's output stochastic opposed to deterministic?
I have a question about a reinforcement learning problem.
I'm training an agent to add or delete pixels in a [12 x 12] 2D space (going to be 3D in the future).
Its action space consists of two ...
- 265
3
votes
1
answer
376
views
Is it possible for value-based methods to learn stochastic policies?
Is it possible for value-based methods to learn stochastic policies? I'm trying to get a clear picture of the different categories for RL algorithms, and while doing so I started to think about ...
- 211
8
votes
3
answers
15k
views
What is the difference between a stochastic and a deterministic policy?
In reinforcement learning, there are the concepts of stochastic (or probabilistic) and deterministic policies. What is the difference between them?
- 39.1k
16
votes
3
answers
4k
views
Is the optimal policy always stochastic if the environment is also stochastic?
Is the optimal policy always stochastic (that is, a map from states to a probability distribution over actions) if the environment is also stochastic?
Intuitively, if the environment is ...
- 39.1k
3
votes
1
answer
2k
views
Can Q-learning be used to derive a stochastic policy?
In my understanding, Q-learning gives you a deterministic policy. However, can we use some technique to build a meaningful stochastic policy from the learned Q values? I think that simply using a ...
- 153