15
votes
Accepted
What is the difference between a stochastic and a deterministic policy?
A deterministic policy is a function of the form $\pi_{\mathbb{d}}: S \rightarrow A$, that is, a function from the set of states of the environment, $S$, to the set of actions, $A$. The subscript $_{\...
- 39.5k
11
votes
Is the optimal policy always stochastic if the environment is also stochastic?
Is the optimal policy always stochastic (that is, a map from states to a probability distribution over actions) if the environment is also stochastic?
No.
An optimal policy is generally ...
- 30.2k
8
votes
Accepted
Is Q-learning only capable of learning a deterministic policy?
If we assume a tabular setting, then Q-learning converges to the optimal state-action value function, from which an optimal policy can be derived, provided a few conditions are met.
In finite MDPs, ...
- 39.5k
7
votes
Did Alphago zero actually beat Alphago 100 games to 0?
Did AlphaGo and AlphaGo [Zero] play 100 repetitions of the same sequence of boards, or were there 100 different games?
There were 100 different games. You can view some example games between AlphaGo [...
- 30.2k
6
votes
Is the optimal policy always stochastic if the environment is also stochastic?
I would say no.
For example, consider the multi-armed bandit problem. So, you have $n$ arms which all have a probability of giving you a reward (1 point, for example), $p_i$, $i$ being between 1 and ...
- 251
3
votes
Accepted
Determining to terminate at a reward or not
I am trying to understand how you would determine whether it is better for the agent to terminate at the state with the number 3 or to continue to the state with a number 4 to collect the more reward?
...
- 30.2k
3
votes
Accepted
What is the loss for policy gradients with continuous actions?
This update rule can still be applied in the continuous domain.
As pointed out in the comments, suppose we are parameterising our policy using a Gaussian distribution, where our neural networks take ...
- 4,665
3
votes
Accepted
What is the motivation behind using a deterministic policy?
You're right! Behaving according to a deterministic policy while still learning would be a terrible idea in most cases (with the exception of environments that "do the exploring for you"; see comments)...
- 2,028
2
votes
Accepted
Can Q-learning be used to derive a stochastic policy?
No it is not possible to use Q-learning to build a deliberately stochastic policy, as the learning algorithm is designed around choosing solely the maximising value at each step, and this assumption ...
- 30.2k
2
votes
What is the difference between a stochastic and a deterministic policy?
Deterministic Policy :
Its means that for every state you have clear defined action you will take
For Example: We 100% know we will take action A from state X.
Stochastic Policy :
Its mean that for ...
- 121
1
vote
Accepted
How is policy iteration capable of improving on a deterministic policy?
These statements are not true for policy iteration and dynamic programming:
Since the policy is stochastic and the initial state is the same, we'll always take the same path and evaluate the same ...
- 30.2k
1
vote
Is a learned policy, for a deterministic problem, trained in a supervised process, a stochastic policy?
Is the policy (based in the neural network) a stochastic policy? even if the action space is discrete?
Yes. A discrete action space does not require a deterministic policy - it is possible to assign ...
- 30.2k
1
vote
What is the difference between a stochastic and a deterministic policy?
Apart from the answers above,
Stochastic Policy function: $\pi (s_1s_2 \dots s_n, a_1 a_2 \dots a_n): \mathcal S \times \mathcal A \rightarrow [0,1]$ is the probability distribution function, that, ...
- 141
Only top scored, non community-wiki answers of a minimum length are eligible