12 votes
Accepted

What is the difference between a stochastic and a deterministic policy?

A deterministic policy is a function of the form $\pi_{\mathbb{d}}: S \rightarrow A$, that is, a function from the set of states of the environment, $S$, to the set of actions, $A$. The subscript $_{\...
user avatar
  • 33.7k
9 votes

Is the optimal policy always stochastic if the environment is also stochastic?

Is the optimal policy always stochastic (that is, a map from states to a probability distribution over actions) if the environment is also stochastic? No. An optimal policy is generally ...
user avatar
  • 23.8k
7 votes
Accepted

Is Q-learning only capable of learning a deterministic policy?

If we assume a tabular setting, then Q-learning converges to the optimal state-action value function, from which an optimal policy can be derived, provided a few conditions are met. In finite MDPs, ...
user avatar
  • 33.7k
6 votes

Is the optimal policy always stochastic if the environment is also stochastic?

I would say no. For example, consider the multi-armed bandit problem. So, you have $n$ arms which all have a probability of giving you a reward (1 point, for example), $p_i$, $i$ being between 1 and ...
user avatar
6 votes

Did Alphago zero actually beat Alphago 100 games to 0?

Did AlphaGo and AlphaGo [Zero] play 100 repetitions of the same sequence of boards, or were there 100 different games? There were 100 different games. You can view some example games between AlphaGo [...
user avatar
  • 23.8k
3 votes
Accepted

Determining to terminate at a reward or not

I am trying to understand how you would determine whether it is better for the agent to terminate at the state with the number 3 or to continue to the state with a number 4 to collect the more reward? ...
user avatar
  • 23.8k
3 votes
Accepted

What is the loss for policy gradients with continuous actions?

This update rule can still be applied in the continuous domain. As pointed out in the comments, suppose we are parameterising our policy using a Gaussian distribution, where our neural networks take ...
user avatar
3 votes
Accepted

What is the motivation behind using a deterministic policy?

You're right! Behaving according to a deterministic policy while still learning would be a terrible idea in most cases (with the exception of environments that "do the exploring for you"; see comments)...
user avatar
2 votes
Accepted

Can Q-learning be used to derive a stochastic policy?

No it is not possible to use Q-learning to build a deliberately stochastic policy, as the learning algorithm is designed around choosing solely the maximising value at each step, and this assumption ...
user avatar
  • 23.8k
2 votes

What is the difference between a stochastic and a deterministic policy?

Deterministic Policy : Its means that for every state you have clear defined action you will take For Example: We 100% know we will take action A from state X. Stochastic Policy : Its mean that for ...
user avatar
1 vote
Accepted

How is policy iteration capable of improving on a deterministic policy?

These statements are not true for policy iteration and dynamic programming: Since the policy is stochastic and the initial state is the same, we'll always take the same path and evaluate the same ...
user avatar
  • 23.8k
1 vote

Is a learned policy, for a deterministic problem, trained in a supervised process, a stochastic policy?

Is the policy (based in the neural network) a stochastic policy? even if the action space is discrete? Yes. A discrete action space does not require a deterministic policy - it is possible to assign ...
user avatar
  • 23.8k
1 vote

What is the difference between a stochastic and a deterministic policy?

Apart from the answers above, Stochastic Policy function: $\pi (s_1s_2 \dots s_n, a_1 a_2 \dots a_n): \mathcal S \times \mathcal A \rightarrow [0,1]$ is the probability distribution function, that, ...
user avatar
  • 141

Only top scored, non community-wiki answers of a minimum length are eligible