Linked Questions
14 questions linked to/from What is the difference between a stochastic and a deterministic policy?
8
votes
1
answer
7k
views
What is the difference between a stationary and a non-stationary policy?
In reinforcement learning, there are deterministic and non-deterministic (or stochastic) policies, but there are also stationary and non-stationary policies.
What is the difference between a ...
- 39.6k
3
votes
2
answers
1k
views
What is a probability distribution in machine learning?
If we were learning or working in the machine learning field, then we frequently come across the term "probability distribution". I know what probability, conditional probability, and ...
- 1,036
8
votes
2
answers
917
views
What is experience replay in laymen's terms?
I've been reading Google's DeepMind Atari paper and I'm trying to understand the concept of "experience replay". Experience replay comes up in a lot of other reinforcement learning papers (...
- 241
5
votes
2
answers
993
views
Given two optimal policies, is an affine combination of them also optimal?
If there are two different optimal policies $\pi_1, \pi_2$ in a reinforcement learning task, will the linear combination (or affine combination) of the two policies $\alpha \pi_1 + \beta \pi_2, \alpha ...
- 53
3
votes
2
answers
1k
views
When should one prefer using Total Variational Divergence over KL divergence in RL
In RL, both the KL divergence (DKL) and Total variational divergence (DTV) are used to measure the distance between two policies. I'm most familiar with using DKL as an early stopping metric during ...
- 531
1
vote
1
answer
2k
views
What kind of problems is DQN algorithm good and bad for?
I know this is a general question, but I'm just looking for intuition. What are the characteristics of problems (in terms of state-space, action-space, environment, or anything else you can think of) ...
- 763
3
votes
1
answer
1k
views
Why do RL implementations converge on one action?
I have seen this happening in implementations of state-of-the-art RL algorithms where the model converges to a single action over time after multiple training iterations. Are there some general ...
3
votes
1
answer
384
views
Does stochasticity of an environment necessarily mean non-stationarity in MDPs?
Is a stochastic environment necessarily also non-stationary? To elaborate, consider a two-state environment ($s_1$ and $s_2$), with two actions $a_1$ and $a_2$. In $s_1$, taking action $a_1$ has a ...
- 75
4
votes
2
answers
505
views
What is the difference between return and expected return?
At a time step $t$, for a state $S_{t}$, the return is defined as the discounted cumulative reward from that time step $t$.
If an agent is following a policy (which in itself is a probability ...
- 41
4
votes
1
answer
435
views
Is tabular Q-learning considered interpretable?
I am working on a research project in a domain where other related works have always resorted to deep Q-learning. The motivation of my research stems from the fact that the domain has an inherent ...
- 141
0
votes
1
answer
320
views
Which policy do I need to use in updating Q function?
Policy function can be of two types: deterministic policy and stochastic policy.
Deterministic policy is of the form $\pi : S \rightarrow A$
Stochastic policy is defined using conditional probability ...
- 187
1
vote
1
answer
312
views
What is the difference between the definition of a stationary policy in reinforcement learning and contextual bandit?
A stationary policy is a function that maps a state to a probability distribution of actions.
In a contextual bandit problem, a state itself does not include the history. But in a reinforcement ...
- 227
0
votes
1
answer
229
views
What is the equation for $\pi_*$ in terms of $q_*(s,a)$?
I am trying to solve the following exercise from Sutton and Barto:
Sutton and Barto Exercise 3.27 Give an equation for $\pi_*$ in terms of $q_*(s,a)$
However, I am struggling to do so. I know that $\...
- 145
-1
votes
1
answer
227
views
Is my understanding correct regarding the difference between policy and plan?
I am confused regarding the difference between policy and plan in reinforcement learning. According to my understanding, when we calculate the value of state using Bellman equation in deterministic ...
- 111