New answers tagged value-functions
1
My understanding from your question is that you have the following data generated from a random policy:
$$[s_0, s_1, s_2 . . . s_n]$$
That is, the state observed at each time step.
You know nothing more about the MDP, such as the transition or reward functions. Although the MDP is discrete and fully observable (and thus usual RL theory is supported), you do ...
0
What am I missing here?
You are not missing anything mathematically.
Potentially what you are missing is that the discount factor $\gamma$, is part of the problem definition. In reinforcement learning (RL), you do not always solve problems to obtain the highest total sum of rewards. Instead you solve problems to obtain the highest expected return on any ...
2
The value function is defined as $v_\pi(s) = \mathbb{E}_\pi[G_t | S_t = s]$ where $G_t$ are the (discounted) returns from time step $t$. The expectation is taken with respect to the policy $\pi$ and the transition dynamics of the MDP.
Now, as you pointed out the optimal value function is defined as $v_*(s) = \max_\pi v_\pi(s)\; ; \;\forall s \in \mathcal{S}$....
Top 50 recent answers are included
