Skip to main content
Artificial Intelligence

Return to Question

deleted 3 characters in body
Source Link
bonzo_pippinpaddle
bonzo_pippinpaddle
  • 59
  • 6

The Sutton and Barto reinforcement learning textbook states that

the value of a state under an optimal policy must equal the expected return for the best action from that state.

That is, $$v_*(s) = \max_a q_*(s, a).$$

I am having trouble gaining an intuition for this. Since state values can be written as an expectation of the action values under a given policy, I am not sure I see how

$$v_*(s) = \sum_a \pi_*(a|s)q_*(s,a) = \max_a q_*(s, a).$$

I'd appreciate any insights!

The Sutton and Barto reinforcement learning textbook states that

the value of a state under an optimal policy must equal the expected return for the best action from that state.

That is, $$v_*(s) = \max_a q_*(s, a).$$

I am having trouble gaining an intuition for this. Since state values can be written as an expectation of the action values under a given policy, I am not sure I see how

$$v_*(s) = \sum_a \pi_*(a|s)q_*(s,a) = \max_a q_*(s, a).$$

I'd appreciate any insights!

The Sutton and Barto reinforcement learning textbook states that

the value of a state under an optimal policy must equal the expected return for the best action from that state.

That is, $$v_*(s) = \max_a q_*(s, a).$$

I am having trouble gaining intuition for this. Since state values can be written as an expectation of the action values under a given policy, I am not sure I see how

$$v_*(s) = \sum_a \pi_*(a|s)q_*(s,a) = \max_a q_*(s, a).$$

I'd appreciate any insights!

edited body; edited tags; edited title
Source Link
nbro
nbro
  • 41.4k
  • 12
  • 114
  • 205

Reinforcement Learning: relationship between optimal Why must the value of a state and action values inunder an MDP (Sutton and Barto)optimal policy equal the expected return for the best action from that state?

The Sutton and Barto reinforcement learning textbook states that ``the value of a state under an optimal policy must equal the expected return for the best action from that state''.

the value of a state under an optimal policy must equal the expected return for the best action from that state.

That is, $$v_*(s) = \max_a q_*(s, a).$$ I

I am having trouble gaining an intuition for this. Since state values can be written as an expectation of the action values under a given policy, I am not sure I see how   

$$v_*(s) = \sum_a \pi_*(a|s)q_*(s,a) = \max_a q_*(s, a).$$ I'd

I'd appreciate any insights! Thanks so much!

Reinforcement Learning: relationship between optimal state and action values in an MDP (Sutton and Barto)

The Sutton and Barto reinforcement learning textbook states that ``the value of a state under an optimal policy must equal the expected return for the best action from that state''. That is, $$v_*(s) = \max_a q_*(s, a).$$ I am having trouble gaining intuition for this. Since state values can be written as an expectation of the action values under a given policy, I am not sure I see how  $$v_*(s) = \sum_a \pi_*(a|s)q_*(s,a) = \max_a q_*(s, a).$$ I'd appreciate any insights! Thanks so much!

Why must the value of a state under an optimal policy equal the expected return for the best action from that state?

The Sutton and Barto reinforcement learning textbook states that

the value of a state under an optimal policy must equal the expected return for the best action from that state.

That is, $$v_*(s) = \max_a q_*(s, a).$$

I am having trouble gaining an intuition for this. Since state values can be written as an expectation of the action values under a given policy, I am not sure I see how 

$$v_*(s) = \sum_a \pi_*(a|s)q_*(s,a) = \max_a q_*(s, a).$$

I'd appreciate any insights!

Source Link
bonzo_pippinpaddle
bonzo_pippinpaddle
  • 59
  • 6

Reinforcement Learning: relationship between optimal state and action values in an MDP (Sutton and Barto)

The Sutton and Barto reinforcement learning textbook states that ``the value of a state under an optimal policy must equal the expected return for the best action from that state''. That is, $$v_*(s) = \max_a q_*(s, a).$$ I am having trouble gaining intuition for this. Since state values can be written as an expectation of the action values under a given policy, I am not sure I see how $$v_*(s) = \sum_a \pi_*(a|s)q_*(s,a) = \max_a q_*(s, a).$$ I'd appreciate any insights! Thanks so much!