Skip to main content
Share Your Experience: Take the 2024 Developer Survey
4 votes

Bellman equation and inverse matrix method

In reinforcement learning, the apostrophe character $(')$ appended to a signal usually represents the signal at the next time step. For example, $s'$ is the state in the time step immediately after $s$...
DeepQZero's user avatar
  • 1,399
2 votes
Accepted

Proof of existence of optimal policy

There's still some missing link since from what you described above we merely know that $v^*$ is the unique solution to the BOE with Bellman optimality operator. Whether it is the optimal state value ...
cinch's user avatar
  • 2,277
2 votes
Accepted

Proof of bellman optimality equations

This theorem is basically the policy improvement theorem in S&L's RL book page 78, though expressed differently. The reasons for your confused line 4 & 5 are essentially same as your reference'...
cinch's user avatar
  • 2,277
1 vote
Accepted

Calculating state value using bellman equation question:

Should the * be initialized as 0 and then calculated using the equation? This would work if you repeated the calculation multiple times, and that would be similar to a solution method called value ...
Neil Slater's user avatar
  • 32.5k
1 vote

Why must the value of a state under an optimal policy equal the expected return for the best action from that state?

I was also confused when Sutton and Barto claimed in chapter 3, as you stated, that the value of a state under an optimal policy must equal the expected return for the best action from that state. ...
AMT's user avatar
  • 111

Only top scored, non community-wiki answers of a minimum length are eligible