Share Your Experience: Take the 2024 Developer Survey

# Tag Info

### Bellman equation and inverse matrix method

In reinforcement learning, the apostrophe character $(')$ appended to a signal usually represents the signal at the next time step. For example, $s'$ is the state in the time step immediately after $s$...
• 1,399
Accepted

### Proof of existence of optimal policy

There's still some missing link since from what you described above we merely know that $v^*$ is the unique solution to the BOE with Bellman optimality operator. Whether it is the optimal state value ...
• 2,277
Accepted

### Proof of bellman optimality equations

This theorem is basically the policy improvement theorem in S&L's RL book page 78, though expressed differently. The reasons for your confused line 4 & 5 are essentially same as your reference'...
• 2,277
1 vote
Accepted

### Calculating state value using bellman equation question:

Should the * be initialized as 0 and then calculated using the equation? This would work if you repeated the calculation multiple times, and that would be similar to a solution method called value ...
• 32.5k
1 vote

### Why must the value of a state under an optimal policy equal the expected return for the best action from that state?

I was also confused when Sutton and Barto claimed in chapter 3, as you stated, that the value of a state under an optimal policy must equal the expected return for the best action from that state. ...
• 111

Only top scored, non community-wiki answers of a minimum length are eligible