# Questions tagged [bellman-equations]

For questions related to the Bellman equations in the context of reinforcement learning (and other artificial intelligence subfields).

46 questions
Filter by
Sorted by
Tagged with
26 views

### Can we also estimate $V_{\pi}$ with SARSA?

For SARSA, I know we can estimate the action value $Q(s,a)$, and the relationship between $V(s)$ and $Q(s,a)$ is $V_{\pi}(s) = \sum_{a \in \mathcal{A}} \pi(a|s)Q_{\pi} (s,a)$. So my question is, can ...
48 views

### What is so special about the Bellman Optimality Principle?

In the context of Decision Making and Game Theory, "Bellman's Equations and Bellman's Conditions of Optimality" are said to be some of the most important mathematical principles in this ...
• 287
165 views

### Determining to terminate at a reward or not

I am practicing the Bellman equation on Grid world examples and in this scenario, there are numbered grid squares where the agent can choose to terminate and collect the reward equal to the amount ...
• 123
72 views

### What is the Bellman equation for V(s) in the case of a deterministic environment?

I am currently trying to practice reinforcement learning for an agent on a grid. The grid is deterministic. Since the grid is deterministic, to calculate the value for each grid square from the reward ...
• 123
24 views

### Determine Gridworld values

I am learning Reinforcement learning for games following Gridworld examples. Apologies in advance if this is a basic question, very new to reinforcement learning. I am slightly confused in scenarios ...
• 123
312 views

• 133
1 vote
82 views

### What is the difference between gradient decent in neural networks and temporal difference in reinforcement learning?

I am studying Q-learning in reinforcement learning. My question is about the Bellman equation. In Q-learning, the Bellman equation is often introduced as follows. \begin{align} Q_{new}(s,a) &= Q_{...
1 vote
69 views

356 views

### How can we find the value function by solving a system of linear equations?

I am following the book "Reinforcement Learning: An Introduction" by Richard Sutton and Andrew Barto, and they give an example of a problem for which the value function can be computed ...
• 143
193 views

### What do the terms 'Bellman backup' and 'Bellman error' mean?

Some RL literature use terms such as: 'Bellman backup' and 'Bellman error'. What do these terms refer to?
• 339
101 views

• 393