Questions tagged [bellman-operators]
For questions related to Bellman operators in the context of reinforcement learning.
5
questions
3
votes
1
answer
302
views
What do the terms 'Bellman backup' and 'Bellman error' mean?
Some RL literature use terms such as: 'Bellman backup' and 'Bellman error'. What do these terms refer to?
- 339
1
vote
0
answers
113
views
Off-policy Bellman Operators: Writing Operator and Weight Update Function for a 2-State System
I am studying for RL on my own and was trying to solve this question I came across.
Write an operator function $T(w, \pi, \mu, l, g)$ that takes weights $w$, a target policy $\pi$, a behaviour policy ...
1
vote
1
answer
202
views
How to derive matrix form of the Bellman operators?
Reading the Retrace paper (Safe and efficient off-policy reinforcement learning) I saw they often use a matrix form of the Bellman operators, for example as in the picture below. How do we derive ...
- 223
8
votes
2
answers
812
views
Why are the Bellman operators contractions?
In these slides, it is written
\begin{align}
\left\|T^{\pi} V-T^{\pi} U\right\|_{\infty} & \leq \gamma\|V-U\|_{\infty} \tag{9} \label{9} \\
\|T V-T U\|_{\infty} & \leq \gamma\|V-U\|_{\infty} \...
- 181
23
votes
1
answer
10k
views
What is the Bellman operator in reinforcement learning?
In mathematics, the word operator can refer to several distinct but related concepts. An operator can be defined as a function between two vector spaces, it can be defined as a function where the ...
- 33.8k