Questions tagged [bellman-operators]
For questions related to Bellman operators in the context of reinforcement learning.
5
questions
1
vote
1answer
21 views
What do the terms 'Bellman backup' and 'Bellman error' mean?
Some RL literature use terms such as: 'Bellman backup' and 'Bellman error'. What do these terms refer to?
2
votes
0answers
39 views
Off-policy Bellman Operators: Writing Operator and Weight Update Function for a 2-State System
I am studying for RL on my own and was trying to solve this question I came across.
Write an operator function $T(w, \pi, \mu, l, g)$ that takes weights $w$, a target policy $\pi$, a behaviour policy ...
1
vote
1answer
84 views
How to derive matrix form of the Bellman operators?
Reading the Retrace paper (Safe and efficient off-policy reinforcement learning) I saw they often use a matrix form of the Bellman operators, for example as in the picture below. How do we derive ...
5
votes
2answers
159 views
Why are the Bellman operators contractions?
In these slides, it is written
\begin{align}
\left\|T^{\pi} V-T^{\pi} U\right\|_{\infty} & \leq \gamma\|V-U\|_{\infty} \tag{9} \label{9} \\
\|T V-T U\|_{\infty} & \leq \gamma\|V-U\|_{\infty} \...
20
votes
1answer
7k views
What is the Bellman operator in reinforcement learning?
In mathematics, the word operator can refer to several distinct but related concepts. An operator can be defined as a function between two vector spaces, it can be defined as a function where the ...
