Questions tagged [bellman-operators]
For questions related to Bellman operators in the context of reinforcement learning.
4
questions
2
votes
0answers
37 views
Off-policy Bellman Operators: Writing Operator and Weight Update Function for a 2-State System
I am studying for RL on my own and was trying to solve this question I came across.
Write an operator function $T(w, \pi, \mu, l, g)$ that takes weights $w$, a target policy $\pi$, a behaviour policy ...
1
vote
1answer
68 views
How to derive matrix form of the Bellman operators?
Reading the Retrace paper (Safe and efficient off-policy reinforcement learning) I saw they often use a matrix form of the Bellman operators, for example as in the picture below. How do we derive ...
5
votes
2answers
151 views
Why are the Bellman operators contractions?
In these slides, it is written
\begin{align}
\left\|T^{\pi} V-T^{\pi} U\right\|_{\infty} & \leq \gamma\|V-U\|_{\infty} \tag{9} \label{9} \\
\|T V-T U\|_{\infty} & \leq \gamma\|V-U\|_{\infty} \...
19
votes
1answer
6k views
What is the Bellman operator in reinforcement learning?
In mathematics, the word operator can refer to several distinct but related concepts. An operator can be defined as a function between two vector spaces, it can be defined as a function where the ...
