# When can we unnest the minimizations/recursions in an value function(bellman optimality equation)?

When reading the following paper(page 4): An Approximate Dynamic Programming Approach for Dual Stochastic Model Predictive Control

I could see that they were able to unnest the minimization's in the bellmans equations due to sampling of the expected values. It would be cumbersome to write down their whole setup so i thought i might understand why with another example:

Consider the value function below:

\begin{align} V^{*}(s_0) &= \min_{a_0 \in A_{s_0}}[r(s_0, a_0) + \sum_{s_1}p(s_1|s_0, a_0)V^{*}(s_1)] \\ &= \min_{a_0 \in A_{s_0}}[r(s_0, a_0) + \sum_{s_1}p(s_1|s_0, a_0)]\min_{a_1 \in A_{s_1}}[r(s_1, a_1) + \sum_{s_2}p(s_2|s_1, a_1)V^{*}(s_2)] \\ &(?\neq, = ?) \min_{a_0 \in A_{s_0}, a_1 \in A_{s_1}}[r(s_0, a_0) + \sum_{s_1}p(s_1|s_0, a_0)][r(s_1, a_1) + \sum_{s_2}p(s_2|s_1, a_1)V^{*}(s_2)] \end{align} when should it be an $$\neq$$ or an $$=$$ in the third equation and why, i.e how can sampling break the recursion and allow us to unnest the minimizations?

(in the paper, they argue that since they use samples to approximate the expected value of the cost-to-go/next state value function they are allowed to unnest the minimizations)

• Crossposted (duplicate) at OR.SE: When can we unnest the minimizations/recursions in value functions(bellman optimality)? Commented Dec 1, 2023 at 17:07
• @SecretAgentMan This question is perfectly fine for this site. There's arguably no better site to ask this question than AI SE.
– nbro
Commented Dec 1, 2023 at 22:04
• removed the question on operations research Commented Dec 3, 2023 at 7:32