I was reading this paper by Hansen.
It says the following:
A correspondence between vectors and one-step policy choices plays an important role in this interpretation of a policy. Each vector in $\mathcal{V}'$ corresponds to the choice of an action, and for each possible observation, choice of a vector in $\mathcal{V}$. Among all possible one-step policy choices, the vectors in $\mathcal{V}'$ correspond to those that optimize the value of some belief state. To describe this correspondence between vectors and one-step policy choices, we introduce the following notation. For each vector $\mathcal{v}_i$ in $\mathcal{V}'$, let $a(i)$ denote the choice of action and, for each possible observation $z$, let $l(i,z)$ denote the index of the successor vector in $\mathcal{V}$. Given this correspondence between vectors and one-step policy choices, Kaelbling et al. (1996) point out that an optimal policy for a finite-horizon POMDP can be represented by an acyclic finite-state controller in which each machine state corresponds to a vector in a nonstationary value function.
I am unable to guess how the left-side finite-state controller is formed from the right side belief space diagram. Does the above text provide enough explanation for the conversion? If yes, I am not really able to fully get it. Can someone please explain?
