# Can we use a Gaussian process to approximate the belief distribution at every instant in a POMDP?

Suppose $$x_{t+1} \sim \mathbb{P}(\cdot | x_t, a_t)$$ denotes the state transition dynamics in a reinforcement learning (RL) problem. Let $$y_{t+1} = \mathbb{P}(\cdot | x_{t+1})$$ denote the noisy observation or the imperfect state information. Let $$H_{t}$$ denote the history of actions and observations $$H_{t+1} = \{b_0,y_0,a_0,\cdots,y_{t+1}\}$$.

For the RL Partially Observed Markov Decision Process (RL-POMDP), the summary of the history is contained in the "belief state" $$b_{t+1}(i) = \mathbb{P}(x_{t+1} = i | H_{t+1})$$, which is the posterior distribution over the states conditioned on the history.

Now, suppose the model is NOT known. Clearly, the belief state can't be computed.

Can we use a Gaussian Process (GP) to approximate the belief distribution $$b_{t}$$ at every instant $$t$$?

Can Variational GP be adapted to such a situation? Can universal approximation property of GP be invoked here?

Are there such results in the literature?

Any references and insights into this problem would be much appreciated.

• How? If you have no model you literally know nothing. You have no idea how to assign a probability to any state or even what a state is. – FourierFlux May 12 '20 at 9:58
• @FourierFlux: Say you have access to the following (training) data: Input {a_t,y_t} and output {x_t}. It's just during the implementation (testing), we only have access to imperfect information. Could a GP, which captures the distribution over the states, match the belief state? at every time t? Are there such works? – math_phile May 12 '20 at 15:09
• @math_phile You need a model for that. GP is a model. You said you have no model. You need to make some assumptions about what you want to predict. – Innate Imunity is The Way Jan 13 at 10:25