# Model-based RL for time series data

I have time-series data. When I take an action, it impacts the next state, because my action directly determines the next state, but it is not known what the impact is.

To be concrete: I have $$X(t)$$ and $$a(t-1)$$, where $$X(t)$$ is n-dimensional time-series data and $$a(t)$$ is 1-dimensional time-series data. At time $$t$$, they together represent the observation/state space. Also, at time $$t$$, the agent makes a decision about $$a(t)$$. This decision (action) $$a(t)$$ directly defines the next state space $$X(t+1)$$ and rewards, by some function $$f$$ & $$g$$, $$f(a(t), X(t)) = X(t+1)$$ and $$g(a(t), X(t)) = R(t+1)$$.

I have to estimate this impact, i.e. where I will end up (what will be the next state). I decided to use a model-based RL algorithm, because, from my knowledge, model-based RL does exactly this.

Can you advise me on a good paper and Github code, to implement this project?

As I noticed, there do not exist many works on Model-based RL.

• DeepMind's MuZero algorithm learns both a model and policy using RL. – DavidJ Dec 30 '20 at 5:17