# How does a model based agent learn the model?

I want to build model-based RL. I am wondering about the process of building the model.

If I already have data, from real experience:

• $$S_1, a \rightarrow R,S_2$$
• $$S_2, a \rightarrow R,S_3$$

Can I use this information, to build model-based RL? Or it is necessary that the agent directly interact with the environment (I mean the same above-mentioned data should be provided by the agent)?

• u may use q-network (model), fit the model with known real experience before the RL training process instead of a regular step of randomising the weights – datdinhquoc Apr 7 at 9:26
• @datdinhquoc that is wrong, Q-learning is model free. – David Ireland Apr 7 at 9:32
• that q-network has n-action softmax neurons in output layer of qvalues which can be applied to argmax to get max action, isn't it the model – datdinhquoc Apr 7 at 9:34
• no, when people refer to a model they are talking about modelling the environment dynamics such as $p(s'|s, a)$. – David Ireland Apr 7 at 9:35