Model-Free RL##RL

In Model-Free RL, the agent does not have access to a model of the environment. By environment I mean a function which predicts state transition and rewards.

As of the time of writing, model-free methods are more popular and have been researched extensively.

Model-Based RL##RL

In Model-Based RL, the agent has access to a model of the environment.

Main advantage is that this allows the agent to plan ahead by thinking ahead. Agents distill the results from planning ahead into a learned policy. A famous example of Model-Based RL is AlphaZero.

The main downside is that many times a ground-truth representation of the environment is not usually available.

Below is a non-exhaustive taxonomy of RL algorithms, which may help you to visualize better the RL landscape.