2022 Developer Survey is open! Take survey.

# Tag Info

## New answers tagged monte-carlo-methods

1 vote
Accepted

### Given a set of trajectories produced by a fixed policy, what is the the standard approach to estimate Q?

What would be the standard approach in this case? Better use TD learning or Monte Carlo? Both should be fine, but they might lead to different estimates, if both these things apply: The amount of ...
• 23.2k
Your trajectories must contain rewards, so I'm assuming you've forgotten them in your original post, i.e., we must have $$\tau_j = (s_0^j, a_0^j, r_1^j, ..., s_{N_j}, a_{N_j}, r_{N_j+1})$$ Given that ...