Skip to main content
Artificial Intelligence

Return to Question

I've made the title and certain parts of the body more explicit and clearer.
Source Link
nbro
nbro
  • 41.4k
  • 12
  • 114
  • 205

off Can the importance sampling estimator have a non-policy evaluation in reinforcement learningstationary behaviour policy even if the target policy is stationary?

IPSThe inverse propensity score (IPS) estimator, which is used for off-policy evaluation in a contextual bandit problem, is well explained here: Doubly Robust Policy Evaluation andOptimizationin the paper https://arxiv.org/pdf/1503.02834.pdfDoubly Robust Policy Evaluation and Optimization.

The old policy $\mu$, or the behavior policy, is okay to be non-stationary in the IPS estimator even if the new policy $\nu$, or the target policy, should be stationary.

I wonder if that isIs this true for ISthe importance sampling (IS) estimator, which seems to be a variant of IPS, for off-policy evaluation in a reinforcement learning problem. IS?

IS estimator is explained here: Doubly Robust Off-policy Value Evaluation for Reinforcement Learningin this paper https://arxiv.org/abs/1511.03722Doubly Robust Off-policy Value Evaluation for Reinforcement Learning. The

The target policy should be stationary, but can the old policy be non-stationary in the IS estimator?

off-policy evaluation in reinforcement learning

IPS estimator, which is used for off-policy evaluation in a contextual bandit problem, is well explained here: Doubly Robust Policy Evaluation andOptimization https://arxiv.org/pdf/1503.02834.pdf

The old policy $\mu$, or the behavior policy, is okay to be non-stationary in the IPS estimator even if the new policy $\nu$, or the target policy, should be stationary.

I wonder if that is true for IS estimator, which seems to be a variant of IPS, for off-policy evaluation in a reinforcement learning problem. IS estimator is explained here: Doubly Robust Off-policy Value Evaluation for Reinforcement Learning https://arxiv.org/abs/1511.03722. The target policy should be stationary, but can the old policy be non-stationary in the IS estimator?

Can the importance sampling estimator have a non-stationary behaviour policy even if the target policy is stationary?

The inverse propensity score (IPS) estimator, which is used for off-policy evaluation in a contextual bandit problem, is well explained in the paper Doubly Robust Policy Evaluation and Optimization.

The old policy $\mu$, or the behavior policy, is okay to be non-stationary in the IPS estimator even if the new policy $\nu$, or the target policy, should be stationary.

Is this true for the importance sampling (IS) estimator, which seems to be a variant of IPS, for off-policy evaluation in a reinforcement learning problem?

IS estimator is explained in this paper Doubly Robust Off-policy Value Evaluation for Reinforcement Learning.

The target policy should be stationary, but can the old policy be non-stationary in the IS estimator?

edited tags
Link
Hunnam
Hunnam
  • 227
  • 1
  • 6
Source Link
Hunnam
Hunnam
  • 227
  • 1
  • 6

off-policy evaluation in reinforcement learning

IPS estimator, which is used for off-policy evaluation in a contextual bandit problem, is well explained here: Doubly Robust Policy Evaluation andOptimization https://arxiv.org/pdf/1503.02834.pdf

The old policy $\mu$, or the behavior policy, is okay to be non-stationary in the IPS estimator even if the new policy $\nu$, or the target policy, should be stationary.

I wonder if that is true for IS estimator, which seems to be a variant of IPS, for off-policy evaluation in a reinforcement learning problem. IS estimator is explained here: Doubly Robust Off-policy Value Evaluation for Reinforcement Learning https://arxiv.org/abs/1511.03722. The target policy should be stationary, but can the old policy be non-stationary in the IS estimator?