Skip to main content

Questions tagged [potential-reward-shaping]

For questions about potential-based reward shaping, which was introduced in the paper "Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping" by Andrew Y. Ng et al, in 1999.

Filter by
Sorted by
Tagged with
0 votes
0 answers
27 views

Using reinforcement learning to optimize a state-based potential function

I am using reinforcement learning for a project, and the objective function I want to optimize is a potential-based function. In other words, it is preferable for the agent to move to positions with ...
LY Omega's user avatar
6 votes
1 answer
1k views

How to improve the reward signal when the rewards are sparse?

In cases where the reward is delayed, this can negatively impact a models ability to do proper credit assignment. In the case of a sparse reward, are there ways in which this can be negated? In a ...
tryingtolearn's user avatar
3 votes
1 answer
159 views

Expressing Arbitrary Reward Functions as Potential-Based Advice (PBA)

I am trying to reproduce the results for the simple grid-world environment in [1]. But it turns out that using a dynamically learned PBA makes the performance worse and I cannot obtain the results ...
bcxiao's user avatar
  • 33
2 votes
1 answer
599 views

Why does potential-based reward shaping seem to alter the optimal policy in this case?

It is known that every potential function won't alter the optimal policy [1]. I lack of understanding why is that. The definition: $$R' = R + F,$$ with $$F = \gamma\Phi(s') - \Phi(s),$$ where, let's ...
ScientiaEtVeritas's user avatar
3 votes
2 answers
2k views

What should I do when the potential value of a state is too high?

I'm working on a Reinforcement Learning task where I use reward shaping as proposed in the paper Policy invariance under reward transformations: Theory and application to reward shaping (1999) by ...
Marco Favorito's user avatar