Questions tagged [reward-shaping]

For questions related to reward shaping, which is a technique where supplemental rewards are provided to make a problem easier to learn. In general, there is usually an obvious natural reward for any problem. For games, this is usually a win or loss. For financial problems, the reward is usually profit. Reward shaping augments the natural reward signal by adding additional rewards for making progress toward a good solution.

Filter by
Sorted by
Tagged with
8 votes
2 answers
2k views

How do we define the reward function for an environment?

How do you actually decide what reward value to give for each action in a given state for an environment? Is this purely experimental and down to the programmer of the environment? So, is it a ...
Hazzaldo's user avatar
  • 279
8 votes
2 answers
10k views

What are some best practices when trying to design a reward function?

Generally speaking, is there a best-practice procedure to follow when trying to define a reward function for a reinforcement-learning agent? What common pitfalls are there when defining the reward ...
12 rhombi in grid w no corners's user avatar
7 votes
1 answer
3k views

Why does a negative reward for every step really encourage the agent to reach the goal as quickly as possible?

If we shift the rewards by any constant (which is a type of reward shaping), the optimal state-action value function (and so optimal policy) does not change. The proof of this fact can be found here. ...
nbro's user avatar
  • 40.5k
6 votes
2 answers
2k views

Why does shifting all the rewards have a different impact on the performance of the agent?

I am new to reinforcement learning. For my application, I have found out that if my reward function contains some negative and positive values, my model does not give the optimal solution, but the ...
Fishfish's user avatar
4 votes
1 answer
2k views

Is a reward given at every step or only given when the RL agent fails or succeeds?

In reinforcement learning, an agent can receive a positive reward for correct actions and a negative reward for wrong actions, but does the agent also receive rewards for every other step/action?
Dan D.'s user avatar
  • 1,283
2 votes
0 answers
269 views

How to solve a reinforcement learning problem with changing rewards?

I'm working on a problem with non-stationary environments. The state space is discrete and limited. The action is limited too. But the reward for the same action $a$ can change. Even the reward for ...
Zhenzhen Gong's user avatar
1 vote
0 answers
46 views

Reward shaping for dense and sparse rewards

I am working on an RL Problem that drives me nuts. My goal is to control a robot arm in a simulator that has to do 2 things: Hold the arm in a certain position (that is easy and done) If I apply an ...
mavex857's user avatar