Questions tagged [reward-design]

For questions about designing (or defining) reward functions e.g. for reinforcement learning problems.

14 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5 votes
1 answer

How define a reward function for a humanoid agent whose goal is to stand up from the ground?

I'm trying to teach a humanoid agent how to stand up after falling. The episode starts with the agent lying on the floor with its back touching the ground, and its goal is to stand up in the shortest ...
Tirafesi's user avatar
  • 151
3 votes
0 answers

Is better to reward short- or long-term progress in Q-learning?

I have been training some kind of agent to reach a target using a Q-learning based approach, and I have tried two different types of rewards: Long-term reward: $\mathrm{reward} = - \mathrm{distance}(\...
Thomas Wagenaar's user avatar
2 votes
0 answers

In RL, is the quantification of the reward function arbitrary? Does it affect the learning?

There are different ways to set the reward function, such as extrinsic (externally provided rewards), intrinsic (the rewards are generated by the agents themselves based on their internal state and ...
John Prada's user avatar
2 votes
0 answers

How can I discourage the RL agent from drawing in a zero-sum game?

My agent receives $1, 0, -1$ rewards for winning, drawing, and losing the game, respectively. What would be the consequences of setting reward to $-1$ for draws? Would that encourage the agent to win ...
mark mark's user avatar
  • 753
2 votes
0 answers

How to combine two differently equally important signals into the reward function, that have different scales?

I have two signals that I want to use to model my reward. The first one is the CPU TIME: running mean from this diagram: The second one is the MAX RESIDUAL from this diagram: Since they are both ...
tmaric's user avatar
  • 382
2 votes
1 answer

How can I implement the reward function for an 8-DOF robot arm with TRPO?

I need to get an 8-DOF (degrees of freedom) robot arm to move a specified point. I need to implement the TRPO RL code using OpenAI gym. I already have the gazebo environment. But I am unsure of how to ...
user1690356's user avatar
1 vote
0 answers

How to teach DRL agent to avoid erratic, high frequency behavior (when costs for taking actions are already included)?

I have a situation where an agent can take actions to enter a state where rewards can be obtained (or costs can be incurred), but the actions themselves have a small cost. In my reward function, I ...
Vladimir Belik's user avatar
1 vote
0 answers

How to teach Machine Learning Agent to destroy replicating objects in a puzzle game?

I have an unusual but very interesting problem. I have a game that is very similar to Toon Blast (a puzzle mobile game). It's based on a Match-2 mechanic in which you can destroy 2 or more connected ...
Jacob's user avatar
  • 153
1 vote
0 answers

Designing a reward function for my reinforcement learning problem

I'm working on a project lately and I'm trying to solve a problem with reinforcement learning and I have serious issues with shaping the reward function. The problem is designing a device with maximum ...
Hossein's user avatar
  • 11
0 votes
0 answers

How to handle penalty and reward occurring simultaneously

Assume the following scenario: We have an agent that acts on an environment where the agent should never take an action that results in him leaving the environment. For example, imagine an agent ...
kklaw's user avatar
  • 133
0 votes
0 answers

Reinforcement Learning with sparse/delayed reward - should intermediate rewards be decayed over time/training?

I'm thinking of a situation like a game (say, chess) where the real objective/reward is actually determined at the very end. I understand that it's important/helpful to do reward shaping with ...
Vladimir Belik's user avatar
0 votes
0 answers

Reward Function for Reinforcement Learning model

I am trying to create a reinforcement learning model to control the acceleration of a car. I am designing the model such that initially the acceleration is provided and then deceleration is provided ...
Aditya Prakash's user avatar
0 votes
0 answers

How to solve a reinforcement learning problem with a stochastic reward function?

In a discrete time system, an environment has an unknown reward probability $p(r|s,a)$. However, the transition probability $p(s'\mid s,a)$ is deterministic. In my case, the reward for the same action ...
Zhenzhen Gong's user avatar
0 votes
0 answers

Thompson sampling with Bernoulli prior and non-binary reward update

I am solving a problem for which I have to select the best possible servers (level 1) to hit for a given data. These servers (level 1) in turn hit some other servers (level 2) to complete the request. ...
PUNEET AGARWAL's user avatar