Newest 'reward-functions' Questions

0

votes

0answers

48 views

Is my reward function non-Markovian?

I am working on an RL problem where the time when the agent obtains the reward for taking action $a$ in time step $t$ is stochastic. In fact, there is no immediate reward for taking action $a$ in time ...

asked Jan 10 at 22:10

shirin elahi

1

vote

1answer

49 views

If the reward function of an environment depends on some initial conditions, should I create a separate environment for each condition?

I would like some guidance on how to design an Environment for a Reinforcement Learning agent where the stopping conditions and rewards for the environment change based on an initial set of input ...

reinforcement-learning environment reward-design reward-functions

asked Nov 24 '20 at 15:46

RL_NOOB

153 bronze badges

0

votes

0answers

34 views

Thompson sampling with Bernoulli prior and non-binary reward update

I am solving a problem for which I have to select the best possible servers (level 1) to hit for a given data. These servers (level 1) in turn hit some other servers (level 2) to complete the request. ...

reinforcement-learning reward-design reward-functions bayesian-statistics thompson-sampling

asked Nov 16 '20 at 7:56

PUNEET AGARWAL

1

2

votes

0answers

22 views

How can I discourage the RL agent from drawing in a zero-sum game?

My agent receives $1, 0, -1$ rewards for winning, drawing, and losing the game, respectively. What would be the consequences of setting reward to $-1$ for draws? Would that encourage the agent to win ...

reinforcement-learning reward-design reward-functions zero-sum-games

asked Nov 15 '20 at 22:33

mark mark

5861 silver badge17 bronze badges

6

votes

1answer

131 views

Why does a negative reward for every step really encourage the agent to reach the goal as quickly as possible?

If we shift the rewards by any constant (which is a type of reward shaping), the optimal state-action value function (and so optimal policy) does not change. The proof of this fact can be found here. ...

reinforcement-learning proofs reward-shaping reward-functions

asked Nov 1 '20 at 23:09

nbro♦

23.5k5 gold badges42 silver badges100 bronze badges

0

votes

1answer

97 views

What is the optimal value function of the shifted version of the reward function?

Similarly to this question that I asked some time ago, what is the optimal value function of the shifted (by some constant $c$) version of some reward function? More precisely, let's assume that $r(s, ...

reinforcement-learning proofs bellman-equations reward-shaping reward-functions

asked Nov 1 '20 at 0:28

nbro♦

23.5k5 gold badges42 silver badges100 bronze badges

2

votes

2answers

94 views

How should I define the reward function to solve the Wumpus game with deep Q-learning?

I'm writing a DQN agent for the Wumpus game. Is the reward function to train the Q-networks (target network and policy) the same as the score of the game, i.e. +1000 for picking up gold, -1000 for ...

reinforcement-learning dqn deep-rl reward-design reward-functions

asked Oct 20 '20 at 7:40

Edwin Carlsson

212 bronze badges

2

votes

1answer

70 views

What are proxy reward functions?

The understanding I have is that they somehow adjust the objective to make it easier to meet, without changing the reward function. ... the observed proxy reward function is the approximate solution ...

reinforcement-learning definitions papers reward-functions

asked Sep 20 '20 at 14:53

mugoh

3371 silver badge13 bronze badges

3

votes

1answer

230 views

What are the pros and cons of sparse and dense rewards in reinforcement learning?

From what I understand, if the rewards are sparse the agent will have to explore more to get rewards and learn the optimal policy, whereas if the rewards are dense in time, the agent is quickly guided ...

reinforcement-learning comparison reward-functions sparse-rewards dense-rewards

asked Aug 13 '20 at 7:05

strawberry-sunshine

83012 bronze badges

1

vote

0answers

34 views

How to combine two differently equally important signals into the reward function, that have different scales?

I have two signals that I want to use to model my reward. The first one is the CPU TIME: running mean from this diagram: The second one is the MAX RESIDUAL from this diagram: Since they are both ...

reinforcement-learning deep-rl reward-design reward-functions

asked Aug 7 '20 at 11:48

tmaric

3228 bronze badges

9

votes

3answers

244 views

Why is the reward in reinforcement learning always a scalar?

I'm reading Reinforcement Learning by Sutton & Barto, and in section 3.2 they state that the reward in a Markov decision process is always a scalar real number. At the same time, I've heard about ...

reinforcement-learning rewards reward-functions multi-objective-rl

asked Aug 6 '20 at 22:06

Sid Mani

1932 bronze badges

4

votes

2answers

202 views

What are some best practices when trying to design a reward function?

Generally speaking, is there a best-practice procedure to follow when trying to define a reward function for a reinforcement-learning agent? What common pitfalls are there when defining the reward ...

reinforcement-learning reward-design reward-functions reward-shaping inverse-rl

asked Aug 3 '20 at 16:30

12 rhombi in grid w no corners

1555 bronze badges

2

votes

1answer

155 views

Is a reward given at every step or only given when the RL agent fails or succeeds?

In reinforcement learning, an agent can receive a positive reward for correct actions and a negative reward for wrong actions, but does the agent also receive rewards for every other step/action?

reinforcement-learning reward-design reward-functions reward-shaping dense-rewards

asked Jul 21 '20 at 9:59

datdinhquoc

7354 silver badges14 bronze badges

3

votes

2answers

83 views

Can rewards be decomposed into components?

I'm training a robot to walk to a specific $(x, y)$ point using TD3, and, for simplicity, I have something like ...

reinforcement-learning reward-design reward-functions multi-objective-rl reward-hypothesis

asked Jul 9 '20 at 16:49

pinkie pAI

353 bronze badges

4

votes

1answer

117 views

Why does shifting all the rewards have a different impact on the performance of the agent?

I am new to reinforcement learning. For my application, I have found out that if my reward function contains some negative and positive values, my model does not give the optimal solution, but the ...

reinforcement-learning dqn rewards reward-shaping reward-functions

asked Jul 1 '20 at 1:57

Fishfish

411 bronze badge

4

votes

1answer

56 views

How should I handle invalid actions in a grid world?

I'm building a really simple experiment, where I let an agent move from the bottom-left corner to the upper-right corner of a $3 \times 3$ grid world. I plan to use DQN to do this. I'm having trouble ...

reinforcement-learning q-learning dqn reward-design reward-functions

asked May 19 '20 at 3:39

o_yeah

1874 bronze badges

3

votes

1answer

66 views

Why is the reward function $\text{reward} = 1/{(\text{cost}+1)^2}$ better than $\text{reward} =1/(\text{cost}+1)$?

I have implemented a simple Q-learning algorithm to minimize a cost function by setting the reward to the inverse of the cost of the action taken by the agent. The algorithm converges nicely, but ...

reinforcement-learning q-learning rewards reward-design reward-functions

asked May 14 '20 at 9:01

EArwa

774 bronze badges

1

vote

1answer

62 views

What is the relationship between the reward function and the value function?

To clarify it in my head, the value function calculates how 'good' it is to be in a certain state by summing all future (discounted) rewards, while the reward function is what the value function uses ...

reinforcement-learning comparison value-functions reward-functions

asked Apr 3 '20 at 22:06

mason7663

5091 silver badge7 bronze badges

2

votes

1answer

77 views

How can I implement the reward function for an 8-DOF robot arm with TRPO?

I need to get an 8-DOF (degrees of freedom) robot arm to move a specified point. I need to implement the TRPO RL code using OpenAI gym. I already have the gazebo environment. But I am unsure of how to ...

reinforcement-learning gym reward-design reward-functions trust-region-policy-optimization

asked Mar 5 '20 at 2:10

user1690356

211 bronze badge

10

votes

4answers

554 views

Counterexamples to the reward hypothesis

On Sutton and Barto's RL book, the reward hypothesis is stated as that all of what we mean by goals and purposes can be well thought of as the maximization of the expected value of the cumulative ...

reinforcement-learning sutton-barto reward-design reward-functions reward-hypothesis

asked Nov 19 '19 at 2:30

Bananin

2011 silver badge3 bronze badges

2

votes

1answer

63 views

What is the difference between success rate and reward when dealing with binary and sparse rewards?

In OpenAI Gym "reward" is defined as: reward (float): amount of reward achieved by the previous action. The scale varies between environments, but the ...

reinforcement-learning terminology papers reward-functions hindsight-experience-replay

asked Oct 30 '19 at 12:09

rrz0

2231 silver badge6 bronze badges

2

votes

1answer

244 views

What is the optimal value function of the scaled version of the reward function?

Consider the reward function $r(s, a)$ with optimal state-action value function $q_*(s, a)$. What would be the optimal state-action value function of $c r(s, a)$, for $c \in \mathbb{R}$? Would it be $...

reinforcement-learning proofs bellman-equations reward-shaping reward-functions

asked Sep 15 '19 at 20:58

nbro♦

23.5k5 gold badges42 silver badges100 bronze badges

2

votes

1answer

103 views

How does the initialization of the value function and definition of the reward function affect the performance of the RL agent?

Is there any empirical/theoretical evidence on the effect of initial values of state-action and state values on the training of an RL agent (the values an RL agent assigns to visited states) via MC ...

reinforcement-learning value-functions reward-design reward-functions weights-initialization

asked Jul 6 '19 at 20:47

DuttaA

5,0873 gold badges17 silver badges45 bronze badges

1

vote

1answer

622 views

What is the reward system of reinforcement learning?

Can you describe this reward system in more detail? I understand that the environment sends a signal indicating whether or not the action taken by the agent was 'good' or not, but it seems too simple. ...

reinforcement-learning markov-decision-process rewards reward-functions

asked May 10 '19 at 23:43

12 rhombi in grid w no corners

1555 bronze badges

4

votes

1answer

287 views

Can the rewards be stochastic when the transition model is deterministic?

Suppose we have a deterministic environment where knowing $s,a$ determines $s'$. Is it possible to get two different rewards $r\neq r'$ in some state $s_{\text{fixed}}$? Assume that $s_{\text{fixed}}$ ...

reinforcement-learning markov-decision-process reward-design reward-functions

asked Feb 12 '19 at 10:33

rewardbabe

411 bronze badge

2

votes

1answer

47 views

Why does potential-based reward shaping seem to alter the optimal policy in this case?

It is known that every potential function won't alter the optimal policy [1]. I lack of understanding why is that. The definition: $$R' = R + F,$$ with $$F = \gamma\Phi(s') - \Phi(s),$$ where, let's ...

reinforcement-learning reward-functions reward-shaping potential-reward-shaping

asked Jan 26 '19 at 14:11

ScientiaEtVeritas

1555 bronze badges

3

votes

1answer

106 views

Why is the equation $r(s', a, s') =\sum_{r \in \mathcal{R}} r \frac{p\left(s^{\prime}, r \mid s, a\right)}{p\left(s^{\prime} \mid s, a\right)}$true?

I am referring to eq. 3.6 (page 49) based on Sutton's online book and can be found in an image below. I could not make sense of the final derivation of the equation $r(s, a, s')$. My question is ...

reinforcement-learning math notation reward-functions probability-theory

asked Oct 19 '18 at 9:34

alfa_80

1333 bronze badges

8

votes

1answer

2k views

What are other ways of handling invalid actions in scenarios where all rewards are either 0 (best reward) or negative?

I created an OpenAI Gym environment, and I would like to check the performance of the agent from OpenAI Baselines DQN approach on it. In my environment, the best possible outcome for the agent is 0 - ...

reinforcement-learning q-learning dqn implementation reward-functions

asked May 29 '17 at 9:02

AlexGuevara

2631 silver badge8 bronze badges

Stack Exchange Network

current community

your communities

more stack exchange communities

Questions tagged [reward-functions]

Is my reward function non-Markovian?

If the reward function of an environment depends on some initial conditions, should I create a separate environment for each condition?

Thompson sampling with Bernoulli prior and non-binary reward update

How can I discourage the RL agent from drawing in a zero-sum game?

Why does a negative reward for every step really encourage the agent to reach the goal as quickly as possible?

What is the optimal value function of the shifted version of the reward function?

How should I define the reward function to solve the Wumpus game with deep Q-learning?

What are proxy reward functions?

What are the pros and cons of sparse and dense rewards in reinforcement learning?

How to combine two differently equally important signals into the reward function, that have different scales?

Why is the reward in reinforcement learning always a scalar?

What are some best practices when trying to design a reward function?

Is a reward given at every step or only given when the RL agent fails or succeeds?

Can rewards be decomposed into components?

Why does shifting all the rewards have a different impact on the performance of the agent?

How should I handle invalid actions in a grid world?

Why is the reward function $\text{reward} = 1/{(\text{cost}+1)^2}$ better than $\text{reward} =1/(\text{cost}+1)$?

What is the relationship between the reward function and the value function?

How can I implement the reward function for an 8-DOF robot arm with TRPO?

Counterexamples to the reward hypothesis

What is the difference between success rate and reward when dealing with binary and sparse rewards?

What is the optimal value function of the scaled version of the reward function?

How does the initialization of the value function and definition of the reward function affect the performance of the RL agent?

What is the reward system of reinforcement learning?

Can the rewards be stochastic when the transition model is deterministic?

Why does potential-based reward shaping seem to alter the optimal policy in this case?

Why is the equation $r(s', a, s') =\sum_{r \in \mathcal{R}} r \frac{p\left(s^{\prime}, r \mid s, a\right)}{p\left(s^{\prime} \mid s, a\right)}$true?

What are other ways of handling invalid actions in scenarios where all rewards are either 0 (best reward) or negative?

Hot Network Questions

Questions tagged [reward-functions]

Related Tags