Questions tagged [reward-functions]
For questions about rewards functions (e.g. in the context of reinforcement learning, which may be denoted as $R(s, a)$).
17
questions
7
votes
2
answers
2k
views
How are the reward functions $R(s)$, $R(s, a)$ and $R(s, a, s')$ equivalent?
In this video, the lecturer states that $R(s)$, $R(s, a)$ and $R(s, a, s')$ are equivalent representations of the reward function. Intuitively, this is the case, according to the same lecturer, ...
- 40.5k
8
votes
2
answers
2k
views
How do we define the reward function for an environment?
How do you actually decide what reward value to give for each action in a given state for an environment?
Is this purely experimental and down to the programmer of the environment? So, is it a ...
- 279
8
votes
2
answers
10k
views
What are some best practices when trying to design a reward function?
Generally speaking, is there a best-practice procedure to follow when trying to define a reward function for a reinforcement-learning agent? What common pitfalls are there when defining the reward ...
2
votes
2
answers
553
views
How should I define the reward function to solve the Wumpus game with deep Q-learning?
I'm writing a DQN agent for the Wumpus game.
Is the reward function to train the Q-networks (target network and policy) the same as the score of the game, i.e. +1000 for picking up gold, -1000 for ...
12
votes
4
answers
2k
views
Counterexamples to the reward hypothesis
On Sutton and Barto's RL book, the reward hypothesis is stated as
that all of what we mean by goals and purposes can be well thought of as the maximization of the expected value of the cumulative ...
- 221
2
votes
1
answer
376
views
How can I implement the reward function for an 8-DOF robot arm with TRPO?
I need to get an 8-DOF (degrees of freedom) robot arm to move a specified point. I need to implement the TRPO RL code using OpenAI gym. I already have the gazebo environment. But I am unsure of how to ...
- 21
7
votes
1
answer
3k
views
Why does a negative reward for every step really encourage the agent to reach the goal as quickly as possible?
If we shift the rewards by any constant (which is a type of reward shaping), the optimal state-action value function (and so optimal policy) does not change. The proof of this fact can be found here.
...
- 40.5k
4
votes
2
answers
1k
views
Why does the definition of the reward function $r(s, a, s')$ involve the term $p(s' \mid s, a)$?
Sutton and Barto define the state–action–next-state reward function, $r(s, a, s')$, as follows (equation 3.6, p. 49)
$$
r(s, a, s^{\prime}) \doteq \mathbb{E}\left[R_{t} \mid S_{t-1}=s, A_{t-1}=a, S_{t}...
- 107
2
votes
1
answer
184
views
Why is the equation $r(s', a, s') =\sum_{r \in \mathcal{R}} r \frac{p\left(s^{\prime}, r \mid s, a\right)}{p\left(s^{\prime} \mid s, a\right)}$true?
I am referring to eq. 3.6 (page 49) based on Sutton's online book and can be found in an image below.
I could not make sense of the final derivation of the equation $r(s, a, s')$. My question is ...
- 123
13
votes
3
answers
2k
views
Why is the reward in reinforcement learning always a scalar?
I'm reading Reinforcement Learning by Sutton & Barto, and in section 3.2 they state that the reward in a Markov decision process is always a scalar real number. At the same time, I've heard about ...
- 233
8
votes
1
answer
7k
views
Suitable reward function for trading buy and sell orders
I am working to build a deep reinforcement learning agent which can place orders (i.e. limit buy and limit sell orders). The actions are ...
- 189
8
votes
1
answer
2k
views
What are other ways of handling invalid actions in scenarios where all rewards are either 0 (best reward) or negative?
I created an OpenAI Gym environment, and I would like to check the performance of the agent from OpenAI Baselines DQN approach on it.
In my environment, the best possible outcome for the agent is 0 - ...
- 263
6
votes
2
answers
2k
views
Why does shifting all the rewards have a different impact on the performance of the agent?
I am new to reinforcement learning. For my application, I have found out that if my reward function contains some negative and positive values, my model does not give the optimal solution, but the ...
- 61
4
votes
2
answers
983
views
How to apply Q-learning when rewards is only available at the last state?
I have a scheduling problem in which there are $n$ slots and $m$ clients. I am trying to solve the problem using Q-learning so I have made the following state-action model.
A state $s_t$ is given by ...
- 301
4
votes
1
answer
2k
views
Is a reward given at every step or only given when the RL agent fails or succeeds?
In reinforcement learning, an agent can receive a positive reward for correct actions and a negative reward for wrong actions, but does the agent also receive rewards for every other step/action?
- 1,283
2
votes
0
answers
269
views
How to solve a reinforcement learning problem with changing rewards?
I'm working on a problem with non-stationary environments. The state space is discrete and limited. The action is limited too. But the reward for the same action $a$ can change. Even the reward for ...
1
vote
1
answer
93
views
In addition to the reward function, which other functions do I need to implement Q-learning?
In general, $Q$ function is defined as
$$Q : S \times A \rightarrow \mathbb{R}$$
$$Q(s_t,a_t) = Q(s_t,a_t) + \alpha[r_{t+1} + \gamma \max\limits_{a} Q(s_{t+1},a) - Q(s_t,a_t)] $$
$\alpha$ and $\gamma$...
- 187