Share Your Experience: Take the 2024 Developer Survey

Questions tagged [reward-functions]

For questions about rewards functions (e.g. in the context of reinforcement learning, which may be denoted as $R(s, a)$).

94 questions
Filter by
Sorted by
Tagged with
29 views

Can/should a reward function depend on something other than state in a DQN

Question: Is it OK to have a reward function on a DQN or any RL algorithm that depends on variables other than the enviroment state? I'm asking because, so far I'm learning from tutorials, but I've ...
25 views

A question about the reward calculation in the Hindsight Experience Replay algorithm

I' m try to implement the HER algorithm from scratch in order to use it in the PandaReach-v3 environment. I already developed the same algorithm for the bitflip environment and it works as expected. ...
• 155
86 views

If $p(s'|s,a) = 0$, would the reward the reward $r(s,a,s')$ be infinite? [duplicate]

In chapter 2 of Barto and Sutton's RL book, the four argument probability function $p: S \times R \times S \times A \to [0,1]$ is reduced to three arguments $p: S \times S \times A \to [0,1]$ as ...
17 views

how to choose the reward function to solve an optimization problem with DQN

I am currently working on solving an optimization problem related to the facilities layout using Deep Q-Networks (DQN). The primary challenge I am facing revolves around designing an effective reward ...
28 views

Using multi-dimensional rewards to prevent intermediary reward bias

I came up with a new type of reward scheme to avoid intermediary reward bias and am unable to find any literature references to it. My questions are Is this novel? Is this useful? The approach is as ...
61 views

How and whether to apply Reinforcement Learning in an Environment with a precise and always available Evaluation?

Say we want to train an agent $A$ in an environment $E$ which provides a continuous loss $L$. That is, we want $A$ to choose its actions $a$ so that it minimizes the mistake it does, i.e., it ...
• 143
1 vote
58 views

What is the name of the reward function that utilizes the rewards of the next n steps?

I have a problem with continuous time, observation and action space. I am discretizing the time to be able to apply the usual Reinforcement Learning algorithms (I chose PPO). The problem consists of a ...
59 views

Why is R(s) more restrictive than R(s, a) in an MDP?

I am quite new to RL. I would like to know why a state-dependent reward function R(s) is more restrictive than a state-action-dependent reward function R(s, a)? And why is it that a policy can be ...
58 views

How to handle penalty and reward occurring simultaneously

Assume the following scenario: We have an agent that acts on an environment where the agent should never take an action that results in him leaving the environment. For example, imagine an agent ...
• 185
27 views

If the agent is at the same state but at different times and receives a different reward, wouldn't this be violating somehow the MDP assumption?

I've been trying to train an agent, I've received and read suggestions to improve its speed to reach the goal. The suggestion is to use a time penalty, for example, adding $-0.1$ to the reward each ...
35 views

Could you explain the derivation of the expectation equation of equation 3.6 in Sutton & Barto? [duplicate]

I don't understand the last equality. Here is my derivation $r(s,a,s')=\sum_{r\in R} r p(r|s,a,s')=\sum_{r\in R} r \frac{p(s,a,s'|r)p(r)}{p(s,a,s')}$ Could you give me the correct steps to derive them?...
• 205
124 views

Can a reward function have various cases?

I'm doing a Q-learning algorithm and I'm designing my reward function. Basically I'm working on optimizing a network while changing some parameters. My metric to measure its optimization is the delay ...
1 vote
29 views

What does "shuffle the comparisons into one dataset" mean?

I couldn't understand the wording here. What does "shuffle the comparisons into one dataset" mean? How does the method they use don't have $K \choose 2$ forward passes for K completions? Do ...
1 vote
64 views

How to setup a reinforcement learning model that changes the values of $x$ to maximize $y$ in $y = f(x)$?

Assuming a relation such that $y = f(x)$, where $y$ represents a scalar and $x \in 20 \times 1$ vector consisting of zeros and ones, I want to set up a reinforcement learning model that changes the ...
• 111
74 views

In RL, is the quantification of the reward function arbitrary? Does it affect the learning?

There are different ways to set the reward function, such as extrinsic (externally provided rewards), intrinsic (the rewards are generated by the agents themselves based on their internal state and ...
230 views

Will my Q values keep going up forever?

In Q-learning,the q values can be updated by the bellman equation. What happens with my Q values is that they keep going up forever, in accordance with the more I train. After 10,000 training episodes,...
1 vote
270 views

Is it necessary to have a constant reward in the terminal state?

I have downloaded the grid world project form this link. I have executed the project multiple times using: python gridworld.py -k 20 -a q -r -0.2 -s 90 I have ...
• 111
1 vote
45 views

Would the optimal policy remain same, if I replace R with V*?

In the context of RL, say I'm performing Value Iteration on a reward function R1. And the converged optimal policy is P1 and values are V1. Then, let's say I set rewards to be R2=V1 and perform value ...
1 vote