# Questions tagged [reward-functions]

For questions about rewards functions (e.g. in the context of reinforcement learning, which may be denoted as $R(s, a)$).

87 questions
Filter by
Sorted by
Tagged with
52 views

### Why is R(s) more restrictive than R(s, a) in an MDP?

I am quite new to RL. I would like to know why a state-dependent reward function R(s) is more restrictive than a state-action-dependent reward function R(s, a)? And why is it that a policy can be ...
45 views

### How to handle penalty and reward occurring simultaneously

Assume the following scenario: We have an agent that acts on an environment where the agent should never take an action that results in him leaving the environment. For example, imagine an agent ...
19 views

### If the agent is at the same state but at different times and receives a different reward, wouldn't this be violating somehow the MDP assumption?

I've been trying to train an agent, I've received and read suggestions to improve its speed to reach the goal. The suggestion is to use a time penalty, for example, adding $-0.1$ to the reward each ...
34 views

### Could you explain the derivation of the expectation equation of equation 3.6 in Sutton & Barto? [duplicate]

I don't understand the last equality. Here is my derivation $r(s,a,s')=\sum_{r\in R} r p(r|s,a,s')=\sum_{r\in R} r \frac{p(s,a,s'|r)p(r)}{p(s,a,s')}$ Could you give me the correct steps to derive them?...
79 views

### Can a reward function have various cases?

I'm doing a Q-learning algorithm and I'm designing my reward function. Basically I'm working on optimizing a network while changing some parameters. My metric to measure its optimization is the delay ...
1 vote
28 views

### What does "shuffle the comparisons into one dataset" mean?

I couldn't understand the wording here. What does "shuffle the comparisons into one dataset" mean? How does the method they use don't have $K \choose 2$ forward passes for K completions? Do ...
1 vote
56 views

### How to setup a reinforcement learning model that changes the values of $x$ to maximize $y$ in $y = f(x)$?

Assuming a relation such that $y = f(x)$, where $y$ represents a scalar and $x \in 20 \times 1$ vector consisting of zeros and ones, I want to set up a reinforcement learning model that changes the ...
70 views

### In RL, is the quantification of the reward function arbitrary? Does it affect the learning?

There are different ways to set the reward function, such as extrinsic (externally provided rewards), intrinsic (the rewards are generated by the agents themselves based on their internal state and ...
113 views

### Will my Q values keep going up forever?

In Q-learning,the q values can be updated by the bellman equation. What happens with my Q values is that they keep going up forever, in accordance with the more I train. After 10,000 training episodes,...
1 vote
183 views

### Is it necessary to have a constant reward in the terminal state?

I have downloaded the grid world project form this link. I have executed the project multiple times using: python gridworld.py -k 20 -a q -r -0.2 -s 90 I have ...
1 vote
42 views

### Would the optimal policy remain same, if I replace R with V*?

In the context of RL, say I'm performing Value Iteration on a reward function R1. And the converged optimal policy is P1 and values are V1. Then, let's say I set rewards to be R2=V1 and perform value ...
1 vote
168 views

### Is there a reward function that would encourage exploration in this case?

I am new to Reinforcement Learning. I am trying to train PPO agent for citylearn. The goal is to lower two environmental variables from observations. The default reward function is ...
1 vote
183 views

213 views

### Is my reward function non-Markovian?

I am working on an RL problem where the time when the agent obtains the reward for taking action $a$ in time step $t$ is stochastic. In fact, there is no immediate reward for taking action $a$ in time ...
1 vote