Questions tagged [reward-functions]

For questions about rewards functions (e.g. in the context of reinforcement learning, which may be denoted as $R(s, a)$).

88 questions
Filter by
Sorted by
Tagged with
3 views

name of Reward function that utilizes the rewards of the next n steps

I have a problem with continuous time, observation and action space. I am discretizing the time to be able to apply the usual Reinforcement Learning algorithms (I chose PPO). The problem consists of a ...
52 views

Why is R(s) more restrictive than R(s, a) in an MDP?

I am quite new to RL. I would like to know why a state-dependent reward function R(s) is more restrictive than a state-action-dependent reward function R(s, a)? And why is it that a policy can be ...
351 views

How can I implement the reward function for an 8-DOF robot arm with TRPO?

I need to get an 8-DOF (degrees of freedom) robot arm to move a specified point. I need to implement the TRPO RL code using OpenAI gym. I already have the gazebo environment. But I am unsure of how to ...
1 vote
597 views

How to scale all positive continuous reward?

My RL project has all positive continuous rewards for every step and the goal is to have the maximum cumulative reward (episodic reward). The problem is that the rewards are too close and all between ...
185 views

How define a reward function for a humanoid agent whose goal is to stand up from the ground?

I'm trying to teach a humanoid agent how to stand up after falling. The episode starts with the agent lying on the floor with its back touching the ground, and its goal is to stand up in the shortest ...
46 views

How to handle penalty and reward occurring simultaneously

Assume the following scenario: We have an agent that acts on an environment where the agent should never take an action that results in him leaving the environment. For example, imagine an agent ...
3k views

Why does a negative reward for every step really encourage the agent to reach the goal as quickly as possible?

If we shift the rewards by any constant (which is a type of reward shaping), the optimal state-action value function (and so optimal policy) does not change. The proof of this fact can be found here. ...
20 views

If the agent is at the same state but at different times and receives a different reward, wouldn't this be violating somehow the MDP assumption?

I've been trying to train an agent, I've received and read suggestions to improve its speed to reach the goal. The suggestion is to use a time penalty, for example, adding $-0.1$ to the reward each ...
34 views

Could you explain the derivation of the expectation equation of equation 3.6 in Sutton & Barto? [duplicate]

I don't understand the last equality. Here is my derivation $r(s,a,s')=\sum_{r\in R} r p(r|s,a,s')=\sum_{r\in R} r \frac{p(s,a,s'|r)p(r)}{p(s,a,s')}$ Could you give me the correct steps to derive them?...
81 views

Can a reward function have various cases?

I'm doing a Q-learning algorithm and I'm designing my reward function. Basically I'm working on optimizing a network while changing some parameters. My metric to measure its optimization is the delay ...
2k views

Why does shifting all the rewards have a different impact on the performance of the agent?

I am new to reinforcement learning. For my application, I have found out that if my reward function contains some negative and positive values, my model does not give the optimal solution, but the ...
1 vote
28 views

What does "shuffle the comparisons into one dataset" mean?

I couldn't understand the wording here. What does "shuffle the comparisons into one dataset" mean? How does the method they use don't have $K \choose 2$ forward passes for K completions? Do ...
116 views

Will my Q values keep going up forever?

In Q-learning,the q values can be updated by the bellman equation. What happens with my Q values is that they keep going up forever, in accordance with the more I train. After 10,000 training episodes,...
1 vote
57 views

How to setup a reinforcement learning model that changes the values of $x$ to maximize $y$ in $y = f(x)$?

Assuming a relation such that $y = f(x)$, where $y$ represents a scalar and $x \in 20 \times 1$ vector consisting of zeros and ones, I want to set up a reinforcement learning model that changes the ...
70 views

In RL, is the quantification of the reward function arbitrary? Does it affect the learning?

There are different ways to set the reward function, such as extrinsic (externally provided rewards), intrinsic (the rewards are generated by the agents themselves based on their internal state and ...
1 vote
172 views

Is there a reward function that would encourage exploration in this case?

I am new to Reinforcement Learning. I am trying to train PPO agent for citylearn. The goal is to lower two environmental variables from observations. The default reward function is ...
1 vote
184 views

Is it necessary to have a constant reward in the terminal state?

I have downloaded the grid world project form this link. I have executed the project multiple times using: python gridworld.py -k 20 -a q -r -0.2 -s 90 I have ...
1 vote
2k views

What is the relationship between the reward function and the value function?

To clarify it in my head, the value function calculates how 'good' it is to be in a certain state by summing all future (discounted) rewards, while the reward function is what the value function uses ...
1 vote
42 views

Would the optimal policy remain same, if I replace R with V*?

In the context of RL, say I'm performing Value Iteration on a reward function R1. And the converged optimal policy is P1 and values are V1. Then, let's say I set rewards to be R2=V1 and perform value ...
1 vote
186 views

1 vote
83 views

Are there any deep RL algorithms that work well on finite MDPs and non-trivial terminal rewards?

I notice that most Deep Reinforcement Learning (DRL) works focus on Markov Decision Process (MDP) with an infinite time horizon. Are there any algorithms that work well on finite MDP and non-trivial ...
1 vote
145 views

Given the daily stock prices of the last 3 years, how should I sample the training data for episodic RL?

I am playing around with a stock trading agent trained via (deep) reinforcement learning, including memory replay. The agent is trained for 1000 episodes, where each episode consists of 180 timesteps (...
1 vote
63 views

How to teach Machine Learning Agent to destroy replicating objects in a puzzle game?

I have an unusual but very interesting problem. I have a game that is very similar to Toon Blast (a puzzle mobile game). It's based on a Match-2 mechanic in which you can destroy 2 or more connected ...
71 views

What research has been done on learning non-Markovian reward functions?

Recently, some work has been done planning and learning in Non-Markovian Decision Processes, that is, decision-making with temporally extended rewards. In these settings, a particular reward is ...