Questions tagged [reward-functions]
For questions about rewards functions (e.g. in the context of reinforcement learning, which may be denoted as $R(s, a)$).
26
questions with no upvoted or accepted answers
5
votes
1
answer
185
views
How define a reward function for a humanoid agent whose goal is to stand up from the ground?
I'm trying to teach a humanoid agent how to stand up after falling. The episode starts with the agent lying on the floor with its back touching the ground, and its goal is to stand up in the shortest ...
- 151
4
votes
0
answers
71
views
What research has been done on learning non-Markovian reward functions?
Recently, some work has been done planning and learning in Non-Markovian Decision Processes, that is, decision-making with temporally extended rewards. In these settings, a particular reward is ...
- 41
3
votes
0
answers
137
views
Is better to reward short- or long-term progress in Q-learning?
I have been training some kind of agent to reach a target using a Q-learning based approach, and I have tried two different types of rewards:
Long-term reward: $\mathrm{reward} = - \mathrm{distance}(\...
- 1,197
2
votes
0
answers
70
views
In RL, is the quantification of the reward function arbitrary? Does it affect the learning?
There are different ways to set the reward function, such as extrinsic (externally provided rewards), intrinsic (the rewards are generated by the agents themselves based on their internal state and ...
- 21
2
votes
0
answers
253
views
How to solve a reinforcement learning problem with changing rewards?
I'm working on a problem with non-stationary environments. The state space is discrete and limited. The action is limited too. But the reward for the same action $a$ can change. Even the reward for ...
2
votes
0
answers
33
views
How can I discourage the RL agent from drawing in a zero-sum game?
My agent receives $1, 0, -1$ rewards for winning, drawing, and losing the game, respectively. What would be the consequences of setting reward to $-1$ for draws? Would that encourage the agent to win ...
- 763
2
votes
0
answers
103
views
How to combine two differently equally important signals into the reward function, that have different scales?
I have two signals that I want to use to model my reward.
The first one is the CPU TIME: running mean from this diagram:
The second one is the MAX RESIDUAL from this diagram:
Since they are both ...
- 382
2
votes
1
answer
352
views
How can I implement the reward function for an 8-DOF robot arm with TRPO?
I need to get an 8-DOF (degrees of freedom) robot arm to move a specified point. I need to implement the TRPO RL code using OpenAI gym. I already have the gazebo environment. But I am unsure of how to ...
- 21
1
vote
0
answers
28
views
What does "shuffle the comparisons into one dataset" mean?
I couldn't understand the wording here.
What does "shuffle the comparisons into one dataset" mean?
How does the method they use don't have $K \choose 2$ forward passes for K completions? Do ...
1
vote
0
answers
42
views
Would the optimal policy remain same, if I replace R with V*?
In the context of RL, say I'm performing Value Iteration on a reward function R1. And the converged optimal policy is P1 and values are V1.
Then, let's say I set rewards to be R2=V1 and perform value ...
1
vote
0
answers
25
views
If we have a working reward function, would adding another action have a significant effect on the agent performance if task remains the same?
If we have a working reward function, providing the desired behavior and optimal policy in a continuous action/state-space problem, would adding another action significantly affect the possible ...
- 11
1
vote
0
answers
662
views
What is commonly done for standardization/normalization of the targets in Deep Q-Learning?
I have been searching a lot about standardization/normalization of rewards and targets for the DQN algorithm. For the rewards, I now use the gym wrapper, which only scales but not shifts the rewards ...
- 55
1
vote
0
answers
96
views
Normalisation of reward function
Problem
Currently, I have some problems defining a reward function for my RL project and mainly with how to normalise the score such that the highest possible score for all instances of the ...
- 31
1
vote
0
answers
63
views
How to teach Machine Learning Agent to destroy replicating objects in a puzzle game?
I have an unusual but very interesting problem. I have a game that is very similar to Toon Blast (a puzzle mobile game). It's based on a Match-2 mechanic in which you can destroy 2 or more connected ...
- 153
1
vote
0
answers
83
views
Are there any deep RL algorithms that work well on finite MDPs and non-trivial terminal rewards?
I notice that most Deep Reinforcement Learning (DRL) works focus on Markov Decision Process (MDP) with an infinite time horizon.
Are there any algorithms that work well on finite MDP and non-trivial ...
- 111
1
vote
1
answer
597
views
How to scale all positive continuous reward?
My RL project has all positive continuous rewards for every step and the goal is to have the maximum cumulative reward (episodic reward). The problem is that the rewards are too close and all between ...
1
vote
0
answers
145
views
Given the daily stock prices of the last 3 years, how should I sample the training data for episodic RL?
I am playing around with a stock trading agent trained via (deep) reinforcement learning, including memory replay. The agent is trained for 1000 episodes, where each episode consists of 180 timesteps (...
- 111
0
votes
0
answers
5
views
name of Reward function that utilizes the rewards of the next n steps
I have a problem with continuous time, observation and action space. I am discretizing the time to be able to apply the usual Reinforcement Learning algorithms (I chose PPO). The problem consists of a ...
0
votes
0
answers
46
views
How to handle penalty and reward occurring simultaneously
Assume the following scenario:
We have an agent that acts on an environment where the agent should never take an action that results in him leaving the environment. For example, imagine an agent ...
- 143
0
votes
0
answers
20
views
If the agent is at the same state but at different times and receives a different reward, wouldn't this be violating somehow the MDP assumption?
I've been trying to train an agent, I've received and read suggestions to improve its speed to reach the goal. The suggestion is to use a time penalty, for example, adding $-0.1$ to the reward each ...
0
votes
0
answers
34
views
Reward Function for Reinforcement Learning model
I am trying to create a reinforcement learning model to control the acceleration of a car. I am designing the model such that initially the acceleration is provided and then deceleration is provided ...
0
votes
0
answers
29
views
Bouding the state using the reward in RL
I'm wondering what the common approaches are bounding out state $s\in\mathbf{R}$ to some values $\in[s_0,s_1]$ is required. So in my case, for example, the state represents an angle of rotation, that ...
- 371
0
votes
0
answers
125
views
Shaping reward so that it maximizes multiple components together
I am fairly new to RL and I compete in AWS DeepRacer student league. The main task there is to create a reward function. All the hyperparameters and action space are fixed. So far, I know how to shape ...
0
votes
0
answers
213
views
How to solve a reinforcement learning problem with a stochastic reward function?
In a discrete time system, an environment has an unknown reward probability $p(r|s,a)$. However, the transition probability $p(s'\mid s,a)$ is deterministic.
In my case, the reward for the same action ...
0
votes
0
answers
213
views
Is my reward function non-Markovian?
I am working on an RL problem where the time when the agent obtains the reward for taking action $a$ in time step $t$ is stochastic. In fact, there is no immediate reward for taking action $a$ in time ...
0
votes
0
answers
53
views
Thompson sampling with Bernoulli prior and non-binary reward update
I am solving a problem for which I have to select the best possible servers (level 1) to hit for a given data. These servers (level 1) in turn hit some other servers (level 2) to complete the request. ...
