2022 Developer Survey is open! Take survey.

# Questions tagged [reward-functions]

For questions about rewards functions (e.g. in the context of reinforcement learning, which may be denoted as $R(s, a)$).

65 questions
Filter by
Sorted by
Tagged with
1 vote
0 answers
20 views

### If we have a working reward function, would adding another action have a significant effect on the agent performance if task remains the same?

If we have a working reward function, providing the desired behavior and optimal policy in a continuous action/state-space problem, would adding another action significantly affect the possible ...
• 11
2 votes
1 answer
67 views

### How to construct a reward function for a "wait and see" problem

I'm working on a problem that I think could probably be represented as a reinforcement learning task, but I'm uncertain about how to design the reward function. The core task is essentially a ...
1 vote
1 answer
52 views

### How should I write the reward function to teach the agent the rules of this card game?

I'm quite new to reinforcement learning. I've been training the model for the following problem but the mean reward is stuck. In a 5 by 5 board, each position can contain a card with a color (0-4) ...
• 111
1 vote
0 answers
41 views

### What is commonly done for standardization/normalization of the targets in Deep Q-Learning?

I have been searching a lot about standardization/normalization of rewards and targets for the DQN algorithm. For the rewards, I now use the gym wrapper, which only scales but not shifts the rewards ...
• 23
0 votes
1 answer
31 views

### Does $R_{s}=E[R_{t}|S_{t}=s]$ indicate the reward we might expect on getting on average moving from any other state to $s$?

I'm trying to understand correctly what each "variable" in RL is and I'm not sure about $R_{s}$ the reward function. I used to think that it's the reward we may expect on average after ...
• 489
1 vote
0 answers
29 views

### Normalisation of reward function

Problem Currently, I have some problems defining a reward function for my RL project and mainly with how to normalise the score such that the highest possible score for all instances of the ...
• 21
1 vote
0 answers
38 views

### How to teach Machine Learning Agent to destroy replicating objects in a puzzle game?

I have an unusual but very interesting problem. I have a game that is very similar to Toon Blast (a puzzle mobile game). It's based on a Match-2 mechanic in which you can destroy 2 or more connected ...
• 153
2 votes
1 answer
51 views

### Is there a mathematical formalism to deal with a missing reward signal?

Typically, a Reinforcement Learning learning problem is formalized as finding an optimal policy for a Markov Decision Process (MDP). In many real-life situations, however, an agent can only get ...
• 163
-1 votes
1 answer
106 views

### Is it a bad practice to use cumulative rewards in reinforcement learning

I am using a DDPG agent for doing prediction on the position on an asset in a stock trading-like environment. I am using the cumulative reward as the reward for each timestep. Since it is trained over ...
• 49
1 vote
1 answer
240 views

### How should I define the reward function for a stock trading-like game?

Problem setting Consider a game like trading a stock At each step, the agent can buy / sell a stock. Trade is a pair of ...
• 19
2 votes
1 answer
127 views

### How to encourage the reinforcement-learning agent to reach the goal as quickly as possible, and what's the effect of discount factor?

I am trying to use reinforcement learning to solve a task and compare its performance to humans. The task is to find a single target in a fixed number of locations. At each step, the agent will pick ...
• 173
1 vote
1 answer
75 views

### How do I compute the value function when the reward is only at the end in the context of actor-critic algorithms?

Consider the actor-critic reinforcement learning setting (actor and critic parameterized by a neural network). The reward is given only at the end of the episode (or when there is a timeout there is ...
• 121
1 vote
0 answers
42 views

### Are there any deep RL algorithms that work well on finite MDPs and non-trivial terminal rewards?

I notice that most Deep Reinforcement Learning (DRL) works focus on Markov Decision Process (MDP) with an infinite time horizon. Are there any algorithms that work well on finite MDP and non-trivial ...
1 vote
1 answer
91 views

### How are rewards calculated for episodic tasks like playing chess or tic-tac-toe?

I am new to Reinforcement Learning and trying to understand the concept of reaping rewards during episodic tasks. I think in games like tic-tac-toe, rewards will be in terms of a win or lose. But does ...
1 vote
1 answer
69 views

5 votes
1 answer
227 views

### How to improve the reward signal when the rewards are sparse?

In cases where the reward is delayed, this can negatively impact a models ability to do proper credit assignment. In the case of a sparse reward, are there ways in which this can be negated? In a ...
2 votes
1 answer
72 views

### Can the rewards be matrices when using DQN?

I have a basic question. I'm working towards developing a reward function for my DQN. I'd like to train an RL agent to edit pixels on an image. I understand that convolutions are ideal for working ...
• 323
2 votes
1 answer
88 views

### How can I go from $R(s)$ to $R(s,a)$ in this specific MDP?

I'm trying to implement a research paper, as explained in this other post, here the author of the paper assumed R as a function of both states and actions, while the code (and the MDP) I'm using to ...
2 votes
0 answers
37 views

1 vote
1 answer
315 views

### What is the relationship between the reward function and the value function?

To clarify it in my head, the value function calculates how 'good' it is to be in a certain state by summing all future (discounted) rewards, while the reward function is what the value function uses ...
• 573
2 votes
1 answer
146 views

### How can I implement the reward function for an 8-DOF robot arm with TRPO?

I need to get an 8-DOF (degrees of freedom) robot arm to move a specified point. I need to implement the TRPO RL code using OpenAI gym. I already have the gazebo environment. But I am unsure of how to ...
3 votes
3 answers
428 views

### Is the policy really invariant under affine transformations of the reward function?

In the context of a Markov decision process, this paper says it is well-known that the optimal policy is invariant to positive affine transformation of the reward function On the other hand, ...
• 171
12 votes
4 answers
1k views

### Counterexamples to the reward hypothesis

On Sutton and Barto's RL book, the reward hypothesis is stated as that all of what we mean by goals and purposes can be well thought of as the maximization of the expected value of the cumulative ...
• 221