The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [rewards]

For questions related to the concept of "reward" (e.g. in reinforcement learning).

Filter by
Sorted by
Tagged with
2
votes
1answer
25 views

Doubt in Deep-Q learning with sparse rewards

I am working on a deep reinforcement learning problem, when I got stuck at the following questions. They are rather general and not specific to my specific problem. The solution uses a sparse reward ...
2
votes
1answer
51 views

Can someone please help me validate my MDP?

Problem Statement : I have a system with four states - S1 through S4 where S1 is the beginning state and S4 is the end/terminal state. The next state is always better than the previous state i.e if ...
0
votes
0answers
30 views

Finding optimal Value function and Policy for an MDP

I am solving an RL MDP problem which is model based. I have an MDP which has four possible states S1-S4 and four different actions A1-A4, with S4 being terminal state and S1 is the beginning state. ...
2
votes
1answer
42 views

Can I have different rewards for a single action based on which state it transitions to?

I am working on an MDP where there are four states and ten actions. I am supposed to derive the optimal policy to reach the desired state. At any state, a particular action can take you to any of the ...
2
votes
0answers
23 views

Developmental systems that try to explain or understand the reward value in the reinforcement learning?

Are there methods (possibly logical or (how they are called in the literature) relational) that allows for the developmental systems to understand or explain the value of the received reward during ...
2
votes
1answer
568 views

Should RL rewards diminish over time?

Should a reward be cumulative or diminish over time? For example, say an agent performed a good action at time $t$ and received a positive reward $R$. If reward is cumulative, $R$ is carried on ...
1
vote
0answers
45 views

Is it possible to use Reward Function of type R(s, a, s') if more than one action is applied?

I am applying a reinforcement learning agent (PPO2, stable baselines implementation) to a custom built environment using OpenAI Gym. One reward function (formualted as loss function, that is, all ...
1
vote
0answers
34 views

Reward problem in A2C with multiple simultaneous discrete actions

I've built an A2C model whose actor's network has two different kinds of discrete actions, so the critic would take state and action (note that critic takes 2 actions because in each timestep we will ...
2
votes
0answers
28 views

Will the RL agent implemented as a neural network fine-tune itself?

Normally, when you develop a neural network, train it for object recognition (on normal objects like bike, car, plane, dog, cloud, etc.), and it turns out to perform very well, you would like to fine-...
2
votes
1answer
23 views

Using heuristic dense rewards in a sparse problem

If I am training an agent to try and navigate a maze as fast as possible, a simple reward would be something like \begin{align} R(\text{terminal}) &= N - \text{time}\ \ , \ \ N \gg \text{...
0
votes
0answers
15 views

Problems while training a DQN Agent on DSTC dataset

I am trying to create a dialogue policy model on DSTC data. This model takes in a state of the conversation and outputs an act the machine must take. I am creating this model using reinforcement ...
0
votes
1answer
78 views

What is the reward system of reinforcement learning?

Can you describe this system in more detail? I understand that the environment sends a signal indicating whether or not the action taken by the agent was 'good' or not, but it seems too simple. ...
0
votes
0answers
32 views

How do I define the reward function in the case of self-driving raspberry pi car?

I am working on a self driving car powered by a raspberry pi. My first step is to use PPO to teach it to not run into walls. But I am having trouble getting it to work. I want to allow the car to ...
2
votes
0answers
10 views

How is GARB implemented in PGRD-DL to calculate gradients w.r.t. internal rewards?

In section 3 of this paper the author outlines how GARB was adapted to reduce the variance in updating parameters to an internal reward function estimator. I have read it a number of times and ...
0
votes
1answer
45 views

Encourage Deep Q to seek short-term reward

I understand that gamma is an important factor in determining the rewards for a deep Q agent, however during testing of my network I am noticing that the agent is outputting more actions to "do ...
1
vote
0answers
21 views

Deciding the rewards for different actions in Pong for a DQN agent

I am attempting to implement an agent that learns to play in the Pong environment, the environment was created in PyGame and I return the pixel data and score at each frame. I use a CNN to take a ...
4
votes
0answers
33 views

What research has been done on learning non-Markovian reward functions?

Recently, some work has been done planning and learning in Non-Markovian Decision Processes, that is, decision-making with temporally extended rewards. In these settings, a particular reward is ...
1
vote
1answer
37 views

Do we have to consider the feasability of an action when defining the reward function of a MDP?

Do we have to consider if (s is given) an action a can lead to s' when defining a reward function? To be more specific: Let's say I have a 1D Map like: |A|B|C|D| ...
2
votes
2answers
76 views

Can Reinforcement Learning solve problems, where certain elements in the environement are randomly located?

I want to solve a problem using Reinforcement Learning on a 20x20 board. An agent (a mouse) has to get the highest possible rewards as fast as possible by collecting cheese, which there are 10 in ...
2
votes
2answers
104 views

Reinforcement Learning with long term rewards and fixed states and actions

I have read a lot about RL algorithms, that update the action-value function at each step with the currently gained reward. The requirement here is, that the reward is obtained after each step. I ...
3
votes
1answer
50 views

Given specific rewards, how can I calculate the returns for each time step?

Let's use Excercise 3.8 from Sutton, Barto - Introduction to RL: Suppose $\gamma = 0.5$ and following sequence of rewards is received $R_1=-1$ , $R_2=2$ , $R_3=6$ , $R_4=3$ , $R_5=2$ , with $T=5$ ...
2
votes
1answer
42 views

How do I avoid an agent to tend to terminate in a negative state when time needs to be taken into account?

In an unknown environment, how do I avoid an agent to tend to terminate its trajectory in a negative state when time needs to be taken into account? Suppose the following example to make my question ...
4
votes
1answer
597 views

Suitable reward function for trading buy and sell orders

I am working to build an deep reinforcement learning agent which can place orders (i.e. limit buy and limit sell orders). The actions are ...
0
votes
3answers
147 views

For some reasons, a reward becomes a penalty if

I am working to build a reinforcement agent with DQN. The agent would be able to place buy and sell orders for a day trading purpose. I am facing a little problem with that project. The question is "...
3
votes
1answer
261 views

Why does the “reward to go” trick in policy gradient methods work?

In policy gradient method, there's a trick to reduce a variance of policy gradient. We use causality, and remove part of the sum over rewards so that only actions happened after the reward are taken ...
3
votes
1answer
60 views

Is my interpretation of the return correct?

Sutton and Barto 2018 define the discounted return $G_t$ the following way (p 55): Is my interpretation correct? Or should all "1" be in the same column?
3
votes
2answers
1k views

What should I do when the potential value of a state is too high?

I'm working on a Reinforcement Learning task where I use reward shaping as proposed in the paper Policy invariance under reward transformations: Theory and application to reward shaping (1999) by ...
12
votes
4answers
1k views

What would motivate a machine?

Currently, within the AI development field, the main focus seems to be on pattern recognition and machine learning. Learning is about adjusting internal variables based on a feedback loop. Maslow's ...