The 2024 Developer Survey results are live! See the results

# Tag Info

Accepted

### How should I handle invalid actions (when using REINFORCE)?

Just ignore the invalid moves. For exploration, it is likely that you won't just execute the move with the highest probability, but instead choose moves randomly based on the outputted probability. If ...
• 4,295

### How should I handle invalid actions (when using REINFORCE)?

Usually softmax methods in policy gradient methods using linear function approximation use the following formula to calculate the probability of choosing action $a$. Here, weights are $\theta$, and ...
• 3,837
Accepted

### Can reinforcement learning be used for tasks where only one final reward is received?

RL can be used for cases where you have sparse rewards (i.e. at almost every step all rewards are zero), but, in such a setting, the experience the agent receives during the trajectory does not ...
• 41k
Accepted

### What are some best practices when trying to design a reward function?

Designing reward functions Designing a reward function is sometimes straightforward, if you have knowledge of the problem. For example, consider the game of chess. You know that you have three ...
• 41k

• 10.4k
Accepted

### Reinforcement Learning algorithm with rewards dependent both on previous action and current action

The answer to both your concerns is: Add the previous action choice to the state representation. It is all you need to do. It gives the agent the data it needs to learn the association of negative ...
• 32.8k
Accepted

### Can the rewards be matrices when using DQN?

Generally speaking, is it better for rewards to be a scalar, or is using matrices okay? Rewards need to be scalar, real values to match to standard theory of Markov decision processes (MDPs) and ...
• 32.8k
Accepted

### In RL, is it possible to design a multiplicative/exponential reward function? A reward func that depends on current accumulated reward?

The main thing you will need to do is add the accumulated reward (total_score_so_far) to the state. In order to predict future reward with any accuracy, the agent ...
• 32.8k
Accepted

### Reward design or Inverse reinforcement learning?

It depends on the domain you are in. Inverse RL (IRL) would be most advantageous in domains in which: It's hard to specify the reward by hand: for example, it would be hard to hand-specify a reward ...
• 380

### How should I handle invalid actions (when using REINFORCE)?

An experimental paper exist in arxiv about the effect of whether to mask or to give negative rewards to invalid actions. There are some references in this paper which also discuss the effects and the ...
• 165
Accepted

### Expressing Arbitrary Reward Functions as Potential-Based Advice (PBA)

Is the method itself defective or anything wrong with my code? There does indeed appear to be an issue with the code, the publications are fine (I know most of those authors and would very much trust ...
• 10.4k
Accepted

### How should I define the reward function in the case of Connect Four?

The classic working reward scheme for two player zero sum games (i.e. if I win, you lose and vice versa) is simply: +1 for a win 0 for a draw -1 for a loss These rewards should be associated with ...
• 32.8k