Tag Info

Accepted

Why does the "reward to go" trick in policy gradient methods work?

An important thing we're going to need is what is called the "Expected Grad-Log-Prob Lemma here" (proof included on that page), which says that (for any $t$): \mathbb{E}_{\tau \sim \pi_{\...
• 9,379
Accepted

Why is the reward in reinforcement learning always a scalar?

If you have multiple types of rewards (say, R1 and R2), then it is no longer clear what would be the optimal way to act: it can happen that one way of acting would maximize R1 and another way would ...
• 231
Accepted

What is the difference between expected return and value function?

There is a strong relationship between a value function and a return. Namely that a value function calculates the expected return from being in a certain state, or taking a specific action in a ...
• 23.8k

Why is the reward in reinforcement learning always a scalar?

Rather than the survey by Liu et al. recommended above, I'd suggest you read the following survey paper for an overview of MORL (disclaimer - I was a co-author on this, but I genuinely think it is a ...

How do I handle negative rewards in policy gradients with the cross-entropy loss function?

It depends on your loss function, but you probably need to tweak it. If you are using an update rule like loss = -log(probabilities) * reward, then your loss is ...
• 161

Why is the reward in reinforcement learning always a scalar?

Markov decision problems are usually defined with a reward function $r:\mathcal{S}\times\mathcal{A}\rightarrow\mathbb{R}$, and in these cases the rewards are expected to be scalar real values. This ...
• 522

Why cannot an AI agent adjust the reward function directly?

Why do both approaches prevent the AI agent from changing its reward function at will? In RL for optimal control, the reward function is part of the problem formulation. That is, it describes the ...
• 23.8k
Accepted

Should RL rewards diminish over time?

RL agents - implemented correctly - do not take previous rewards into account when making decisions. For instance value functions only assess potential future reward. The state value or expected ...
• 23.8k

What would motivate a machine?

This is an interesting question actually. There's a quite realistic idea about "where can the curiosity originate from" in the book "On intelligence" written by Jeff Hawkins and Sandra Blakeslee. It'...
• 109
Accepted

What would motivate a machine?

The current method to implement motivation is some kind of artificial reward. Deepmind's DQN for example is driven by the score of the game. The higher the score, the better. The AI learns to adjust ...
• 1,644

Reinforcement Learning with long term rewards and fixed states and actions

You don't need to have a reward on every single timestep, reward at the end is enough. Reinforcement learning can deal with temporal credit assignment problem, all algorithms are designed to work with ...
• 2,226

In RL, if I assign the rewards for better positional play, the algorithm is learning nothing?

What you are proposing is closer to a heuristic for searching than a reward for RL. This is a blurred line, but generally if you start analysing the problem yourself, breaking it down into components ...
• 23.8k
Accepted

Why is regret so defined in MABs?

In short, you don't regret your bad luck that you could do nothing about, you regret your bad choices that you could have done something about if only you knew. The point of regret as a metric ...
• 23.8k
Accepted

• 23.8k