Questions tagged [value-function]

For questions related to the concept of value (or performance, or quality, or utility) function (as defined in reinforcement learning and other AI sub-fields). An example of this type of functions is the Q function (used e.g. in the Q-learning algorithm).

Filter by
Sorted by
Tagged with
1
vote
1answer
17 views

Value-based methods for stochastic policies

Is it possible for value-based methods to learn stochastic policies? I'm trying to get a clear picture of the different categories for RL algorithms, and while doing so I started to think about ...
2
votes
1answer
34 views

Q Learning for FrozenLake environment not converging to V* values from Value Iteration

I am trying to learn tabular Q learning, value iteration using the classical algorithms (no neural networks) by using a table of states and actions. I was trying it out on FrozenLake environment in ...
1
vote
3answers
57 views

How to stop DQN Q function from increasing during learning?

Following the DQN algorithm with experience replay: We calculate the $loss=(Q(s,a)-(r+Q(s+1,a)))^2$. Assume I have positive but changing rewards. Meaning, $r>0$. Thus, since the rewards are ...
4
votes
3answers
109 views

How can a collaboration game be defined mathematically?

One of the common conceptions in AI is the idea of game theory. We see that in the predominance of chess and other games in the literature as metrics of AI success. We see it in the names of machine ...
2
votes
1answer
45 views

Is there any grid world dataset or generator for reinforcement learning?

I would like to start programming a multi task reinforcement learning model. For this, I need not just one maze or grid world (or just model-based), but many with different reward functions. So, I am ...
2
votes
1answer
49 views

Why is the state value function sufficient to determine the policy if a model is available?

In section "5.2 Monte Carlo Estimation of Action Values" of the second edition of the reinforcement learning book by Sutton and Barto, this is stated: If a model is not available, then it is ...
1
vote
1answer
142 views

What are the value functions used in reinforcement learning?

In reinforcement learning, we often define two functions, the state-value function $$V^\pi(s) = \mathbb{E}_{\pi} \left[\sum_{k=0}^{\infty} \gamma^{k}R_{t+k+1} \Bigg| S_t=s \right]$$ and the state-...
3
votes
1answer
157 views

An example of a unique value function which is associated with multiple optimal policies

In the 4th paragraph of http://www.incompleteideas.net/book/ebook/node37.html it is mentioned: Whereas the optimal value functions for states and state-action pairs are unique for a given MDP, ...