2022 Developer Survey is open! Take survey.

# Questions tagged [value-functions]

For questions related to the concept of value (or performance, or quality, or utility) function (as defined in reinforcement learning and other AI sub-fields). An example of this type of functions is the Q function (used e.g. in the Q-learning algorithm), also known as the state-action value function, given that $Q: S \times A \rightarrow \mathbb{R}$, where $S$ and $A$ are respectively the set of states and actions of the environment.

90 questions
Filter by
Sorted by
Tagged with
5 views

### PPO custom implementation: do metrics like value loss, actor loss and entropy move a certain way?

I'm wondering whether problems with a custom PPO implementation (problem couldn't be made into a gym environment) can be debugged considering how the losses change over time. In my current experiment, ...
36 views

### Can we also estimate $V_{\pi}$ with SARSA?

For SARSA, I know we can estimate the action value $Q(s,a)$, and the relationship between $V(s)$ and $Q(s,a)$ is $V_{\pi}(s) = \sum_{a \in \mathcal{A}} \pi(a|s)Q_{\pi} (s,a)$. So my question is, can ...
1 vote
37 views

1 vote
73 views

42 views

### What are the recurrences used for updating state value function in $TD$ and $TD(\lambda)$ learning?

There are two types of value functions in reinforcement learning: State value function $V^{\pi} (s)$, state-action value function $Q^{\pi}(s, a)$. State value function: This value tells us how good ...
• 3,029
21 views

### Is using same features weights in Linear Value function approximation for estimating SARSA Q is right choice? (weight overblown problem)

I'm trying to use SARSA with Linear Value Function Approximation. the current problem is that weights get bigger every epoc/cycle. Previously i have used similar Sarsa algorithm, which it weights ...
1 vote
181 views

### Is there any difference between an objective function and a value function?

I found the usage of both objective function and value function in the same context. Context #1: In the paper titled Generative Adversarial Nets by Ian J. Goodfellow et al. We simultaneously train G ...
• 3,029
358 views

### How can we find the value function by solving a system of linear equations?

I am following the book "Reinforcement Learning: An Introduction" by Richard Sutton and Andrew Barto, and they give an example of a problem for which the value function can be computed ...
• 143
1 vote
34 views

### Is it possible to have values of the states equal to $0$ at the end of the value iteration?

I am new to Reinforcement Learning and I am trying to self learn it. I have already posted some quesiton here and your answershave been really useful to me, so here I am posting another one. I am ...
• 57
35 views

### A2C value function outputs keep increasing

I was implementing the A2C algorithm with as close to baseline setup as possible, and this is the code I came up with. The problem is that even after multiple rechecks, the program isn't showing ...
• 51