Questions tagged [value-functions]

For questions related to the concept of value (or performance, or quality, or utility) function (as defined in reinforcement learning and other AI sub-fields). An example of this type of functions is the Q function (used e.g. in the Q-learning algorithm), also known as the state-action value function, given that $Q: S \times A \rightarrow \mathbb{R}$, where $S$ and $A$ are respectively the set of states and actions of the environment.

5 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
62 views

In Soft Actor Critic, why is the action sampled from current policy instead of replay buffer on value function update?

While reading the original paper of Soft Actor Critic, I came across on page number 5, under equation (5) and (6) $$ J_{V}(\psi)=\mathbb{E}_{\mathbf{s}_{t} \sim \mathcal{D}}\left[\frac{1}{2}\left(V_{\...
2
votes
0answers
32 views

What's an example of a simple policy but a complex value function?

Hado van Hasselt, a researcher at DeepMind, mentioned in one of his videos (from 7:20 to 8:20) on Youtube (about policy gradient methods) that there are cases when the policy is very simple compared ...
1
vote
1answer
37 views

The are some fundamental learning theories for developing an AI that imitates human behavior

Most if not all AI systems are not to imitate human, but to finally out-perform human. Examples include using AI to play a game, classification problems, auto-driving, and goal-oriented chatbots. ...
0
votes
0answers
35 views

Is using Bellman Optimality Equation to evaluate states a bad idea when episode number is low?

I am trying to build an RL agent that interacts with an environment, a 2D grid of dimensions 20*10: each (i,j) square in the grid gives out some reward to the agent when it visits that square. Each ...
0
votes
0answers
44 views

Understanding V- and Q-functions

Assume the existence of a Markov Decision Process consisting of: State space $S$ Action space $A$ Transition model $T: S \times A \times S \to [0,1]$ Reward function $R: S \times A \times S \to \...