1
$\begingroup$

I am new to Reinforcement Learning and I am trying to self learn it. I have already posted some quesiton here and your answershave been really useful to me, so here I am posting another one.

I am studying the value iteration, and while doing the simulation using python, I get that at some states it is associated a value of $0$. I think I have to mention that I have tried to assign to the ststes an initial value different from zero, in order to simulate the fact that the agent already have some information about the enviroment before starting.

So, my questio is:

Is it possible to have values of the states equal to $0$ at the end of the value iteration?

$\endgroup$
2
$\begingroup$

Is it possible to have values of the states equal to 0 at the end of the value iteration?

Yes.

For a start, all terminal states should have a value of zero. This is not usually learned or calculated, but is by definition because the value represents the sum of expected future rewards and a terminal state should not have any. However, if the terminal states are implemented as "absorbing" states which always return 0 reward and do not transition away, then they can be learned as having a value of zero by e.g. value iteration. Caveat: This only works for a discounted return with $\gamma \lt 1$.

In addition, it is entirely possible to have an expected return of zero from any state. This might be composed of multiple positive and negative rewards that happen to sum to zero in expectation. A simple deterministic path through the state space that ended in a terminal state without receiving any positive or negative reward along the way would also have a value of zero.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.