I don't quite understand why the wumpus world problem is deterministic.

I mean, if the agent starts at [1,1] and there is a pit at [1,2] then the agent cannot determine if there is a pit at [2,1] or [1,2]. He has to take a risk. Shouldn't the problem be non-deterministic?


1 Answer 1


When a reinforcement learning problem is described as deterministic, that means the environment is deterministic. In turn that means:

  • All rewards are single values (instead of distributions) and always occur in the same way for the same state/action pair.
  • All state transitions are fixed, performing the action $a$ in state $s$ leads to the same state $s'$ each time.

This determinism does not apply to the learning process or algorithm. Often these have some randomness to ensure exploration. It also says nothing about the agent's knowledge of the environment - a model-free agent starts with no knowledge of the environment and must discover the results.

The results of e.g. moving to [1,2] and falling into a pit are deterministic in that this will always happen in the same way on repeated attempts. You can take advantage of this when designing the agent - if it discovers a pit on one episode, it will know it is there from then on.

If you know an environment is deterministic, then you can take advantage of this knowledge. In simple environments you can set a high learning rate, because there are no probabilities to approximate.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .