Consider an environment, where an agent intends to move from cell "A" to cell "G", avoiding obstacles (cells marked with shading). The agent can move forward, rotate 90º to the left, or rotate 90º to the right, and can identify the type of cell in front of him. Here's a diagram.

enter image description here

Classify the characteristics of the environment, according to:

  • Accessible or non-accessible
  • Deterministic or non‐deterministic
  • Episodic or non‐episodic
  • Discrete or continuous
  • Static or dynamic
  • 1
    $\begingroup$ This was clearly a homework problem/question. Please, next time, do not just ask for a solution, but show what you have tried so far, explain your thoughts, what you know or not. $\endgroup$ – nbro Dec 20 '20 at 13:00

The environment is :

  • Deterministic : the future state of the environment (and the agent) is defined by the agent's action.

  • Partially Observable (partially accessible) : since his sensors don't see what's beyond the shaded barrier, the environment is partially observable (accessible).

  • Sequential (non-episodic): Since the agent's current action will constrain and affect the space of possibilities (options, choices) of future actions, current decisions are not independent of future ones.

  • Discrete : Environment is not continuous, that is, every action/step/case is finite (either turn left, right, forward or backward), you don't have other actions, which makes it a discrete task environment.

  • Static : the environment is not changing, it's static, only the agent is moving.

  • 1
    $\begingroup$ It's not true that the environment is non-episodic. The task could be episodic and, in principle, it should be fully observable (although they say that the agent can only see in front of it). We can't really say if it's episodic or not, without more info about the task (i.e. the rewards and what happens when the agent gets to G). Also, we can't say if it's "static" or "dynamic", whatever that means in this context (probably stationary vs non-stationary). $\endgroup$ – nbro Dec 20 '20 at 13:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.