1
$\begingroup$

I'm fairly new to reinforcement learning concepts, and I'm trying to implement a simple custom environment. In my custom environment, I have a scenario where I have multiple continuous state spaces, for example, length(l), and breadth(b), from which I calculate say, area(a) = l*b. I calculate the reward based on the area. Here I check if the area lies between the range I'm expecting it to and reward it accordingly.

Since my reward is based on the area which is f(l, b) = l*b, should I declare the observation space as the range of values my area can attain? Or should I declare my observation space as the values the length(l) and breadth(b) can attain? Or is my understanding wrong?

Area(a) = f(l, b) = l*b
Reward = +1 (if 15 < a < 20)
         -.1 (otherwise)
$\endgroup$
3
  • $\begingroup$ Don't put 2 questions in the title. One post should contain only one question. Decide what is your main question and put it in the title. The others can be follow-up questions, but ideally asked in a separate post. $\endgroup$
    – nbro
    Commented Feb 1, 2023 at 9:46
  • $\begingroup$ Hey! This was the first time I posted on such a forum and I wasn't aware of this. Will keep this in mind. Thank you! :) $\endgroup$
    – AlphaBit95
    Commented Feb 1, 2023 at 15:44
  • $\begingroup$ Please, just do as I suggested for this post too, not just for the future. $\endgroup$
    – nbro
    Commented Feb 1, 2023 at 20:11

1 Answer 1

2
$\begingroup$

The observation space and the state space are not the same in general. There exist problems where the state space cannot be fully observed, which goes by the name Partially Observable Reinforcement Learning (or some variation of it). This is relevant in environments with imperfect information. To me, it does not look like you are dealing with partial observation here, therefore state and observation space are identical.

Let's say e.g. for a poker game the state and the observation space are different, because the state space is the space of all the cards and money of all players (+ the order of play) and the community cards, while the observation space from one agent is only his own cards + community cards and the money. The observation space does not include the cards of the opposite players.

$\endgroup$
3
  • $\begingroup$ Hey! Thanks for the clarification! It helped me understand better. I was mainly concerned with this as I'm trying to train a DDPG model for such a scenario. I was really confused if my input states to the Actor-network would be the state space or the observation space(as most of the papers and blogs use the observation space). Since they are the same, would you say that my input to the network would be my state space itself? $\endgroup$
    – AlphaBit95
    Commented Feb 1, 2023 at 15:42
  • $\begingroup$ The input to your network should be the states at a given time, not the whole state space. So in your case the inputs should be the concrete values of l and b as far as I can read from your question. $\endgroup$ Commented Feb 1, 2023 at 16:30
  • $\begingroup$ Noted. Thank you very much! $\endgroup$
    – AlphaBit95
    Commented Feb 2, 2023 at 9:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .