I am trying to define the number of states and possible actions for a reinforcement learning problem that I want to solve with Q-learning, but I am a bit confused, as I'm totally new to reinforcement learning.

The problem I'm trying to solve is to assign different groups with people in the sample group having sequential numbers. Let's say there are three in each group.

Group 1, Group2, Group3.

1:{"group: Group1, "number": 1},
2:{"group: Group2, "number": 2},
3:{"group: Group3, "number": 3},
4:{"group: Group2, "number": 4},
5:{"group: Group1, "number": 5},
6:{"group: Group3, "number": 6},
7:{"group: Group3, "number": 7},
8:{"group: Group2, "number": 8},
9:{"group: Group1, "number": 9},

An optimal output will be a case where the numbers are sequential with respect to the group. For example, all in group1 should have number 1, 2, 3, or 4,5,6 or 7, 8, 9. and not 1, 5, 9 as in the dictionary above.

In other words, group1, group2, group3 represent the group ids, which means I have 3 groups. The number represents seat numbers. All in each of the groups need to sit close to each other e.g seat numbers 1,2,3 or 4,5,6, or 7,8,9.

I am wondering if the possible state will be all possible combinations of the group and numbers, in which case it will be 1680 and the possible action is the number of numbers to swap to get the desired output which is 9.

Any useful information will be very much appreciated.

  • $\begingroup$ Hello. Welcome to AI SE. I don't fully understand your problem even with your example. Particularly, I don't understand this description "assign different groups with people in the sample group having sequential numbers.". What is a "sample group" here? It seems that you have 3 groups. How are these groups different from the "sample group"? What do these sequences of numbers 1, 2, 3 or 7, 8, 9 represent in your problem? Please, edit your post to clarify all these points. $\endgroup$
    – nbro
    Jul 14 at 19:27
  • $\begingroup$ @nbro thank you for your comment. I have added more information. Hope it is clearer now. The same group is Group1 or Group2, or Group3. The numbers can be 1,2,3 or 4,5,6 or 5,7,6 for any of the groups. $\endgroup$
    – SAB6
    Jul 15 at 19:14
  • $\begingroup$ Thank you for the edit. It is easier to understand your goal state here, but not easy to understand the rules of your environment. It seems trivial to sort by group id and then assign "seat numbers" sequentially, and that appears to be a perfect solution. As you are only asking about the state space, that may not be a problem, but I think you need to give a little bit more background so that readers understand why you want a state representation at all $\endgroup$ Jul 15 at 19:17
  • $\begingroup$ Also I think you need to be clearer on whether the goal is simply that a group must consist of sequential number assignments, or whether that sequence relates to the row (person?) id. E.g. if it is the former, then your starting state could be 123213321 and an acceptable end state might be 333111222 (there are 6 such end states, and yes I count 1280 states overall using this representation) $\endgroup$ Jul 15 at 19:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.