How to Represent Boardless Board Game as Input to RL Model?

Question

I am currently doing my thesis project by creating an Imitation Learning (IL) agent that learns to play the board game Hive, which lacks a traditional 2D board. Pieces are placed relative to one another along their hexagonal edges, and my state space representation for the game currently is a set of Cartesian Co-ordinates for a piece in play, along with an enum representing the piece (i.e., 'white Beetle 1' etc.) - where the first piece in play starts at (0, 0). It is also worth noting that the game isn't merely 2D either - pieces may, in some cases, stack on top of one another.

How do I create an input, and output state for this model in PyTorch given there is no fixed board state in this game? I wish to input game states as an array, where each move is a new input - in essence like a timeseries.

In the paper 'AZ-Hive case study' (found here), the game is represented by a 24x24 tile grid, with pieces represented as integers, with integers being added to one another to represent a stack of pieces. Is there perhaps a quick fix here by transposing my Cartesian co-ordinates onto a Numpy array of 28x28 zeroes and the enumerations for each piece as their value?

Having asked chatGPT, it suggested an input dict of fixed length such as the following snippet as an example:

    {
      "player_turn": "black",
      "pieces": [
        {
          "type": "ant",
          "position": (0, 0),
          "connections": [(0, 1), (1, 0)]
        },
        {
          "type": "grasshopper",
          "position": (0, 1),
          "connections": [(0, 0), (0, 2), (1, 1)]
        },
        {
          "type": "beetle",
          "position": (0, 2),
          "connections": [(0, 1), (1, 2)]
        }
      ]
    }

I am, however, unsure of this input as it relies on far too many dimensions for me to get my head around as an input.

I wish to create a PyTorch model that can capture some of the intricacies and strategies of the game, whilst not having to rewrite my current game representation entirely, as my move validator etc. is extensive. Any feedback in input and output of this model is greatly appreciated!

Answer 1

well, in the first place, you can even merely output a graphical representation of the game, just like a picture taken of the game on the table, and thus using any pretrained VC model...

However, this would make your game training definitely slower, as the input size is bigger, so it requires usually more FLOPS (however, this is not true as the "hardness of learning the game")

In the second place, usually you have to exploit some structure of the game if you don't want to fall in the previous case.

I've never seen that game, so I can't really help on that, however, I would suggest you to give a look at Graph Neural Networks, because as soon as you solve the problem for the state representation, you will probably also have a problem on the action representation, which I see it being easily solved with the output structure of a GNN, so, personal opinion, I would take this as a great opportunity to learn them (they indeed manages graphs, that is what ChatGPT is suggesting you to do)

Answer 2

I'm familiar with Hive, and the 24x24 hexagonal board space should work fine. Most games I've played have used less than 12x12. The pieces can interact with each other diagonally. To conceptualize, using a cube grid with offset rows might be more accessible.

You'd want the first piece placed to occupy the center of the board and then expand outward.