I am requesting research, articles, abstracts or interesting opinions that will help me create a complex causal neural network. There are many detailed resources on causal discovery, image recognition, spatial invariance. However, none of them take into account an animated changing game state, and all are very specific and hard to use for complex problems. Here is the core of the problem:

  I am making a 2D strategy simulation/game and would like to create a neural network that can learn and deduce all the game's rules. The game, similar to Conway's Game of Life, takes the form of a 2-dimensional image, with red green and blue values for each pixel/cell. The game goes in turns, where every turn, the pixels of the image are evaluated, and changed based on the rules dictated by their RGB values. The network will train on a large data set of animated images, representing the changes to the game's state, with each frame being a turn.


  1. if red is above 50%, all adjacent pixels have their blue values halved.
  2. if blue is below 75%, between 2-8 random pixels within a 10 tile radius will increase their green values by 25%
  3. if green value is divisible by 5, all adjacent pixels will turn white (100%red, 100% blue, 100% green)
  4. if pixel is black, 15% of the map's pixels will be turned white at random locations.
  5. if pixel is white, a random pixel on the image will have their RGB values randomized.
  6. if green value is above 80%, the first 3 pixels to the immediate north will all have their values switched with each other.

The rules are very arbitrary and feature a mix of conditional checks as well as random changes to the map. This is intended to challenge the network's spatial and causal recognition. There is also the intention of adding or removing rules in the future, with the network being flexible enough to learn the new rules or changes to old rules, if found.

The main constraint I placed is that there will be no feature engineering, and the game's rules will always be hidden from the network. The challenge is for this theoretical network to analyze short and long term changes to the game state, discover and deduce as much of the game's hidden rules as possible, in the time it's given to analyze the data. It will eventually need to design its own generated image, designed in order to maximize the lifetime of an arbitrary choice of color. For example, it may be tasked to keep as much of the image red as possible, or possibly keep the green values low. The idea is that giving it new design constraints should be intuitive and easy, without having to re-code or re-train the entire network every time. I would also like to avoid using pooling, sampling, or any other tricks for image-based neural networks to detect patterns. This is because some of the game's rules are arbitrarily random, and may not have direct spatial correlation, while other rules do.


  • 1
    $\begingroup$ great project, now what about how to implement it? where are the implementation details? how do you encode the input if you don't want images? $\endgroup$
    – Alberto
    Commented Sep 25, 2023 at 13:13
  • $\begingroup$ Welcome to AI Stack Exchange. In general, we don't do "requests for resources" - even if we did, I think your setup is so unique, especially with your self-imposed constraints, that there will be little to no papers, tutorials or code that could apply to the project in general. I suggest you ask a more specific question about your project setup or approach that could have an objective answer, and that is not "please find me some helpful resources". $\endgroup$ Commented Sep 25, 2023 at 16:50
  • $\begingroup$ thank you for the replies. i guess the difficulty lies in connecting all the resources i've found into a functional network. i'm going to do more personal research in the future so that i can narrow down my question into something simpler and more possible to answer. applying machine learning to the realm of video game design should be easy in theory, but in practice there's so many major differences between game states and linear statistical data that makes it hard to translate. $\endgroup$ Commented Sep 26, 2023 at 8:40
  • $\begingroup$ - how do you encode the input if you don't want images? that's actually what i'm trying to figure out. i will be using images, but it will be several frames of the image changing according to the rules. i'm lost on how to feed sequential turn-based information into my network in a way that makes it feasible to break it into quantifiable, linear chunks. $\endgroup$ Commented Sep 26, 2023 at 8:41


You must log in to answer this question.