I have built a wildfire 'simulation' in unity. And I want to train an RL agent to 'control' this fire. However, I think my task is quite complicated, and I can't work out to get the agent to do what I want.
A fire spreads in a tree-like format, where each node represents a point burning in the fire. When a node has burned for enough time, it spreads in all possible cardinal directions (as long as it does not spread to where it came from). The fire has a list of 'perimeter nodes' which represent the burning perimeter of the fire. These are the leaf nodes in the tree. The rate of spread is calculated using a mathematical model (Rothermel model) that takes into account wind speed, slope, and parameters relating to the type of fuel burning.
I want to train the agent to place 'control lines' in the map, which completely stops the fire from burning. The agent will ideally work out where the fire is heading and place these control lines ahead of the fire such that it runs into these lines. Please could you guide me (or refer me to any reading that would be useful) on how I can decide the rules by which I give the model rewards?
Currently, I give positive rewards for the following:
- the number of fire nodes contained by a control line increases.
And I give negative rewards for:
- the number of fire nodes contained by a control line does not increase.
- the agent places a control line (these resources are valuable and can only be used sparingly).
I end the session with a win when all nodes are contained, and with a loss if the agent places a control line out of the bounds of the world.
I am currently giving the agent the following information as observations:
- the direction that the wind is heading, as a bearing.
- the wind speed
- the vector position that the fire is started at
- the current percentage of nodes that are contained
- the total number of perimeter nodes
I am new to RL, so I don't really know what is the best way to choose these parameters to train on. Please could you guide me to how I can better solve this problem?