What is the best practice in order to learn the optimal weight vector $W^*$? By optimal I mean the weights that will produce the agent with the highest win-rate.
I have an agent that plays a imperfect information game and I want to find the optimal weights via Reinforcement learning. Each turn, for each move $a$, the agent calculates a heuristic value, $h(a)$ that is a linear function of $n$ features. That is to say, the heuristic value for move $a$ is
$$h(a) = w_1f_1(a)+ w_2f_2(a)+...+w_nf_n(a)$$ where $\forall i, w_i \in[0,1]$
The Heuristic agent plays a distribution over the moves that is corelated to the value of the moves (moves with higher value have higher probability to be played)
- This question might be very basic, I am new to RL.
- Currently, the agent uses $n=13$ features.
- I have access to daily data of $10^6$ games of agent vs human.
- I have a game engine that allows me to run agent-vs-agent games.
- The Heuristic agent is a bit weaker than average recreational humans (win rate of 49%).
- The MCTS agent is a bit stronger than average human recreational (win rate of 58%).
- I have no good reason to think that linear weights are optimal. Just thought it's an easier start.
- The Reward is observed only at the round's end. There is no good way to evaluate the reward before the round's end.