I have this question that I'm kinda stuck on.
It's a game scenario in which we set up an expectimax tree. In the game, you have 3 dice with sides 1-4 that you roll at the beginning. Then, depending on the roll, the player can choose one of the dice to reroll or not reroll anything. Points are assigned like so:
- 10 points if there's 2 of a kind
- 15 if there's 3 of a kind
- 7 if there's a series like 1-2-3 or 2-3-4
- Otherwise, or if the sum is higher than the rewards from above, the score = sum of the rolls
For additional context, this is an example expectimax tree I came up with, for the case that the player rolled a 1,2,4 and is considering rerolling or not:
Now lets introduce a new agent -- a robot that's supposed to help the human player. we assume
- the human player choses any action with uniform probability regardless of the initial roll
- there's a robot that, given a configuration of dice and the human's desired action, actually implements the action with probability 1-p and overrides it with a "no reroll" order with probability p>0. It has no effect if the human's decision is already to not reroll.
For that scenario, I came up with this expectimax tree:
Now for the part I'm actually stuck on -- lets define A, B, C, and D as the expected reward of performing actions "reroll die 1", "reroll die 2", "reroll die 3", and "no reroll." How do we find $R_H$, the expected reward for the human acting without the robot's help, and $R_{AH}$ the expected reward for if the robot helps? We can't use p in the expression, we only have access to A,B,C,D and we're supposed to write it in the form $X + Y_p$
*EDIT: I asked again and the question was worded weirdly. They said we should definitely use p. What was meant by not using p is X and Y themselves can't contain p. But Y will be multiplied by p in the final simplified form.
For $R_{H}$ I think the answer should be $\frac{(A + B + C + D}{4}$ because of uniform distribution over A-D.
I'm supposing that $R_{AH}$ would be $\frac{(A + B + C)(1-p) + D + Dp}{4}$? Because the robot doesn't override with probability $1-p$, but he can only override A-C and does with probability $p$.
I think something feels slightly wrong about my answer but I'm not sure what.