I have this question that I'm kinda stuck on.

It's a game scenario in which we set up an expectimax tree. In the game, you have 3 dice with sides 1-4 that you roll at the beginning. Then, depending on the roll, the player can choose one of the dice to reroll or not reroll anything. Points are assigned like so:

  • 10 points if there's 2 of a kind
  • 15 if there's 3 of a kind
  • 7 if there's a series like 1-2-3 or 2-3-4
  • Otherwise, or if the sum is higher than the rewards from above, the score = sum of the rolls

For additional context, this is an example expectimax tree I came up with, for the case that the player rolled a 1,2,4 and is considering rerolling or not: enter image description here

Now lets introduce a new agent -- a robot that's supposed to help the human player. we assume

  • the human player choses any action with uniform probability regardless of the initial roll
  • there's a robot that, given a configuration of dice and the human's desired action, actually implements the action with probability 1-p and overrides it with a "no reroll" order with probability p>0. It has no effect if the human's decision is already to not reroll.

For that scenario, I came up with this expectimax tree: enter image description here

Now for the part I'm actually stuck on -- lets define A, B, C, and D as the expected reward of performing actions "reroll die 1", "reroll die 2", "reroll die 3", and "no reroll." How do we find $R_H$, the expected reward for the human acting without the robot's help, and $R_{AH}$ the expected reward for if the robot helps? We can't use p in the expression, we only have access to A,B,C,D and we're supposed to write it in the form $X + Y_p$

*EDIT: I asked again and the question was worded weirdly. They said we should definitely use p. What was meant by not using p is X and Y themselves can't contain p. But Y will be multiplied by p in the final simplified form.

For $R_{H}$ I think the answer should be $\frac{(A + B + C + D}{4}$ because of uniform distribution over A-D.

I'm supposing that $R_{AH}$ would be $\frac{(A + B + C)(1-p) + D + Dp}{4}$? Because the robot doesn't override with probability $1-p$, but he can only override A-C and does with probability $p$.

I think something feels slightly wrong about my answer but I'm not sure what.

  • 1
    $\begingroup$ Are you sure you are not allowed to use p? The expected reward R_AH is a function of p. $\endgroup$
    – Cohensius
    Feb 24 at 7:29
  • $\begingroup$ Hi. Could you please put your main specific question in the title? "Expectimax World Problem — Dice Rolling Game" is not a question, but the title of an article. $\endgroup$
    – nbro
    Feb 24 at 13:34
  • 1
    $\begingroup$ @Cohensius I asked the instructors and they clarified that you can use p. What was meant by not using p is X and Y themselves can't contain p. But Y will be multiplied by p in the final simplified form. It was a slight wording confusion. I added my best guess at a solution. However, I'm not 100% sure, and would appreciate confirmation :) $\endgroup$
    – Manny
    Feb 24 at 17:06
  • $\begingroup$ @nbro done! sorry $\endgroup$
    – Manny
    Feb 24 at 17:06

If I understand,

  • With probability $p$ the robot select no re-roll (action $D$).
  • With probability $1-p$ the human uniformly select an action between $A,B,C,D$.

Thus the expected reward for if the robot helps is

$$R_{AH}= p\cdot D+(1-p)\frac{A+B+C+D}{4} = \frac{(A+B+C)(1-p)+D+3pD}{4}$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.