I'm now reading the following blog post but on the epsilon-greedy approach, the author implied that the epsilon-greedy approach takes the action randomly with the probability epsilon, and take the best action 100% of the time with probability 1 - epsilon.
So for example, suppose that the epsilon = 0.6 with 4 actions. In this case, the author seemed to say that each action is taken with the following probability (suppose that the first action has the best value):
- action 1: 55% (.40 + .60 / 4)
- action 2: 15%
- action 3: 15%
- action 4: 15%
However, I feel like I learned that the epsilon-greedy only takes the action randomly with the probability of epsilon, and otherwise it is up to the policy function that decides to take the action. And the policy function returns the probability distribution of actions, not the identifier of the action with the best value. So for example, suppose that the epsilon = 0.6 and each action has 50%, 10%, 25%, and 15%. In this case, the probability of taking each action should be the following:
- action 1: 35% (.40 * .50 + .60 / 4)
- action 2: 19% (.40 * .10 + .60 / 4)
- action 3: 25% (.40 * .25 + .60 / 4)
- action 4: 21% (.40 * .15 + .60 / 4)
Is my understanding not correct here? Does the non-random part of the epsilon (1 - epsilon) always takes the best action, or does it select the action according to the probability distribution?