(This does not fully answer your question. Actually, it just attempts to make sure you understood certain things).

The distribution that you learn should put more weight/mass/density on the best actions. So, if you sample during inference, you should get the best actions more frequently than other actions.

You would use the same action every time if you are certain that the optional policy is deterministic, which is the case of finite MDPs, so not the case of continuous actions space MDPs.

There are cases where the best policy is really stochastic (e.g. rock-paper-scissors, which is naturally represented as Markov Game, so not a finite MDP), so choosing always the same action is likely not a good idea, so the best approach during inference would actually be to sample.

So, you would need to pick the same action every time if you have a finite MDP (finite action and state spaces).

See also this answer

(This does not fully answer your question. Actually, it just attempts to make sure you understood certain things).

The distribution that you learn should put more weight/mass/density on the best actions. So, if you sample during inference, you should get the best actions more frequently than other actions.

You would use the same action every time if you are certain that the optional policy is deterministic, which is the case of finite MDPs.

There are cases where the best policy is really stochastic (e.g. rock-paper-scissors, which is naturally represented as Markov Game, so not a finite MDP), so choosing always the same action is likely not a good idea, so the best approach during inference would actually be to sample.

So, you would need to pick the same action every time if you have a finite MDP (finite action and state spaces).

See also this answer

(This does not fully answer your question. Actually, it just attempts to make sure you understood certain things).

The distribution that you learn should put more weight/mass/density on the best actions. So, if you sample during inference, you should get the best actions more frequently than other actions.

You would use the same action every time if you are certain that the optional policy is deterministic, which is the case of finite MDPs, so not the case of continuous actions space MDPs.

There are cases where the best policy is really stochastic (e.g. rock-paper-scissors, which is naturally represented as Markov Game, so not a finite MDP), so choosing always the same action is likely not a good idea, so the best approach during inference would actually be to sample.

So, you would need to pick the same action every time only if you have a finite MDP (finite action and state spaces).

See also this answer

(This does not fully answer your question. Actually, it just attempts to make sure you understood certain things).

The distribution that you learn should put more weight/mass/density on the best actions. So, if you sample during inference, you should get the best actions more frequently than other actions.

You would use the same action every time if you are certain that the optional policy is deterministic, which is the case of finite MDPs, so not the case of continuous actions space MDPs.

There are cases where the best policy is really stochastic (e.g. rock-paper-scissors, which is naturally represented as Markov Game, so not a finite MDP), so choosing always the same action is likely not a good idea, so the best approach during inference would actually be to sample.

So, you would need to pick the same action every time if you have a finite MDP (finite action and state spaces).

See also this answer