Reinforce Learning: Do I have to ignore hyper parameter(?) after training done in Q-learning?

Question

Learner might be in training stage, where it update Q-table for bunch of epoch.

In this stage, Q-table would be updated with gamma(discount rate), learning rate(alpha), and action would be chosen by random action rate.

After some epoch, when reward is getting stable, let me call this "training is done". Then do I have to ignore these parameters(gamma, learning rate, etc) after that?

I mean, in training stage, I got an action from Q-table like this:

if rand_float < rar:
    action = rand.randint(0, num_actions - 1)
else:
    action = np.argmax(Q[s_prime_as_index])

But after training stage, Do I have to remove rar, which means I have to get an action from Q-table like this?

action = np.argmax(self.Q[s_prime])

Answer 1

Epsilon (your rar parameter) is used to have the possibility of a random action which handles the problem of exploration and exploitation.

If you have a deterministic world and are certain that your learner has learned a policy with performance to your liking, removing it will ensure that only the greedy policy being learned will be taken and that you can expect the same performance afterwards.

The agent will still be learning at this point because alpha (learning rate) is not set to 0. Because of this, if the world is stochastic instead and possibly changes slowly over time (e.g. motor speed due to battery power), then having epsilon non zero still allows the agent to explore and overcome these changes when they happen before it's performance gets low enough to change is behaviour (e.g. q values dropping).

If instead you simply want to test the performance of the policy learned up until a timestep, setting alpha to 0 in a compares the epsilon-greedy policy, setting both alpha and epsilon to 0 compares the greedy policy.