During the learning phase, why don't we have a 100% exploration rate, to allow our agent to fully explore our environment and update the Q values, then during testing we bring in exploitation? Does that make more sense than decaying the exploration rate?
No - imagine if you were playing an Atari game and took completely random actions. Your games would not last very long and you would never get to experience all of the state space because the game would end too soon. This is why you need to combine exploration and exploitation to fully explore the state space.
While theoretically you can do something like this if you're very confident you'll cover most of the state space in exploration, this is still a suboptimal strategy. Even in the case of multi-armed bandits, this strategy can be much less sample efficient than $\epsilon$-greedy, and exploration is much easier in this case.
So, even if your strategy miraculously works on a decently sized MDP, it'll be worse than combining exploration and exploitation.