When you are training a system using stochastic gradient descent, your system will converge towards some local minimum. If the local minimum was a good one, we would be fine with it. However, we cannot know how good a found solution is in comparison to other solutions of which we do not know their quality because they have been insufficiently explored. So, continuing to explore is a good way to escape comparatively bad local minima even if training has progressed already for quite a bit.
Besides that, maybe even more importantly towards the end of training, one also wants the system to perform well, i.e. robustly, in the presence of noise and not just under ideal circumstances. So, introducing some randomness, i.e. noise, into the network's policy can also lead to more robust policies being learned since the agent gets trained on how to best recover failure/unforeseen transitions into unexpected states.