I am trying to understand some of the different approaches used to overcome sparse rewards in a reinforcement learning setting for a research project. Particularly, I have looked at curiosity driven learning, where an agent learns an intrinsic reward function based on the uncertainty of the next state that the agent will end up in as he takes action a in state s. The greater the uncertainty of the next state, the higher the rewards. This will incentive agent's to be more exploratory and it is used particularly in some games where a huge number of steps is needed before the agent reaches the terminal state where is he only then rewarded.
The curiosity driven approach as demonstrated in this paper: https://pathak22.github.io/noreward-rl/ is able to learn faster than if a 0 rewards were used for each state, action.
To my knowledge, using different reward functions will affect the optimal policy obtained. Would curiosity driven learning therefore lead to a different policy as compared to whether a 0 reward was used ? Assume that for a 0 immediate reward system, it is able to derive a policy that reaches the goal state. Which of these 2 policies will be more optimal ?