Following Pytorch's actor critic, I understand that the critic is a function mapping from the state space to the reward space, meaning, the critic approximates the state-value funcion.
However, according to This paper (you don't need to read it, just a glance at the nice picture at page 2 is enough), the critic is a function mapping from the action space to the reward, meaning it approximates the action value funcion
I am confused.
When people say "actor critic" - what do they mean by "critic"?
Is the term "Critic" ambiguous in RL?