A relatively recent but interesting paper that discusses this topic in more detail is Reward is enough (Artificial Intelligence, 2021) by David Silver, Satinder Singh, Doina Precup, and Richard S. Sutton (so by some of the godfathers of RL, who are all at DeepMind).
Their reward-is-enough hypothesis (RIEH) (page 4) is
Hypothesis (Reward-is-Enough). Intelligence, and its associated abilities, can be understood as subserving the maximisation of reward by an agent acting in its environment.
This hypothesis is slightly different from the reward hypothesis (RH), which states that all goals can be represented by rewards and the achievement of those goals can be viewed or formulated as the maximization of rewards, because the RIEH also states that the abilities needed to achieve the main goal in the environment arise from the maximization of the reward, so the RIEH is a stronger hypothesis than the RH.
The authors give examples to explain the RIEH (emphasis mine).
Sophisticated abilities may arise from the maximisation of simple rewards in complex environments. For example, the minimisation of hunger in a squirrel’s natural environment demands a skilful ability to manipulate nuts that arises from the interplay between (among other factors) the squirrel’s musculoskeletal dynamics; objects such as leaves, branches, or soil that the squirrel or nut may be resting upon, connected to, or obstructed by; variations in the size and shape of nuts; environmental factors such as wind, rain, or snow; and changes due to ageing, disease or injury. Similarly, the pursuit of cleanliness in a kitchen robot demands a sophisticated ability to perceive utensils in an enormous range of states that includes clutter, occlusion, glare, encrustation, damage, and so on.
They also try to argue why language, perceptron, social intelligence and general intelligence could all arise from the maximization of a single reward signal (e.g. survival).
Moreover, they also say that similar sophisticated abilities associated with intelligence could arise from the maximization of different reward signals, i.e. the emergence of these abilities is robust to the choice of reward objective.
Additionally, they also talk about prior knowledge and learning, but, in my view, they should have emphasized/noted that, for example, perception, without the suitable sensors (inductive bias) cannot emerge: this is not a limitation of the RIEH, as it says nothing about how these abilities actually arise, or the nature of the agent needed for them to arise, or which specific reward signal should be maximized.
In the end, they also conjecture that RL is the main framework that could be used to find out whether these conjectures/speculations are true or not.
They do not go into philosophical arguments, such as the Chinese-Room argument or the problem consciousness: their argument to address these issues would probably be that any ability (even consciousness, if it's an ability) required to achieve the ultimate goal would arise in the process of maximization of the reward.