Why is reinforcement learning not the answer to AGI?

Question

I previously asked a question about How can an AI freely make decisions?. I got a great answer about how current algorithms lack agency.

The first thing I thought of was reinforcement learning, since the entire concept is oriented around an agent getting rewarded for performing a correct action in an environment. It seems to me that reinforcement learning is the path to AGI.

I'm also thinking: what if an agent was proactive instead of reactive? That would seem like a logical first step towards AGI. What if an agent could figure out what questions to ask based on their environment? For example, it experiences an apple falling from a tree and asks "What made the Apple fall?". But it's similar to us not knowing what questions to ask about say the universe.

Answer 1

Some AI researchers do think RL is a path to AGI, and your intuition about how an agent would need to be proactive in selecting actions to learn about is exactly the area these researchers are now focused on.

Much of the work in this area is focused on the idea of curiosity, and since 2014 this idea has gained a lot of traction in the research community.

So, maybe RL can lead to AGI. We don't know for sure yet.

However, many of the classic arguments against AGI aren't addressed by the RL approach. For instance, if like Searle, you think computers just don't have the right kind of hardware to do thinking, then running an RL algorithm on that hardware isn't going to yield AGI, just ever increasingly robust narrow AI. Ultimately Searle's arguments get into issues of metaphysics, so it isn't clear that there exists any argument that would convince someone like Searle that a particular computer-based technique is AGI-capable.

There are also other arguments. For example, the cognativist school of thought thinks that statistical learning approaches to AI, and in particular, the black-box approaches of statistically-driven RL, are unlikely to lead to general intelligence because they do not engage in the kind of systematic reasoning process that proponents of cognativism assume is necessary for general intelligence. Some more extreme proponents of this school might say that a logical planning algorithm like STRIPS is innately more intelligent than any approach based on deep learning, because it involves sound logical deduction rather than mere statistical calculation. In particular, STRIPS can correctly generalize to any new domain, as long as it is fed the correct sense data, while an RL approach will need to learn how to act there.

So, while there are definitely reasons to be optimistic about RL as a direction for achieving AGI, it's definitely not yet settled.