RL is a learning algorithm, just like evolutionary strategies or SGD. ANNs can be used as the value function in RL, which is learned. The classic example is deep RL with AlphaGo. However, there are many RL methods which don’t use NNs, like Q learning as you mentioned.

A clarification by Sutton and Barto which may clear up the area of confusion: RL is both a general field of models/techniques as well as a learning algorithm. It depends on the context, but it seems like what you are referring to is RL as a method.