After reading some literature on RL it seems that Stochastic approximation theory underlies all of it.

There's a lot of substantial and difficult theory in this area requiring measure theory leading to martingales and Stochastic approximations.

The standard RL texts at best mention the relevant theorem and then move on, what I'm curious about is if the field of RL is really Stochastic process theory in disguise? Basically I'm curious if RL is really a CS/ECE topic or really belongs in the math department instead?

How much actual theory can be developed without this background?

After reading some literature on reinforcement learning (RL), it seems that stochastic approximation theory underlies all of it.

There's a lot of substantial and difficult theory in this area requiring measure theory leading to martingales and stochastic approximations.

The standard RL texts at best mention the relevant theorem and then move on.

Is the field of RL is really stochastic approximation theory in disguise? Is RL just a less rigorous version of stochastic approximation theory?