Is there any difference between a control and an action in reinforcement learning?

Question

There are reinforcement learning papers (e.g. Metacontrol for Adaptive Imagination-Based Optimization) that use (apparently, interchangeably) the term control or action to refer to the effect of the agent on the environment at each time step.

Is there any difference between the terms control or action or are they (always) used interchangeably? If there is a difference, when is one term used as opposed to the other?

The term control likely comes from the field of optimal control theory, which is related to reinforcement learning.

Answer 1

There's no difference. As they too often do, ML researchers take concepts from other disciplines, conveniently forget to cite sources and change the terminology, leading to much confusion. RL is a textbook example (pun intended). Optimal control researchers have been studying very similar problems long before RL ones, and used standard symbols and terms ($x$ for states, $u$ for controls). Then RL researchers came and changed just about everything.

See the paper A Tour of Reinforcement Learning: The View from Continuous Control (2018), by Benjamin Recht, which discusses reinforcement learning from a control and optimization perspective.

See also this tweet https://twitter.com/beenwrekt/status/1134536093980864514?s=21 (by Benjamin Recht) regarding the presentation of Sham Kakade.