2
$\begingroup$

Let's assume that there is a sequence of pairs $(x_i, y_i), (x_{i+1}, y_{i+1}), \dots$ of observations and corresponding labels. Let's also assume that the $x$ is considered as independent variable and $y$ is considered as the variable that depends on $x$. So, in supervised learning, one wants to learn the function $y=f(x)$.

Can reinforcement learning be used to learn $f$ (possibly, even learning the symbolic form of $f(x)$)?

Just some sketches how can it be done: $x_i$ can be considered as the environment and each $x_i$ defines some set of possible "actions" - possible symbolic form of $f(x)$ or possible numerical values of parameters for $f(x)$ (if the symbolic form is fized). And concrete selected action/functional form $f(x, a)$ (a - set of parameters) can be assigned reward from the loss function: how close the observation $(x_i, y_i)$ is to the value that can be inferred from $f(x)$.

Are there ideas or works of RL along the framework that I provided in the previous passage?

$\endgroup$
2
$\begingroup$

Any supervised learning problem can be cast as an equivalent reinforcement learning one.

Suppose you have the training dataset $\mathcal{D} = \{ (x_i, y_i \}_{i=1}^N$, where $x_i$ is an observation and $y_i$ the corresponding label. Then let $x_i$ be a state and let $f(x_i) = \hat{y}_i$, where $f$ is your (current) model, be an action. So, the predicted label of observation $x_i$ corresponds to the action taken in state $x_i$. The reward received after having taken action $f(x_i)$ in state $x_i$ can then be defined as the loss $|f(x_i) - y_i|$ (or any other suitable loss).

The minimization of this loss is then equivalent to the maximization of the (expected) reward. Therefore, in theory, you could use trajectories of the form $$T=\{(x_1, f(x_1), |f(x_1) - y_1|), \dots, (x_N, f(x_N), |f(x_N) - y_N|)\}$$ to learn a value function $q$ (for example, with Q-learning) or a policy $\pi$, which then, given a new state $x_{\text{new}}$ (an observation) produces an action $f(x_{\text{new}})$ (the predicted label). However, note that the learned policy might not be able to generalise to observations not present in the training dataset.

See this question Is supervised learning a subset of reinforcement learning? for more details regarding the relationship between reinforcement learning and supervised learning. See also Reinforcement Learning and its Relationship to Supervised Learning (2004) by Andrew G. Barto and Thomas G. Dietterich, who give a good overview of supervised and reinforcement learning and their relationship. The paper Learning to predict by the methods of temporal differences (1988), by Richard Sutton, should also give you an overview of reinforcement learning from a supervised learning perspective. However, note that this does not mean that a reinforcement learning problem can be cast as an equivalent supervised learning one. See section 1.3.3 Converting Reinforcement Learning to Supervised Learning of the mentioned paper Reinforcement Learning and its Relationship to Supervised Learning for more details.

Reinforcement learning can thus be used for classification and regression tasks. See, for example, Reinforcement Learning for Visual Object Detection (2016) by Mathe et al.

$\endgroup$
  • 1
    $\begingroup$ I think worth adding that although this re-casting is possible, it is usually the case that applying RL to straightforward regression and classification tasks will be inefficient (and a waste of the OP's time) $\endgroup$ – Neil Slater Aug 28 at 9:22
  • $\begingroup$ @NeilSlater Yes, this is also written in the linked answer and article. I do not want to exclude a possible exception to the rule, so I left this detail out. $\endgroup$ – nbro Aug 28 at 9:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.