What is the difference between the prediction (value estimation) and control problems in reinforcement learning?
Are there scenarios in RL where the problem cannot be distinctly categorised into the aforementioned problems and is a mixture of the problems?
Examples where the problem cannot be easily categorised into one of the aforementioned problems would be nice.
Nbro's answer already addresses the basic definitions, so I won't repeat that. Instead I'll try to elaborate a bit on the other parts of the question.
Are there scenarios in RL where the problem cannot be distinctly categorised into the aforementioned problems and is a mixture of the problems?
I'm not sure about cases where the "problem" can't be distinctly categories... but often, when we're actually interested in control as a problem, we still also actually deal with the prediction problem as a part of our training algorithm. Think of $Q$-learning, Sarsa, and all kinds of other algorithms related to the idea of "Generalized Policy Iteration". Many of them work (roughly) like this:
You could view these techniques in this way, as handling both of the problems at the same time, but there's also something to be said for the argument that they're really mostly just tackling the prediction problem. That's where all the "interesting" learning happens. The solution to the control problem is directly derived from the solution to the prediction problem in a single, small step. There are different algorithms, such as Policy Gradient methods, that directly aim to address the control problem instead.
An interesting (in my opinion :)) tangent is that in some problems, one of these problems may be significantly easier than the other, and this can be important to inform your selection of algorithm. For example, suppose you have a very long "road" where you can only move to the left or the right, you start on the left, and the goal is all the way to the right. In this problem, a solution to the control problem is trivial to express; just always go right. For the prediction problem, you need something much more powerful to be able to express all the predictions of values in all possible states.
In other problems, it may be much easier to quickly get an estimate of the value, but much more complicated to actually express how to obtain that value. For example, in StarCraft, if you have a much larger army, it is easy to predict that you will win. But you will still need to execute some very specific, long sequences of actions to achieve that goal.
Prediction is the problem of predicting any feature of the environment. In reinforcement learning, the typical feature is the reward or return, but this doesn't have to be always the case. See Multi-timescale nexting in a reinforcement learning robot (2011) by Joseph Modayil et al.
Control is the problem of estimating a policy. Clearly, the term control is related to control theory. In fact, the term control is often used as a synonym for action. See Is there any difference between a control and an action in reinforcement learning?. Similarly, the term controller is also used as a synonym for agent. For example, in the paper Metacontrol for Adaptive Imagination-Based Optimization (2017) by Jessica B. Hamrick et al. the term meta-controller is used to refer to an agent. A controlled system can also refer to the environment.
Section 14.1 of the book Reinforcement learning: an introduction (2nd edition) provides more details about the distinction between prediction and control and how this distinction is related to psychological concepts.
