Supervised learning
The supervised learning (SL) problem is formulated as follows.
You are given a dataset $\mathcal{D} = \{(x_i, y_i)_{i=1}^N$, which is assumed to be drawn i.i.d. from an unknown joint probability distribution $p(x, y)$, where $x_i$ represents the $i$th input and $y_i$ is the corresponding label. You choose a loss function $\mathcal{L}: V \times U \rightarrow \mathbb{R}$. Then your goal is to minimize the so-called empirical risk
$$R_{\mathcal{D}}[f]=\frac{1}{N} \sum_{i=1}^N \mathcal{L}(x_i, f(x_i)) \tag{0}\label{0}$$
with respect to $f$. In other words, you want to find the $f$ that minimizes the average above, which can also be formally written as
$$
f^* = \operatorname{argmin}_f R[f] \tag{1}\label{1}
$$
The problem \ref{1} is called the empirical risk minimization because it is a proxy problem for the expected risk minimization (but you can ignore this for now).
Reinforcement learning
In reinforcement learning, you typically imagine that there's an agent that interacts, in time steps, with an environment by taking actions. At each time step $t$, the agent takes $a_t$ in the state $s_t$, receives a reward $r_t$ from the environment and the agent and the environment move to another state $s_{t+1}$.
The goal of the agent is to maximize the expected return
$$\mathbb{E}\left[ G_t \right] = \mathbb{E}\left[ \sum_{i=t+1}^\infty R_i \right]$$
where $t$ is the current time step (so we don't care about the past), $R_i$ is a random variable that represents the probable reward at time step $i$, and $G_t = \sum_{i=t+1}^\infty R_i $ is the so-called return (i.e. a sum of future rewards, in this case, starting from time step $t$), which is also a random variable.
In this context, the most important job of the programmer is to define a function $\mathcal{R}(s, a)$, the reward function, which provides the reinforcement (or reward) signal to the RL agent. $\mathcal{R}(s, a)$ will deterministically or stochastically determine the reward that the agent receives every time it takes action $a$ in the state $s$. (Note that $\mathcal{R}$ is different from $R_i$, which is a random variable that represents the reward at time step $i$).
What is the difference between SL and RL?
In RL, you (the programmer) need to define the reward function $\mathcal{R}$ and you want to maximize the expected return. On the other hand, in SL you are given (or you collect) a dataset $\mathcal{D}$, you choose $\mathcal{L}$ in \ref{0}, and the goal is to find the function $f^*$ that minimizes the empirical risk. So, these have different settings and goals, so they are different!
However, every SL problem can be cast as an RL problem. See this answer. Similarly, in certain cases, you can formulate an RL as an SL problem. So, although the approaches are different, they are related.
Is RL an unsupervised learning approach?
In RL, you do not tell the agent what action it needs to take. You only say that the action that was taken was "bad", "good" or "so so". The agent needs to figure out which actions to take based on your feedback. In SL, you explicitly say that, for this input $x_i$, the output should be $y_i$.
Some people may consider RL is an unsupervised learning approach, but I think this is wrong, because, in RL, the programmer still needs to define the reward function, so RL isn't totally unsupervised and it's also not totally supervised. For this reason, many people consider RL an approach that sits between UL and SL.
What is deep learning?
The term/expression deep learning (DL) refers to the use of deep neural networks (i.e. neural networks with many layers, where "many" can refer to more than 1 or 1000, i.e. it depends on the context) in machine learning, either supervised, unsupervised, or reinforcement learning. So, you can apply deep learning to SL, RL and UL. So, DL is not only restricted to SL.
