In the paper, Contextual String Embeddings for Sequence Labeling, the authors state that
\begin{equation} P(x_{0:T}) = \prod_{t=0}^T P(x_t|x_{0:t-1}) \end{equation}
They also state that, in the LSTM architecture, the conditional probability $P(x_{t}|x_{0:t})$ is approximately a function of the network output $h_t$.
\begin{equation} P(x_{t}|x_{0:t}) \approx \prod_{t=0}^{T} P(x_t|h_t;\theta) \end{equation}
Why is this equation true?
In section 2.1 of the paper, the authors state that the goal of character-level language model is to estimate the following joint probability distribution
$$P(\boldsymbol{x}_{0:T}) = P(\boldsymbol{x}_{0}, \boldsymbol{x}_{1}, \dots, \boldsymbol{x}_{T}),$$
which is a joint probability distribution of all characters of a sequence of $T+1$ characters, where $\boldsymbol{x}_i$ is the $i$th character of the sequence. Ideally, the joint probability distribution, $P(\boldsymbol{x}_{0:T})$, should put more mass (or density) on the combination of $T-1$ characters that are more likely (in a given language). For example, in English, the combination of the characters "the" is more likely than the combination "bibliopole". Hence, $P(\text{t}, \text{h}, \text{e})$ should be higher than $P(\text{b}, \text{i}, \text{b}, \text{l}, \text{i}, \text{o}, \text{p}, \text{o}, \text{l}, \text{e})$, where, in this case, $T=2$. So, $P(\boldsymbol{x}_{0:T})$ is the actual character-level language model, which is represented by a joint probability distribution.
Similarly and intuitively, the conditional probability distribution $$P(\boldsymbol{x}_{t} \mid \boldsymbol{x}_{0:t-1})$$
tells us the probability of the next character of the sequence $\boldsymbol{x}_{t}$ given (that is, having observed) the previous characters of the sequence $\boldsymbol{x}_{0:t-1} = (\boldsymbol{x}_{0}, \dots, \boldsymbol{x}_{t-1})$. In general, the next character of a sequence $\boldsymbol{x}_{t}$ can depend on all previous characters of the same sequence, $\boldsymbol{x}_{0:t-1}$. If we are able to learn the conditional model $P(\boldsymbol{x}_{t} \mid \boldsymbol{x}_{0:t-1})$, given a sequence of $t$ characters, $\boldsymbol{x}_{0:t-1}$, we can sample the next character according to the same conditional probability distribution.
Recall that, given two events (or random variables) $A$ and $B$, the joint distribution of them is defined as $P(A, B) = P(A \mid B)P(B) = P(B \mid A)P(A)$, which gives rise to the Bayes' theorem. This can be easily generalised to multiple variables. More specifically, if you consider $B$ to a set of $N$ events (rather than just one), that is, $B=B_1 \cap B_2\cap \dots \cap B_N = B_1, B_2, \dots, B_N$, then $P(A, B) = P(A \mid B)P(B)$ still holds, but we can further decompose it
\begin{align} P(A, B) = P(A, B_1, B_2, \dots, B_N) &= P(A \mid B_1, B_2, \dots, B_N)P(B_1, B_2, \dots, B_N) \\ &= P(A \mid B_1, B_2, \dots, B_N)P(B_1 \mid B_2, \dots, B_N) P(B_2, \dots, B_N) \\ &= \cdots \end{align}
This is called the chain rule (or product rule) of probability. Essentially, we apply the rule $P(A, B) = P(A \mid B)P(B)$ recursively.
Analogously, in the paper, the authors apply this chain rule to express the joint distribution $P(\boldsymbol{x}_{0:T})$ as a product of conditional probability distributions, that is
$$ P(\boldsymbol{x}_{0:T}) = \prod_{t=0}^T P(\boldsymbol{x}_{t}\mid \boldsymbol{x}_{0:t-1}) $$
In the case of the LSTM, the vector $\boldsymbol{h}_t$ is supposed to keep track of the past. More specifically, in the case of the character-level language model, we train an LSTM-based RNN, so that $\boldsymbol{h}_t$ is approximately equal to $\boldsymbol{x}_{0:t-1}$, that is, $\boldsymbol{h}_t \approx \boldsymbol{x}_{0:t-1}$. Hence, the joint probability distribution of the characters above can be now be approximately defined as a function of the vector $\boldsymbol{h}_t$
$$ P(\boldsymbol{x}_{0:T}) \approx \prod_{t=0}^T P(\boldsymbol{x}_{t}\mid \boldsymbol{h}_t; \boldsymbol{\theta}) $$
where $\boldsymbol{\theta}$ are the parameters of the LSTM-based RNN. Note that this is not an equality but an approximation. Intuitively, we train the LSTM so that it learns the interactions of the past characters.
