What actually does the self-loop (within the single hidden state ) in the Hidden Markov model helpful for?
I learn that one of the use cases concerning Natural language Understanding is that it helps a model to stay within the current state in case of (time variable: long sound/ short sound) pronunciation of the same word. But I can't understand what role the self-loop plays here.
Any explanations would be much appreciated.
The input audio is splitted into overlapping frames, for instance having size 40ms at a frame-rate of 20ms. For every frame $t$, some feature vector $O_t$ is observed. An input utterance with three frames would be represented as the sequence $O_1O_2O_3$.
Consider a model with no self-loops, a left-right model with three states and no skips either, with states $S=\{S_1,S_2,S_3\}$. Having no skips means that the transition from $S_1$ to $S_3$ is not allowed, it has probability $a_{13}=0$. This rigid model would be able to generate or recognize only observations of length $T=3$, and would necessarily assign the state $S_i$ to the observation $O_i$. There is no other possible alignment.
By allowing a self-transition in the central state we give temporal flexibility to the model, so different length utterances can be properly aligned and recognized. An observation of length $T=5$, for example, would be written $O_1O_2O_3O_4O_5$, and the model could generate $O_1$ at state $S_1$; $O_2$, $O_3$ and $O_4$ at $S_2$; and $O_5$ at $S_3$.
This self-transition models the number of observations in the second state (the duration $d_2$) with an exponential distribution of average $$\overline{d_2} = \frac{1}{1-a_{22}},$$ where $a_{22}$ is the probability of remaining in $S_2$. Note the extreme values $a_{22}=0$, giving expected duration $1$; and $\overline{d_2}$ growing unbounded as $a_{22}$ approaches $1$.
The symbols and formula used here come from the tutorial by Rabiner (1989).
