I want to try to make an HMM from scratch for POS tagging, which I would extend to grammar checking. I understand there's much better ways for a grammar checker but this is just as a learning experience and for fun.
My idea is to do rule-based grammar checking based on the sequence of POS tags. I would generate the POS tags through my HMM then enforce concrete grammar rules. For example, I would assign POS tags to a sentence then perform the following check (out of many): if an adjective comes after a noun, is there a linking verb in between? If there isn't, then the sentence most likely contains an error.
However, I understand that HMMs seek to maximize the probability of assigning a POS tag based on the POS of the previous tag. My main concern is that if I only introduce sentences with proper grammar, and in turn proper POS tag sequences, then wouldn't my model never assign an improper POS tag and my rule-based check would never work since every POS tag is a valid sequence?
For example, take this slice of a full sentence: "He wants watch...". The issue here is obvious, that it should be "he wants TO watch..." The problem I'm imagining is that if my model only gets data on sentences with proper VB->VB transitions, i.e. the second verb is in the "ing" form, or the second verb is infinitive, etc... then wouldn't the probability of an improper VB->VB transition such as in "he wants watch" be parsed as "NN VB NN" because this improper verb to verb transition would have 0 probability as it's never seen in the data. In this case, I would assume that the model takes the transition VB->NN since "watch" is also a noun, and VB->NN has a non-zero probability
I'm not sure if I'm overthinking this, or if this is valid. Are there any fixes? I imagined that smoothing may fix this without changes to the dataset. I also considered introducing sentences with incorrect grammar to the dataset on purpose, but that seems inefficient. Is it possible to stick to using only a HMM and enforcing concrete rules for a grammar checker?
