Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
It is not quite clear to me whether the statistical approach superseded the rule-based system in the 90s.
McMahon and Smith (1998) report that many other researchers used "hybrids of statistical and formal approaches". What do they mean? Maybe it was still unclear which one was best suited for?
No. In commercial applications, rule-based systems are still widely used, because they are easier to explain and debug: if there is an error, you add or change a rule. In a statistical system you need to adjust the training data, re-train your system, and hope that it solved the problem.
What is sometimes used is a hybrid approach, where eg a stochastic part-of-speech tagger is used in conjunction with post-processing rules.
In some commercial areas (eg finance) there are regulatory constraints that mean you cannot rely purely on statistical systems.
[Note: obviously I cannot cite any sources, as this is not the kind of thing written about in academic papers. But I have in the past ten years worked at a number of businesses working on NLP, and they predominantly use rule-based systems, with a small proportion of stochastic components to support the rule-based one (intent-recognition, sentiment analysis, and pos-tagging).]
