How might AI analyze abusive discussion using natural language grammar?

Question

Opening thoughts

This does not only apply to SE comments, but the idea in general.

This is not a Question for Linguistics.SE; those Questions might come later, after AI analysis. Example Linguistics Quesions:

• What grammar categories might AI use to research for an analysis of abusive vs helpful discussion? (before the AI research, after this OP Question is answered)
• What grammar patterns can we identify from AI researched that analyzed abusive vs helpful discussion? (after the AI research)

This is a Question about how AI might be useful in the real world, thus helping AI programmers decide where to effectively focus energies.

AI analyzes comments and discussion

Many web apps and sites (including Facebook, YouTube, and Stack Exchange) analyze posted content using what some people call AI algorithms.

Presuming this is used also for comments on posts on sites such as these...

AI may take many factors into consideration, viz buzz words (type 'COVID' on a post and watch the info-notice pop up), profanities, bigotous phrases, etc.

I'm curious about the results if AI analyzed just the grammar of a history of comments that were deemed "abusive" juxtaposed against a history that was not deemed abusive.

Why ask?

Creative-analytical thinkers like Steven Levitt (viz Freakonomics) and Malcolm Gladwell like to discuss counter-intuitive findings from research. Levitt says that this is "economics" (nothing to do with money). Even the video game League of Legends has stats on how often certain gaming choices (items, champion, etc) win and lose. But, we need the data. I want to know if "grammar" is a good place to dig.

I would be curious if there were any grammar patterns that might indicate abuse, as might be found by AI research from past comments. Pardon the grammar lingo, but for example:

• Complex subjects
• Imperatives
• Subjunctives
• Passives
• Verbal pauses (Bothering to type out "Um..." in "Um... No.")
• Direct Objects vs Indirect Objects vs "Raised Objects"
• Prepositions

...Say research finds that comments containing "at" were 60% more likely to be flagged as abusive. That would be great content to ask on Linguistics to see if there were other patterns to analyze.

I don't know what should be analyzed, nor do I know what all grammatical categories would go into such an analysis. That would be my next Question for Linguistics.

Scope of my question

I'm trying to ask for open-ended answers, not binary (yes/no) answers, while also narrowing scope. So, let me put it this way...

Can AI be used to analyse abusive vs non-abusive discussions through grammar patterns and categories?

If so, which models or algorithms can be used to achieve that? References are also appreciated.

Answer 1

Part 1: Identifying and or Scoring Posts

There are various levels and types of analysis one might seek, depending on the output desired. At perhaps the simplest level, text classification could place the text into binary buckets of pass and fail. This level is similar to basic spam filtering. More complex classification could involve more buckets and or give a numerical rating.

A perhaps closer term for the task in question is sentiment analysis. From Wikipedia:

Sentiment analysis (also known as opinion mining or emotion AI) is the use of natural language processing, text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, and study affective states and subjective information.

Natural language processing (NLP), itself, can be sub-divided into three broad approaches: symbolic, statistical, and neural. These days, and particularly in the context of AI, the last option is likely given the most attention.

The main reasons for choosing neural networks over traditional hard-coded logic are flexibility, often less coding, and support for elusive, perhaps heuristic logic that may otherwise be difficult to systemise. Conventional statistical methods may resolve some complexity over manual logic, yet possibly at the expense of inferential depth. With deep neural networks, obtaining deeper inference, for better classification and analysis, sometimes can be as simple as training on a network with more parameters, for more neurons.

Neural-based language models like GPT-3 and GPT-NeoX (third-party playground available) highlight the functional yet elusive reasoning sometimes expressed by trained neural networks. Their logic is often heuristic and hidden, a phenomenon termed black box. For those new to language models, I recommend learning more (ie. by searching "GPT-3" on YouTube) and perhaps using a playground, like the one linked above.

No doubt, the specific details of implementation can vary greatly. One could, for example, run the input through a grammar classifier (neural or otherwise), followed by feeding the resultant parts of speech (plus tense) into a sentiment analysis network. Doing so could hypothetically be cheaper to train, possibly, though not necessarily, at the expense of quality. As neural computing gets cheaper, the trend is simply using a bigger network with more training data, hopefully giving a better result with minimal coding.

The amount of time required for development depends on available data -- especially tagged data -- and computing. A person with experience in sentiment analysis could possibly design or adapt a network in days. But preparing the training data could be a challenge, particularly for a large network. If you had access to, say, the set of flagged posts for a popular site, the data may already have the desired level of tagging and already be enough for training.

Indeed, if insufficient data is available, a grammar pre-processor, as described earlier, may help. In general, less data means smaller networks and more coding. With the right skills, off-the-shelf pre-trained language models, like those mentioned earlier, may be able to support or provide text classification, perhaps even for this purpose.

Part 2: Finding Trends in Language Usage

Whether done manually or though AI, once the posts have been categorised, many options exist for identifying trends in grammar and word usage.

At its simplest, basic word frequency could be analysed using conventional coding. For example, a sorted word-frequency mapping could be made and compared between each population of tagged posts, with the most frequent words first.

If context-sensitive properties, such as grammatical mood, tense, or part of speech are of interest, then a grammar pre-processor is likely in order. Such a parser could be neural or conventional. The resulting output could be hierarchical, a simple array, or perhaps a map. Analysis of the results could be simple, like described above for word frequency; or the results could be trained through a neural network to establish more complex inference.

Since language is generally one-dimensionally arranged, sequential pattern mining may be applied in search of unknown patterns. Presumably each population of tagged posts would be analysed separately, followed by taking the difference between the result sets. Those patterns of greatest discrepancy might be insightful. This step could be applied to either the raw tokens (ie. words), or the parsed grammar output.

Other options include cluster analysis and self-organizing map, although special pre-processing, from a broad set of possibilities, may be recommended for these.

Assuming the set of pattern types is open-ended, the set of programming paths is rather unbound.