Are there existing examples of using neural networks for static code analysis?

Question

Background Context:

In the past I've heavily applied various "code quality metrics" to statically analyze code to provide an inkling of how "maintainable" it is and using things like the Maintainability Index alluded to here.

However, a problem that I face is whether a language has libraries that effectively measure such metrics - only then is it usable else it's rather subjective/arbitrary. Given the plethora of languages that one has to deal with in an enterprise system, this is can get rather unwieldy.

Proposed Idea:

Build and train an Artificial Neural Network that "ingests a folder of code" (i.e., all files within that folder/package are assumed to house the "project" whose quality metrics we'd like to compute). This may again be language dependent but let's assume it exists for a language that I'm having the hardest time with (for measuring "maintainability"): Scala.

Using numeric metrics like McCabe's complexity or Cyclomatic complexity maybe "convention" but are not entirely relevant. Few things like class/method length are almost always relevant no matter the language. Thus, providing a few "numeric metrics" + abstract notion of readability by subjective evaluation to train an ANN would be a good balance of "inputs" to the ANN. The output being either a classification of maintainability like low, medium, high etc., or a number between 0 and 1.

Question:

Has this been tried and are there any references? I spent some time digging via Google Scholar but didn't find anything "usable" or worthwhile. It's okay if it's not Scala, but have ANNs been used for measuring code quality (i.e., static analysis) and what are the benefits or disadvantages of something like this?

PS: Hopefully, the question isn't too broad, but if so, please let me know in the comments and I'll try make it as specific as possible.

Answer 1

There's certainly literature on a related topic: code smell detection.

A "code smell" is a sign that code has a maintenance problem, and hints at the presence of technical debt. It is reasonable to suppose that code with a lot of smells is lower quality. Code smells include things like giant classes, high cyclomatic complexity, and more.

Fontana et al. have a good 2016 survey comparing different ML methods for detecting code smells. A reverse citation search on that paper uncovers many other papers that seem relevant, including:

• Rasool et al. 2015, A review of code smell mining techniques
• da Silva Maldonado et al. 2017 Using Natural Language Processing to Automatically Detect Self-Admitted Technical Debt
• Maneerat & Muenchaisri, 2011, Bad-smell prediction from software design model using machine learning techniques
• Mansoor et al. 2017, Multi-objective code-smells detection using good and bad design examples

This seems like a pretty well-studied area. I wasn't able to find a cross-language model though, but I suspect one may well exist.