2
$\begingroup$

Are there any existing approaches for using artificial neural networks (ANN) or evolutionary algorithm (EA) for detecting coding standard violations? Which one would be more suitable?

I don't have any specific programming language in mind, but something similar to PHP_CodeSniffer (following these standards), but instead of using hardcoded rules, the algorithm should learn good techniques, but I'm not sure based on what training data. How would you approach the training session, any suggestions?

$\endgroup$
  • $\begingroup$ This asks for a summary of a whole area of activity, hence it's too broad. $\endgroup$ – Raphael Aug 12 '16 at 9:39
  • $\begingroup$ It asks about the ANN training approach. It has the link to some concrete example. $\endgroup$ – kenorb Aug 12 '16 at 10:08
  • $\begingroup$ If "instead of using hardcoded rules, the algorithm should learn good techniques" means it should learn new rules that have not yet been incorporated in a standard then the problem becomes much more difficult, because what constitutes "good coding" can be highly subjective. For example, some call the use of Exit Fors in VB.Net loops a "code smell," but they can be conducive to writing good programs. Writing AI to detect violations of a definite, existing standard is difficult; using it to create a new standard that will actually be productive in a non-subjective way may be intractable. $\endgroup$ – SQLServerSteve Aug 12 '16 at 17:06
  • $\begingroup$ I don't believe it's too broad, it has references to specific coding standards and I'm asking about specific approaches the training which can be problematic. $\endgroup$ – kenorb Aug 13 '16 at 3:01
4
$\begingroup$

If the system claims that a piece of code has violated standards, then to be useful to the programmer, it really needs to provide more information than just a 'yes/no' classifier: you need some form of explanation about why it is claimed to be wrong.

Clearly ANNs aren't much use for that.

If I were tackling such a problem (and my suspicion is that a lot of effort could be spent trying and failing to reproduce coding standards which are already well-understood), then my inclination would be to use a more explicitly rule-based representation.

Possibilities include:

  • Genetic Programming

  • Learning Classifier Systems

  • Decision Trees

The ever-useful "Essentials of Metaheuristics" has a whole section on the evolution of rulesets. Obviously, nothing prevents you from initializing the evolutionary process with rules known to be useful.

As I point out here, with our current AI algorithms, the success of an approach is very sensitive to human expertise/effort in feature selection/preprocessing, choice of training set etc, so creative experiment with this is vital.

Training set: how about two sets of negative and positive examples, consisting of (features extracted from) bad code and from a refactored version (respectively)?

One elementary choice of features would be to apply a bunch of code complexity metrics and have the learning algorithm combine those. The plus side of working in such a numeric domain is that the learning algorithm might readily find a gradient to exploit. The downside is that the rules (which are then likely of the form if mcabe > 2.8 etc) are still not as informative as might be desired.

For more complex rules (e.g. requiring if elseif else) you may want to extract your features from the abstract syntax tree. You could in principle use the entire tree but to my knowledge ML on graph and tree structures is still in relative infancy.

$\endgroup$
  • $\begingroup$ I don't think that providing more info other than an answer "yes" or "no" is necessarily useless. It can be useful because it might point out which parts of the code are potentially not following the standards. $\endgroup$ – nbro Jun 2 at 16:08

Not the answer you're looking for? Browse other questions tagged or ask your own question.