Is there a way to use python and AI to compare thousands of files and detect the ones containing "unusual" content?

Those files are supposed to have "homogeneous" configuration types with different parameters, but sometimes some of them might contain "illegal" types of configurations. The issue is that there are too many ways to do homogenous and illegal configurations, so it is not possible to statically define them.

Here is one of too many bloc of configuration considered "correct": (the meaning doesn't matter)

vlan XXXX
  name  XXX_NAME

interface vxlan 1
  vxlan vlan XXX vni YYYY

router bgp YYYY
   vlan XXXX
      rd A.B.C.D:YYYY
      route-target both YYYY:YYYY
      redistribute learned

Thousands of files will contains such bloc of configuration (with different parameters, xxxx, yyyy...) But some might contain something else, example

Interface vlan AAAA
  ip address B.B.B.B/Z

Because it is in only a couple of files ("rare") it is more likely that it is not authorized configuration.

I could use simple regular expression to match what is considered legal or illegal, but there is too many of such blocs.

I am wondering if AI could help with this issue?

  • 1
    $\begingroup$ It may be a good idea to provide an example of a simple "correct configuration file" and an "usual/incorrect configuration file" to have a better idea of the problem. Do you also have many examples of these configuration files, or would you need to create them artificially or collect them manually? $\endgroup$
    – nbro
    Jan 4 at 15:59
  • 1
    $\begingroup$ What makes an illegal configuration illegal? How do you detect that? $\endgroup$
    – AlDante
    Jan 5 at 7:03
  • $\begingroup$ Hey, what I call "illegal" is lines different from the most frequent ones (correct). I'll complete the question with an example. $\endgroup$
    – AJN
    Jan 5 at 13:08

1 Answer 1


Here is a quick idea: first calculate the count of how many times each word occurs in these documents (I don't know whether to lowercase them or not, do interface and Interface mean different things?), and sort them in the descending order of occurrence. Most frequent words can be called "keywords" of your configurations (such as vlan), or maybe frequently occurring ip addresses. It is up to you whether to include ips to this list or not.

Then write rules to replace "arbitrary" data (numbers, ips, ...) with a common tag (for example <NUMBER>, <IP>, ...). You should replace all non-keyword words (aka. tokens) with these tags. Use <?> for tokens you don't identify with a regex, or leave them as-is.

Now we can calculate statistics of these token n-grams, and identify those which occur very rarely. This step will require some thingking and tuning, for example how to handle newlines (or just ignore them?), how long n-grams to use, how rare is too rare etc.

This is more of a heuristic than an AI, but it works without any training data. Well, there is a human in the loop so at least it is an Intelligent solution.

  • $\begingroup$ Nice idea @NikoNyrth. I'll give it a shot. $\endgroup$
    – AJN
    Jan 7 at 20:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .