Can I use AI to interpret XML documents?

Question

I think about a system which gets XML documents in various structures but with essentially the same data structure in it. For the example, let's assume each document contains data about one or more persons. So the AI would recognize a name. Somewhere else in the document there is the post address of our fictional person. The AI should now "see" the address and conclude, it belongs to our person. Anywhere else, there is a phone number in the document. Again, our AI should see the connection between our person and this phone number.

This wouldn't be a job for an AI if there wasn't a catch. If the task was merely to find and map strings like addresses and phone numbers, we could simply use a regex to match our "target strings". The catch in this scenario would be this: the XML document might contain other data, which does not belong to our person but is a valid phone number for example and thus will match an regex.

Would it be possible for an AI to learn this? If yes, with which framework would someone create such an AI?

---

Sample XML document:

<?xml version="1.0" encoding="utf-8" ?>
<document>
    <data>
        <foo>
            <bar>
                <person>
                    <name>John Doe</name>
                </person>
            </bar>
            <address>
                <street>Main street 1</street>
                <city>1111 Twilight town</city>
                <country>sample country</country>
            </address>
            <phone>+123 123 123</phone>
        </foo>
        <foo>
            <bar>
                <person>
                    <name>Jane Doe</name>
                </person>
            </bar>
            <address>
                <street>Broadway 42</street>
                <city>4521 Traverse town</city>
                <country>sample country</country>
            </address>
            <phone>+123 412123</phone>
        </foo>
    </data>
    <creator>
        <!-- Note: While this looks like a valid person, -->
        <!-- this data should not be matched by the AI -->
        <name>Sam Smith</name>
        <office>
            <street>Seaside road 5</street>
            <city>4521 Traverse town</city>
            <country>sample country</country>
        </office>
        <phone>+123 555 555</phone>
    </creator>
</document>

Answer 1

XML, HTML and less formal languages all respond quite nicely to being transformed or interrogated within a graph framework. XML and HTML are particularly useful in that they conform strictly to a tree-structure. That means that any good data components can be measured in terms of tree-distance to any other "good" data components.

If you extract your regex-friendly terms and keep track of where within their tree they are found, you may be able to cast those values into a general document-space vector (it might only need to be one-dimensional), allowing you to identify clusters of "good" vs anomalous sections of "bad" data based on a simple distance metric, or anomaly detection algorithm - say an isolation-forest that runs on information density for example.

This depends on your data, and how much of it you can find, ideally already tagged up containing good vs bad.

If you're looking to scrape reliable address-contents, then yes, you're likely to score hits on names, addresses, postcodes and phone-numbers all appearing as tightly connected clustered groups, all within one or two nodes-distance from one another.

Meanwhile, an annotation containing a phone number lodged somewhere else is less likely to be a match.

Different documents will have different threshold densities, and differing anomaly to conformant ratios, so you'd have a task on your hands to figure out some way to automatically tune your parameters on any given document set.

In the past, I've tried doing this against html by flattening all the content into a single string of text and a similar approach yielded half-decent results, but if you're looking at XML, it's fair to expect the structure to yield more information.

Answer 2

You would need to define 'frames', 'templates', or sets of data belonging together to form an address or other kind of data, with typical labels. So phone or tel etc would indicate a phone number, provided that their content also looks like a phone number. That's how you as a human recognise it. So you encode your domain knowledge as entities with possible attributes. Then you try and match the attributes and recognise which entity they belong to. You could have several entities with a shared subset of attributes (like a company or a person, which would both have an address). There would be other clues to tell you which it is. If name ends in "Ltd." or "Co", then it'll be a company, for example.

So you mix heuristics for identifying attributes, templates of which attributes combine to form an entity class you want to identify, and then pick the one that best matches. If you have several entities where all that's filled is phone, then you can't really tell what it is and would discard it. In your example, name matches various entities, but office would not be a valid attribute for a person (unless you decided it was). An oversimplistic heuristic might think "Sam Smith" is a company, which presumably has an office attribute, so you need to be careful with how you design your templates.

A criterion for putting attributes together could be that they are in the same subtree of the data structure. The exact definition really depends on the data and the types of information you want to extract from it.

So yes, it is definitely possible. I'm not really sure which framework, but it should be fairly trivial to code this in a programming language of your choice.