Get the name of a merchant from records

Question

I have a bunch of bank transaction records from which I want to extract merchants' names. In a few subsets of these records, the structure of the string is the same within the subset with only the merchant name changing. For example

subset 1

• XXXXX_ID_TIME_STAMP MERCHANT1 CREDIT
• XXXXX_ID_TIME_STAMP MERCHANT2 CREDIT

subset 2

• BILL PAYMENT BANK_NAME MERCHANT NAME 3
• BILL PAYMENT BANK_NAME MERCHANT NAME 4

In the above two subsets, the structure of the string is the same, only the merchant names changes

and so on ...

Using NLP, I want to extract merchant names in such cases. How should I approach this?

Using regex is not feasible because I'd have to manually go through the complete data, identify all such patterns and create regex strings that'll extract the name. I would also have to do this for every new pattern.

Is there a way where I can train a model that can identify/extract merchants in such cases?

Answer 1

The problem:

You are facing a Natural Language problem called Named Entity Recognition (that's the key word you are looking for). But before you dive deep into it, have in mind it's best suited for user input data (where users are absolutely chaotic) and it looks like you have a system data.

The right way:

You should have some kind of tabular (structured) data, instead of a string. So:

1. Triple check if you have some white-space separator (like tabs).
2. Review you content retrieval source and method.
3. If you have influence over the data source, ask them to generate it the right way.

If none of them help...

The programmatic way:

I can't see a specific Machine Learning model to solve that, but you can combine several techniques (some of them NLP) to help. It's an analytical and explorative process. Here are some insights:

the structure of the string is the same within the subset with only the merchant name changing

• If you have some absolutely identical records, except for the data you are looking for, you could simply look for the diff between them.

• Dictionary retrieval: If you have the list of Merchants, it's as simple as checking if a record contains any of the listed words. If you don't, you can build it as you run other methods. So maybe one subset can help you solving the other, like a Sudoku puzzle.

• Track special characters:

  • If some columns (XXXXX_ID_TIME_STAMP CREDIT BILL PAYMENT) contain numbers (or any special character), you can eliminate them right away.

• Tokenization: you can convert every word to a unique token. If the Merchant Name is composed by 2 or more words, you can use n-grams.

• Frequency Analysis: The names (tokens / n-grams) will probably have similar frequency inter and intra subsets. For example:

  • The bank name might be much more frequent than a merchant name.
  • A merchant might be frequent in one subset but rare in others.

• Divide each record into smaller substrings.

  • If you can eliminate some n-grams inside a record (using methods like special characters or frequency analysis) you'll have smaller (and more structured) problems to handle. For example: [_][_]MERCHANT[_]BANK[_]