0
$\begingroup$

I want to use a machine learning algorithm to detect false address data. I learned about neural networks and machine learning at university, but I don't have much experience in this field.

Do you think it is feasible to use a high level algorithm for this or should I use simple queries and filters to catch out wrong data?

Improve this question
$\endgroup$
  • 1
    $\begingroup$ Can you manually do the same task? $\endgroup$ – pasaba por aqui Jul 27 '18 at 15:11
  • 1
    $\begingroup$ What do you mean by "false"? Data that is not actually in the right format for an address, or addresses that are wrong for a particular location or person? $\endgroup$ – Oliver Mason Jul 27 '18 at 15:43
  • $\begingroup$ - Yes I can do the task manually $\endgroup$ – Fatih Öz Jul 27 '18 at 17:17
  • $\begingroup$ - My mistake the data is sometimes not in the right format, sometimes its in the right format but there are spelling errors and sometimes the street is not in the written city. $\endgroup$ – Fatih Öz Jul 27 '18 at 17:18
  • $\begingroup$ Well... Then you have to teach it the spelling rules of English. I'd honestly just use a regex, or check it against a database. $\endgroup$ – FreezePhoenix Jul 27 '18 at 20:55
1
$\begingroup$

The comments are off base. Having worked in validation of data as a consultant for Nasdac, Amex, and Lexis Nexis I can tell you that using the UNIX sed -r or pcrelib is insufficient to do a stellar cleansing of address data.

Although none of those companies did this at the time I was consulting, the application of current machine learning is easy to infer from the basic characteristics of data. What is needed is a good record of incoming data, cleaned data, and rejected records to constantly use as a reference.

The manual process would be a nightmare. For instance, misspellings are more indicative of true data than properly spelled ones in some but not all cases (unless the falsified data generator is programmed to synthesize the existing distribution of misspellings in authentic data sources).

Try to profile that in static code such that the data validation will adapt to data trends without requiring a maintenance workforce. Those kinds of things never get addressed in a typical IT environment. Non-adaptive validation would be a gross mistake, especially if the volume is high, like for M16 or Liberty Cross or the NSA or Interpol.

What you want is an extremely fast index of well authenticated good addresses (perhaps corroborated by public records with local authentication policies) and a feature extraction from it using auto encoding or some similar and perhaps better methodology and its associated algorithms.

Then you can train the classification of fake and authentic addresses based on extracted feature profiles.

You will also want reinforcement, because the attackers (those trying to create false identities using plausible addresses) will adapt to the training of the current system. The authentication system must stay steps ahead of those trying to defeat it, which they will likely try to do once the existence of authentication automation is detected.

One can fend of attackers by placing misinformation strategically and then tracing the input sources back to the attacker based on the misinformation seeds. That only works if you have law enforcement in your camp.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.