How to efficiently interpret phrases in a Python AI?

Question

I have started to make a Python AI, and thee beginning of its code looks something like this:

print "ARTEMIS starting. . ."

import random
import math
import os

greet = ['HI', 'HELLO', 'HEY', 'GOOD MORNING', 'GOOD DAY', 'GOOD AFTERNOON',               'GOOD EVENING', 'GREETINGS', 'GREETING']
joke = ['TELL ME A JOKE', 'JOKE', 'FUNNY', 'TELL ME SOMETHING FUNNY']
insult = ['YOURE A LOSER', 'YOU ARE A LOSER', 'YOU STINK', 'IDIOT', 'JERK',     'FOOL', 'DUMMY', 'HOOLIGAN', 'YOURE DUMB', 'YOURE STUPID', 'YOU ARE DUMB', 'YOU     ARE STUPID']
maker = ['WHO MADE YOU', 'WHO PROGRAMMED YOU', 'PLEASE TELL ME WHO MADE     YOU', 'PLEASE TELL ME WHO PROGRAMMED YOU']
name = ['ARTEMIS', 'A.R.T.E.M.I.S.', 'HEY ARTEMIS', 'HEY A.R.T.E.M.I.S.',     'ARTIE', 'HEY ARTIE', 'HELLO ARTEMIS', 'HELLO A.R.T.E.M.I.S.', 'HELLO ARTIE']
myAge = ['HOW OLD AM I', 'WHAT IS MY AGE', 'MY AGE']
tip = ['GIVE ME A TIP', 'TIP', 'LESSON', 'GIVE ME A LIFE LESSON', 'LIFE     LESSON', 'DO YOU HAVE A LIFE LESSON TO SHARE']
language = ['WHAT PROGRAMMING LANGUAGE WAS USED TO MAKE YOU', 'WHAT     PROGRAMMING LANGUAGE DO YOU USE', 'PROGRAMMING LANGUAGE']
compliment = ['COOL', 'AWESOME', 'I LIKE YOU', 'EXCELLENT', 'YOURE COOL',     'YOURE AWESOME', 'YOU ARE COOL', 'YOU ARE AWESOME']
maths = ['LETS DO MATH', 'CALCULATE', 'CALCULATOR','DO MATH', 'MATH',      'PLEASE DO MATH', 'DO ARITHMETIC', 'ARITHMETIC', 'PLEASE DO ARITHMETIC']
game = ['GAME', 'LETS PLAY A GAME', 'LETS HAVE FUN', 'WANT TO PLAY A GAME']
gender = ['WHAT GENDER ARE YOU', 'ARE YOU A BOY OR A GIRL', 'ARE YOU MALE OR     FEMALE', 'BOY OR GIRL', 'MALE OR FEMALE', 'GENDER', 'ARE YOU A BOY OR GIRL']
guessWhat = ['GUESS WHAT', 'GUESS WHAT ARTEMIS', 'GUESS WHAT     A.R.T.E.M.I.S.', 'GUESS WHAT ARTIE', 'YOU WONT BELIEVE IT', 'YOU WILL NOT     BELIEVE IT', 'YOU WONT BELIEVE IT ARTEMIS', 'YOU WILL NOT BELIEVE IT ARTEMIS',     'YOU WONT BELIEVE IT A.R.T.E.M.I.S.', 'YOU WILL NOT BELIEVE IT A.R.T.E.M.I.S.',     'YOU WONT BELIEVE IT ARTIE', 'YOU WILL NOT BELIEVE IT ARTIE']
cls = ['CLEAR SCREEN', 'CLEARSCREEN', 'CLS', 'BLANK']
lawsOfRobotics = ['WHAT ARE THE LAWS OF ROBOTICS', 'WHAT ARE THE THREE LAWS     OF ROBOTICS', 'LAWS OF ROBOTICS', 'THREE LAWS OF ROBOTICS']
itsName = ['WHATS YOUR NAME', 'WHAT IS YOUR NAME', 'WHO ARE YOU']

However, I would like to know if I could make it detect "similar" phrases instead of trying to come up with every possible phrase someone would type. How can I do this?

Answer 1

To compare the strings you can use Fuzzy string matching. FuzzyWuzzy is a python package that does this using the Levenshtein distance, which calculates the difference between two strings by counting the number of single character edits (insert, delete, substitute) that have to be done to make them equal.

---

To install the package you can use pip for example:

pip install fuzzywuzzy

Then you can use it to check the similarity between two strings (100 is the highest score; higher score means more similar):

>>> from fuzzywuzzy import fuzz
>>> fuzz.ratio("How are you?", "How are you?")
100
>>> fuzz.ratio("How are you?", "How are you doing?")
80

Or, which might be most interesting to you, find the most similar string from a list of strings:

from fuzzywuzzy import process
choices = ['HI', 'HELLO', 'HEY', 'GOOD MORNING','TELL ME A JOKE', 'JOKE', 'FUNNY','GIVE ME A TIP', 'TIP', 'LESSON','WHAT IS YOUR NAME', 'WHO ARE YOU', 'HOW ARE YOU?']

You can get a list of the phrases with the highest score (parameter limit, 2 in this example):

>>> process.extract("hello there", choices, limit=2)
[('HELLO', 90), ('HEY', 60)]
>>> process.extract("please make a joke", choices, limit=2)
[('JOKE', 90), ('TELL ME A JOKE', 69)]

Or just get the one with the highest score:

>>> process.extractOne("whoe are you?", choices)
('WHO ARE YOU', 96)
>>> process.extractOne("I want a lesson", choices)
('LESSON', 90)

Note:

There are cases where another phrase might give a relatively high score also:

>>> fuzz.ratio("How are you?", "Who are you?")
83

But as long as you also have "Who are you" and "How are you" in the list, they will be detected correctly:

>>> process.extract("Who are you?", choices, limit=2)
[('WHO ARE YOU', 100), ('HOW ARE YOU?', 91)]
>>> process.extract("How are you?", choices, limit=2)
[('HOW ARE YOU?', 100), ('WHO ARE YOU', 91)]

---

Also see: FuzzyWuzzy: Fuzzy String Matching in Python