Using a Neural Network (LSTM) to approve/reject word-type sequences

Question

I would like to train an LSTM neural network to either "approve" or "reject" a string based on the word-type sequence. For instance: "Mike's Airplane" would output "approved", but "Airplane Mike's" would output "reject". My method for doing this is to decompose the string into an array of words. eg.

["Mike's", "Airplane"]

, then convert the array of words to an array of word-types since the actual word is irrelevant. The word types (pronoun, noun, adjective etc.) are defined constants having numerical values. eg.

const wordtypes={propernoun:1, adjective:2, noun:3, ownername:4};
console.log(wordtypes.propernoun); // 1

Mike's Fast Airplane is

["Mike's", "Fast", "Airplane"] 

which becomes:

input:[properNoun, adjective, noun]
output: "approve"

properNoun represents the first word(Mike's), adjective the second word(Fast), and noun the third word(Airplane).

I would then like to use this array to train a Neural Network so that it can approve/reject other word-type sequences.

I am concerned with the methodology/algorithm rather than the syntax; I'm extremely new to Machine Learning and Artificial Neural Networks, so, I am using brain.js and NodeJS because they're relatively easy to use.

• I would like to input multiple parameters for a single word because many words have multiple word types (depending on the context). For example, a word can be both a "noun" and a "verb". How do I represent this input?

• Is this a good application for LSTM? Or is there a better-suited ML
  algorithm? My dilemma is in deriving the proper inputs & methodology to effectively train the Neural Network.

• How is my methodology for accomplishing this approval system?

Answer 1

You already figured out much of the problem. You can solve it with sequence models like LSTM/GRU.

One-hot encode word-types. Assume there are types of [properNoun, adjective, noun] as you said. Then "Mike" will be represented as a vector, [1,0,0], "fast" as [0,1,0], and "airplane" as [0,0,1].

Summing up these, you will train a model that takes [[1,0,0], [0,1,0], [0,0,1]] as input and return binary classification result. Thus you can use LSTM/GRU with return_sequences = False.

An example with Keras is here.

A few things you should be careful about:

• I have shown 2D input here but sequence models take 3D input in form of (batch_size, time_steps, number_of_features) so you will need to reshape even if you train with batch size 1. More details are here.
• Prepad the time dimension with 0 vector, so that number of time steps will be equal for all samples. More details are here.

For multi-word type, you can create multiple data instances and train the network.