# Using a Neural Network (LSTM) to approve/reject word-type sequences

I would like to train an LSTM neural network to either "approve" or "reject" a string based on the word-type sequence. For instance: "Mike's Airplane" would output "approved", but "Airplane Mike's" would output "reject". My method for doing this is to decompose the string into an array of words. eg.

["Mike's", "Airplane"]


, then convert the array of words to an array of word-types since the actual word is irrelevant. The word types (pronoun, noun, adjective etc.) are defined constants having numerical values. eg.

const wordtypes={propernoun:1, adjective:2, noun:3, ownername:4};
console.log(wordtypes.propernoun); // 1


Mike's Fast Airplane is

["Mike's", "Fast", "Airplane"]


which becomes:

input:[properNoun, adjective, noun]
output: "approve"


properNoun represents the first word(Mike's), adjective the second word(Fast), and noun the third word(Airplane).

I would then like to use this array to train a Neural Network so that it can approve/reject other word-type sequences.

I am concerned with the methodology/algorithm rather than the syntax; I'm extremely new to Machine Learning and Artificial Neural Networks, so, I am using brain.js and NodeJS because they're relatively easy to use.

• I would like to input multiple parameters for a single word because many words have multiple word types (depending on the context). For example, a word can be both a "noun" and a "verb". How do I represent this input?

• Is this a good application for LSTM? Or is there a better-suited ML
algorithm? My dilemma is in deriving the proper inputs & methodology to effectively train the Neural Network.

• How is my methodology for accomplishing this approval system?

• Language is complicated. Me and my friend know two Mikes, one of whom is a pilot, and we call him "Airplane Mike". I ask "Whose drink is this?" and my friend replies "Airplane Mike's". Contrived? Yes. This kind of construct is a real problem for NLP systems? Yes. It doesn't stop you making up any set of word rules you like and training an LSTM to learn it, but it may help if you explain whether your goal is just an abstract rule learning thing for teaching yourself RNNs, or whether you intend to use it for real world language processing, or for some other purpose. Jul 22 at 21:09
• Yes, it's a real-world application, but not NLP. It's actually input validation for a web-based system. For example: input: [ adj, noun] output: "reject" input: [ noun, noun, noun] output: "reject" input: [ pronoun, adj, noun] output: "approve" input: [pronoun, noun] output: "approve" etc. The input is of variable length; "noun", "pronoun" etc are defined as integer constants. Jul 23 at 1:26

You already figured out much of the problem. You can solve it with sequence models like LSTM/GRU.

One-hot encode word-types. Assume there are types of [properNoun, adjective, noun] as you said. Then "Mike" will be represented as a vector, [1,0,0], "fast" as [0,1,0], and "airplane" as [0,0,1].

Summing up these, you will train a model that takes [[1,0,0], [0,1,0], [0,0,1]] as input and return binary classification result. Thus you can use LSTM/GRU with return_sequences = False.

An example with Keras is here.

A few things you should be careful about:

• I have shown 2D input here but sequence models take 3D input in form of (batch_size, time_steps, number_of_features) so you will need to reshape even if you train with batch size 1. More details are here.
• Prepad the time dimension with 0 vector, so that number of time steps will be equal for all samples. More details are here.

For multi-word type, you can create multiple data instances and train the network.