I wish to design a neural network that will categorize messages based on criteria I have predefined. It should feature the ability to be proactively trained as it continues its lifecycle. This means a human can intervene in its categorization attempts and determine whether or not it was correct and have it adjust its weights accordingly (without having to retrain all over again).
It is know that ALL input will follow these rules:
- Always of $N$ length
- All messages are transformed to eliminate unnecessary complexity
Here's a brief overview of how an example message might be processed.
Starting with a message $M$:
That's an interesting perspective. I think that you should consider adding more details to your point about the cat being too silly.
The text is then transformed so that extra details are removed:
thats an interesting perspective i think that you should consider adding more details to your point about the cat being too silly
Then it's converted into a vector (appending $0$ to reach length $N$) ready to be processed by the neural network:
[116, 104, 97, 116, 115, 32, 97, . . . , 0, 0, 0, 0]
In my network, I wish all the outputs to be weighted on how well they fit in each category. I need multiple outputs. I'm not really focusing on one particular category per-say, but how well the message fits in all of them.
Following the input $M$ I used as an example, I'd expect the outputs to look something like this after my vector has fed-forward:
The weight values for each output indicate the strength of the category in the overall message.
From message $M$:
That's an interesting perspective.
Would weigh the opinionated category as $0.68703$.
I think that you should consider adding more details to your point about the cat being too silly.
Would weigh the suggestive category as $0.89042$.
Summary and Questions
I'm interested in the architectural design choices of a network that would support my feature set. The main goal is to be able to train my network to categorize messages based on pre-trained (and live-trained) data. I'd like to know things like:
- What type of Neural Network I should use for this purpose? I've researched LSTM & Recurrent networks; which have been mentioned to be good at processing sequences (ie. messages).
- What considerations should I account for when creating this network?.
- How can the overall network support live-training so I can tell my network when its wrong and have it 'correct' itself without having to retrain completely?