I am currently trying to solve a classification task with a recurrent artificial neural network (RNN).
Situation
There are up to 350 inputs (X) mapped on one categorical output (y)(13 differnt classes). The sequence to predict is deterministic in the sense that only specific state transitions are allowed based on the past. A simplified Abstraction of my problem:
- y - Ground Truth: 01020
- y - Model Prediction: 01200
- Valid Transitions: 01, 10, 02, 20
The predicted transition 12 is consequently not valid.
Question
What would be the best way to optimize a model to make as less invalid transition predictions as possible (ideally none)? (temporal shifts of the predictions in comparison to the ground truth are still acceptable)
- By the integration of the knowledge about the valid transitions into the artificial neural network. Is this even possible to code hard restrictions into a RNN?
- By a custom loss function which penalizes these invalid transitions
- Another approach
Current Approach
With a bidirectional recurrent network (one gated recurrent units layer (2000 neurons)) in a many to many fashion an accuracy of 99.5% on the training and 97.5% on the test set could be reached (Implemented with TensorFlow / keras 2.2)
