1
$\begingroup$

I am currently trying to solve a classification task with a recurrent artificial neural network (RNN).

Situation

There are up to 350 inputs (X) mapped on one categorical output (y)(13 differnt classes). The sequence to predict is deterministic in the sense that only specific state transitions are allowed based on the past. A simplified Abstraction of my problem:

  • y - Ground Truth: 01020
  • y - Model Prediction: 01200
  • Valid Transitions: 01, 10, 02, 20

The predicted transition 12 is consequently not valid.

Question

What would be the best way to optimize a model to make as less invalid transition predictions as possible (ideally none)? (temporal shifts of the predictions in comparison to the ground truth are still acceptable)

  • By the integration of the knowledge about the valid transitions into the artificial neural network. Is this even possible to code hard restrictions into a RNN?
  • By a custom loss function which penalizes these invalid transitions
  • Another approach

Current Approach

With a bidirectional recurrent network (one gated recurrent units layer (2000 neurons)) in a many to many fashion an accuracy of 99.5% on the training and 97.5% on the test set could be reached (Implemented with TensorFlow / keras 2.2)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.