I had been looking at some OCR problems and came across this presentation. I implemented it.
In the presentation, there is the LSTM-Stack (diagram and algorithm, slide 32):
Here is a visualization of my LSTM stack in Keras:
I actually made my own attention model and I concatenated the attention part of the input for the 1st LSTM layer. In the hope of it adding some kind of context.
I am training the neural network for a couple of days so far.
Is this a good implementation? Please, give me any suggestions.