How to configure a neural network to selectively change only certain characters in a string?

Question

I'm trying to figure out how to train a neural network to macronize Latin text. Essentially, in Latin, vowels can either be long or short, and length is indicated with a macronized character: i.e. o is short, ō is long. Most advanced texts don't mark vowel length, which can be an issue because vowel length determines pronunciation and word meaning: liber means book, līber means free, for example.

Is there any way to train a neural network to replace only vowels in a word while contextually evaluating it as part of a larger sentence? I've tried training an RNN many-to-many model to map unmacronized text to macronized text, but it ends up being incredibly inefficient and inaccurate because an unmacronized word can map to any macronized word. Ultimately, the only characters in each word that should ever change are the vowels, and even then only to their macronized counterparts: liber can be liber, līber, libēr, or lībēr only.

There is an abundance of training data in the form of parallel macronized and unmacronized text. This feels like a simple question but I've been unable to find any resources specifically detailing how to implement this. Any pointers would be greatly appreciated.

Answer 1

You could wrap each RNN cell in a custom module that is an identity for consonants (output = input when the input is a consonant) and predicts the macronization of vowels (it outputs the result from the RNN cell when the input is a vowel).

And it should always return the hidden state of the RNN cell so that each cell connects to the next cell for any input.