How about this ?
1 - Learn all the basic building blocks of possible sub-sequence
In our words sequence example, that would correspond to phonemes.
(I'm guessing that this step can even be done using unsupervised learning.)
So in the following example :
Hello Laurie, we would have learned 3 phonemes : HE, LO, RI.
2- Learn all subsequence as sequences of 'building blocks'
Using a ClockWorkRNN with timesteps of interval +1 with, let's say, 10-15 timestep (groups), that is fed the next 'phoneme id' in the sequence, we would have a space large enough to record most words (Obviously, the number of timesteps should be size of the biggest word).
This is the subsequences memory RNN.
Its sole purpose is to remember subsequences.
Now, i'm really brainstorming here , taking a very wild guess, but what if :
After training this RNN to a satisfying error rate, we check if the output of the RNN is very different to the next input for a couple of timesteps.
In other word, we see if the neural network has been able to 'guess' the next building block of the subsequence.
If not, then its a point of interest , because there is not a lot of possibilities as of why this would happend : the only one I see is
1 - The RNN is currently receiving another word, thus making this timestep a sub-sequence 'break point'
Do you guys see any points that could prove this theory wrong ?