sentence alignment between same language texts

Question

Take this two texts as an example:

"I will start the recording and share the presentation. I hope it's all clear from what we saw last time on classical cryptography and if you remember we got to discussing perfect security. Ah, you're right, I didn't upload the slides. Wait, they are the same as last time. If you want to get the face on the fly, last time I mean last year. I had some things to do and didn't upload."

"Okay, I will start recording and I will share the presentation again. 1, 0 So I hope it's all clear compared to what we saw last time about classic victory, And if you remember we got to let's say discuss the perfect security. Ah you could see already, I didn't upload the Slides, Wait they are the same as last time, Eh. If you want to have done on the fly, I mean last time, I mean from last year. Morning I had some stuff to do, I didn't upload."

We want to align the sentences so that similar sentences (within a certain degree of difference) are being matched together. Which are the approaches to solve the issue?

[I will start the recording and share the presentation], 
[I will start recording and I will share the presentation]


[I hope it's all clear from what we saw last time on classical cryptography],
[I hope it's all clear compared to what we saw last time about classic victory]

[and if you remember we got to discussing perfect security],
[And if you remember we got to let's say discuss the perfect security]
.
.
.

I have been looking into DTW and perceptual hashing as a way to solve the problem without any concrete result then I saw that in the field of automatic translation sentence alignment is widely used but with the assumption that the two texts have different languages and that there is a one-to-one mapping between words without "gaps" or extra words in between.

Answer 1

First, you should first address the problem of 1. splitting sentences, and 2. the number of sentences in two texts. Then you can use a sentence embedding model and compare the similarity between the sentences in two sets of sentences.

The easiest way could be averaging the ready-made word embedding vectors (word2vec, fastText, etc.) of all words in the sentence and using the resulting vector as the embedding vector for the sentence. Or you can use Doc2Vec directly on the sentences. Then you can use cosine as a similarity criterion between the sentence embedding vectors.

Another easy way is to use the BLEU metric to compare the similarity of the sentences.

Answer 2

With this data, a simple similarity measure that will find a high value between the pairs "discuss" and "discussing", as well as between "classic" and "classical" can be the number of common characters from start divided by the root of the product of lengths. For the same words, similarity will be one, for words with no common prefix, zero. For close words such as the examples mentioned the value will be well above 1/2, useful enough for matching.

Based on this similarity measure, Dynamic Time Warping should solve the problem.

Or, instead of word alignment, character alignment with a simpler similarity function, 1 for equal characters, 0 for different. That would help align vIcTORY with crypTOgRaphY.

I don't see much need for word embedding for this particular application. The only word equivalence not detected by blind character evaluation is "stuff"=="things", but those words share three words on the left and two on the right, so they will be aligned anyway by a blind (and multilingual) method such as char-level DTW. Also, sentence splitting can be obtained as a byproduct of character alignment. Once aligned, the character zip can be cut where either thread has a point, or also a comma, if desired.