1
$\begingroup$

How can we compare, in terms of similarity (and/or meaning), two pieces of text (or documents)?

For example, let's say that I want to determine whether a document is a plagiarized version of another document. Which approach should I use? Could I use neural networks to do this? Or are there other more suitable approaches?

$\endgroup$
0
2
$\begingroup$

There are more than 1 way of doing this:

  1. You can compute the bleu score between them if you are looking at the quality of machine translation. Check this link.
  2. You can convert them into 2 vectors using doc2vec and find the similarity between the vectors using cosine similarity.
  3. Siamese networks are something similar to what you are asking. They are neural nets that use distance metric for learning rather than a loss metric.

I don't understand why do you want to use neural network for comparing two text pieces. Generally comparisons are done by some of the distance metrics not by neural network.

$\endgroup$
0
0
$\begingroup$

It depends what you mean by "comparison", but in general I would think not really.

Neural networks operate on the sub-symbolic level, ie instead of handling discrete symbols (such as letters) they work with numerical values. These values can often be mapped onto symbols (eg through input or output nodes) which typically are letters or words.

If you want to compare texts, you are dealing with symbols, so it would probably be easier to operate on the symbolic level, by manipulating words directly, rather than translating them into numerical values and back, as that usually involves some loss of precision.

But as I said, it is hard to answer your question without knowing more detail about the exact nature of the comparison you're after.

$\endgroup$
1
  • $\begingroup$ I edited the original question to include the example given in the comment by the OP and to try to clarify the question. So, if you have some time, given that this question has a lot of views, I would suggest that you review your answer and try to address the current version of the question more directly. Feel free to edit again the question to make it more "meaningful" and clearer. $\endgroup$ – nbro Jan 19 at 19:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.