NLP: What is expected from the output of a perfect coreference system?

Question

For instance, consider the following piece of text:

'The father of Richard is a very nice guy. He was born in a poor family. Because of that, Richard learnt very good values. Richard is also a very nice guy. However, Richard's mother embarasses the family. She was born rich and she does not know the real value of the money. She did not have to be a hard worker to succeed in life."

How should a perfect coreference should work? Is this solution the perfect solution?

Cluster 1:

'The father of Richard' (first sentence) <-> 'He' (second sentence)

Cluster 2:

'Richard' (third sentence) <-> 'Richard' (forth sentence)

Cluster 3:

'Richard\'s mother' (fifth sentence) <-> She (sixth sentence) <-> she (sixth sentence) <-> She (seventh sentence)  

If I use the coreference library of spacy (neuralcoref), I get these clusters:

Clusters:

[Richard: [The father of Richard, He, Richard, Richard, Richard], a poor family: [a poor family, the family], Richard's mother: [Richard's mother, She, she, She]]

Note that this output says that "Richard" is the same for sentences that is true. However, "He" in the second sentence is not related to "Richard", but to his father. Also "Richard" and the "Father of Richard" are together in the same cluster. Furthermore, "poor family" and "family" should not come together. However, this is realy difficulty since in this case there is some level of ambiguity.'

I know that this is a very difficult problem. The point is not criticize this fantastic library. I am just trying to understand what I should expect as perfect result.

If I change a little the text:

'The mother of Richard is a very nice woman. She was born in a poor family. Because of that, Richard learnt very good values. Richard is a very nice guy. However, Richard's father embarasses the family. He was born rich and he does not know the real value of the money. He did not have to be a hard worker to succeed in life.'

The clusters are:

[Richard: [The mother of Richard, She, Richard, Richard, Richard, He, he, He], a poor family: [a poor family, the family]]

In this case, the clusters become stranger, since "She" and "Richard" are in the same cluster. Furthermore, the "He" related to the "father of Richard" belongs to the cluster, but not "Richard's father".

So, my question is:

What is the perfect result that I should expect from a "perfect" coreference system?

Answer 1

The clusters you mentioned at the beginning seem right. A "perfect" coreference system should be able to find all words/phrases that refer to the same object.

However, in language, there will always be some ambiguity. For example, consider the sentences "Bob and John like building. He has fun doing it.". It is ambiguous whether "He" refers to "Bob" or "John". In that sense, a perfect coreference system may not even exist theoretically. If it did, it would either avoid classifying "He" or recognize that it could go with either "Bob" or "John".