We want to figure out the connection between people, based on their speech. Assume, that a conversation is a poetry with lines belongs to characters. There are a lot of poetries and the lines are mixed. Now we want to define a conversation to which each line belongs to. We assume, that people in a conversation use similar words (their dictionaries should be similar). It means that there is a correlation between words of person A and words belongs to person B, and we could detect a connection between the peoples which had a conversation. What are the next steps for content understanding after NLP? Can some of you advise us about the field of study and tools/libraries, which are dealing with content processing? Maybe, some of you know good articles or online resources, which can help us to dive into this field.


These are really two questions -
1. How to assign lines in a dialogue to particular speakers
2. How to analyse the content of a speaker's utterances

I will attempt to answer the first one.

In discourse analysis (the field that studies conversations), there is the concept of adjacency pairs, linked utterances which typically follow on to each other. A well-formed conversation can be segmented into a series of adjacency pairs, such as greeting/greeting, question/answer/feedback (that's more of an 'adjacency triple'), statement/comment, farewell/farewell. Sometimes it's a bit more tricky, such as question/counter-question/answer/answer:

young person: What beers do you have? (Q)
bartender: How old are you? (Counter-Q)
young person: 21 (A)
bartender: We have a lager or a nice IPA. (A)

I don't know if this has been tried before, but it should be comparatively easy to identify the types of speech acts in a conversation and join them up to form relevant adjacency pairs. This would be a lot more reliable than looking at vocabulary, especially since conversations mostly use higher-frequency words which are part of everyone's active vocabulary.

You would need to take some conversations, classify the individual lines as speech acts (building up an inventory as you do that), and then identify adjacency pairs. This will give you the structure of the conversation, and should also allow you to assign speakers to utterances (assuming a dialogue, ie only two participants). Note that it is not always purely form that determines the speech act: How are you? is a question on the surface, but its function is part of a greeting sequence.

This conversational structure could also assist you in looking at the second part, the content. If you know someone asks questions, and the other participant answers them, then you know exactly where to look for salient utterances that might give you clues as to the topic of the conversation. At least it should enable you to ignore the 'housekeeping' utterances like greetings and farewells etc.

Your second question is hard to answer without knowing more about the purpose of your project. I would rephrase this and ask it as a separate question. People would need to know what kind of conversations they are, and what exactly you want to get out of it.

Oh, and I wouldn't compare conversations with poetry. Completely different things! :)


Not the answer you're looking for? Browse other questions tagged or ask your own question.