# How can I build an AI with NLP that read stories

I want to do an NLP project but I don't know if it's doable or not as I have no experience or knowledge in NLP or ML yet.

The idea is as follows: Let's say we have a story (in the text) that has 10 characters. Can we define them, their characteristics, whole sentences they said, and then analyze emotions within those sentences.

After that, is it possible to generate an audio version of the story where: the text, in general, is narrated by one voice, each individual character's sentences are read in a different voice generated specifically for that character. Finally is it possible to make the tones of the characters voices change depending on the emotions detected in their sentences?

• This doesn't sound like a machine learning problem - if written well you should be able to use a rule based decision on the sentences. For example if you have 'You never listen to my ideas!' Frank yelled, 'Perhaps that's because you've never had a good one.' Paul sneered back. You've got characters clearly stated and emotions. Now you may have to do a little expansion on this to cover all bases but for a machine learning example you need lots of examples where you know the true answer (as in who is truly speaking) to train your model. – Lio Elbammalf Jun 12 at 16:02
• @LioElbammalf it may not be a ML problem, but I think it can be considered AI and on topic here. If you think your comment is good enough to help OP achieve goals, then please turn it into an answer. However, I think significant NLP will be required, as most authors don't restrict themselves to such simply parsed phrasing – Neil Slater Jun 12 at 16:46
• @NeilSlater Indeed, I wasn't implying the question wasn't on topic, only that using machine learning may be the wrong tool for the job. – Lio Elbammalf Jun 13 at 8:35

This is quite an ambitious project, and IMHO well beyond the scope of what a single individual can do (within a reasonable time span) at present.

You need to first analyse the story text to identify the characters. This can already be quite a tricky task, as pronouns and other reference expressions are generally used to make a text less monotonous. If a character is referred to by name, say Jane, then you can assume that a follow-up the young woman will refer to her and not a male character mentioned in the same paragraph. But what about the young scientist? Such expressions can be very opaque, and you'd need a lot of world-knowledge to decode them correctly, as they can refer to any distinctive attribute of the character.

Identifying speech is a bit easier, unless you're talking about indirect speech. Jane was thinking aloud. She wasn't going to be able to do that. It was too hard. -- is that speech or not? Compare to Jane was thinking aloud: "I am not going to be able to do that. It is too hard.", which is the direct speech equivalent. Also, unless you're dealing with a play, most of the text will probably not be speech. For the audio version you will probably only want to deal with direct speech, which is usually (but not always) indicated by quote marks.

Analysing emotions seems to be comparatively easy if you have reached this stage, though if it is just based on keywords in the speech it probably won't be very accurate. If you can assign any descriptive statements to characters, that might be more successful, though by no means trivial.

Generating the text as audio should be straight forward. Most operating systems nowadays have speech synthesis integrated, and you can generally choose different voices, so if your text is marked up properly with which voice should speak which part it would be trivial.

To summarise: The NLP part is the hardest bit of it. As has been mentioned in the comments already, I don't think it's a problem that machine learning can help with, and I would stick to traditional methods of parsing the text into a structural representation and then applying rules to identify the bits you are interested in. The recognition of emotion might be a subtask that is suitable for ML, but in the past I have only applied pattern matching to similar tasks, so I can't really say much about that.

From my own experience in text analysis I would think that you might be able to get decent results with a few simple heuristics, but those will likely fail when it becomes a bit more complicated. A lot hinges on the type of story: children's fairy tales might be easier than War and Peace in that respect.

• Classical methods for parsing text with grammars is a good starting point and can be extended with Complex Narrative Understanding done with machine learning. Kočiský, Tomáš. Deep learning for reading and understanding language. Diss. University of Oxford, 2017. – Manuel Rodriguez Jun 14 at 7:18