I am trying to create my own variant of Google duplex however, it won't make calls but just have a real-time conversation.

My question is, where and how to start?

How do I train my model with real conversation and how do I make speech sound almost human-like? Where do I incorporate RNN and how can I make my model understand nuances?

https://youtu.be/p3PfKf0ndik trying to create something like this.

  • $\begingroup$ So Google Duplex uses the WaveNet from DeepMind for the text-to-speech part. This is also the resons why it sounds so natural in terms of its voice. The other part is probably training an LSTM/RNN to do text conversations by using a lot of data which they collected. This is the part you probably cannot reproduce since you somehow have to collect a lot of conversations and label them in terms of its content, goal etc. $\endgroup$ Dec 30 '18 at 15:49

First of all, you need to realise that you will not be able to do it. Google is a multi-billion dollar company, with a large number of very bright and well-funded researchers. That tells me that it is not something a single person can do by themselves.

Then, you already have some pre-conceptions about it. You want to use a machine learning approach, using neural networks. I would start with a blank slate and think about the problem first, before considering the tools I would want to use.

If you don't know were to start, I would suggest you have a look at Eliza, which is old but still very hard to improve on. There are many open source implementations of it available, and it wouldn't be too hard to implement your own from scratch.

Eliza uses a symbolic approach (ie no neural networks), which has the big advantage that you can see what is going on, and you can actually understand how it works. Have a play around with it, create some chatbots, and see how you can make the conversations sound more human-like. Then, when you have a clearer idea about this, proceed to other frameworks such as wit.ai or Microsoft's LUIS. There you trade in control for ease of use; but I suggest you don't do this without understanding what you're actually doing first.


Not the answer you're looking for? Browse other questions tagged or ask your own question.