If one uses one of the open source implementations of the WaveNet generative speech synthesis design, such as https://r9y9.github.io/wavenet_vocoder/, and trains using something like the CMU's arctic corpus, now can one add a voice that sounds younger, older, less professional, or in some other way distinctive. Must the entire training begin from scratch, or is there a more resource and time friendly way?
$\begingroup$
$\endgroup$
asked Sep 16, 2018 at 14:55
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The original wavenet and the implementation you linked to is globally conditioned on speaker embeddings, which means that the network was given a unique identifier for the person speaking each time it trained on an audio clip. This allows the network to learn to mimic the voice of each person in the training data, but it only learns their voices, not arbitrary people's voices.
You might be able to do what you're describing by globally conditioning on different features of the speakers, such as age etc, though this would require a lot a speakers to span those dimensions.
I think it would definitely be possible to use style transfer for this, similar to the work done on tweaking people's age in images [1]. Some similar work has been done for audio in order to change the accent of a voices between accents [2], so I wouldn't be surprised to see more examples of this in the near future.
[1] https://www.youtube.com/watch?v=2qMw8sOsNg0 [2] https://ai.googleblog.com/2018/03/expressive-speech-synthesis-with.html
