Has there been research done regarding processing speech then building a "speaker profile" based off the processed speech? Things like matching the voice with a speaker profile and matching speech patterns and wordage for the speaker profile would be examples of building the profile. Basically, building a model of an individual based solely off speech. Any examples of this being implemented would be greatly appreciated.


Yes, there is. An extremely quick search found this: Multimodal Speaker Identification Based on Text_and_Speech.

Let me tl;dr for you: (My abstract addition in Italics)

Novel method for speaker identification based on both speech utterances and their transcribed text.

They first transcribed text of each speaker’s is processed by using probabilistic latent semantic indexing (PLSI) that models each speaker’s vocabulary which is closely related to his/her identity, function, or expertise.

The speech to text used by users is DARPA's Efficient, Affordable, Reusable Speech-to-Text (EARS) Program in MetadataExtraction (MDE).

By using Melfrequency cepstral coefficients (MFCCs) and dynamic range is quantized to a number of predefined bins in order to compute MFCC local histograms for each speech utterance, which is time-aligned with the transcribed text.

To test they used RT-03 MDE Training Data Text and Annotations corpus distributed by the Linguistic Data Consortium.

As for results: Identification rate versus Probe ID when 44 speakers are employed. Average identification rates for (a) PLSI: 69%; (b) MFCCs: 66%; (c) Both: 67%.

If you need more papers related, you could use a tool like https://the.iris.ai/ to find related papers.

Post edit: Hopefully now this post complies with the standards.


Speaker identification is quite widely researched domain. Modern approach would be to map speaker information to i-vector, a real-valued vector of 200-400 components that characterizes speaker fully. i-vectors allow very precise speaker identification and verification.

For more information you can check i-vector tutorial

Also you can check state of the art in the results of NIST i-vector challenge

For implementation, you can check the following speaker recognition experiment from Kaldi.

For best accuracy i-vectors are extracted with DNN UBMs, watch out that GMM UBMs are less accurate.

For more in-depth information about speaker recognition methods and algorithms check this textbook.


Deepmind recently created a voice synthesiser along those lines. It seems to be incredibly slow, but it might be possible to create a dumped down version of it.

Apparently the task is called parametric TTS (text to speech). This overview might give you some leads.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.