I want to start a project for my artificial intelligence class about speaker recognition. Basically, I want to train my AI to detect if it's me who's speaking or somebody else. I would like some suggestions or libraries to work with.


The human voice is based on the neural muscular control of vocal apparatus made up of many parts.

  • Diaphragm
  • Vocal cords
  • Throat (constrictors and anti-constrictors)
  • Nasal cavity
  • Cheek
  • Jaw
  • Tongue

These coordinated muscular manipulations produce envelopes (controlling) of audio that can be characterized by periodic and transient wave forms.

  • Volume
  • Pitch
  • Tone (relative volume of harmonics)
  • Consonant transients

Voices are unique to the learning state of neural activity and anatomic attributes, which is a way of saying that vocal habits and the physical attributes of the voice supports the distinguishing of vocal identity.

  • Strength of vocal muscles
  • Connectivity of muscles to bone, tendons, and cartilage
  • Shape of inner surface of vocal pathways
  • Neural coordination of those muscles
  • Neural production of phonetic control to produce linguistic elements
  • Neural serialization of semantic structures (ideas)

The detection of distinguishing features of voices by the ear is equally complex. In a room full of people talking, the brain can learn to track a single voice.

It is important to note that performing voice recognition to determine the identity of the human source is significantly different than performing voice recognition to produce text. To produce text accurately, the NLP must determine language elements and construct a semantic network that represents the vocal content or a text from that representation to be accurate in the case of like sounding words. Fortunately, the identification of the speaker is easier in some ways than the accurate voice to text. Unfortunately, the identification of the speaker has general limitations discussed below.

The first stage of hearing in the ear is mechanical, involving the length of hairs along the cochlear surface, which is like a radio tuner that discriminates all frequencies within a range simultaneously. The software equivalent is a spectrum derived by applying a root mean square to the result of an FFT (fast Fourier transform) to provide magnitudes.

$$ m_f := \sqrt{t_f^2 + {(it_f)}^2} $$

The phase component of the FFT results ($\, \arctan(t, it) \,)$ can be discarded, since it is not correlated with neural control of voice.

The application of the FFT to speech (as with any changing audio) requires windowing over the audio samples using one of the windowing tapers, such as the Hann window or Blackman window. The input is the audio stream or file contents as a sequence of pressure samples, the audio. The output is a sequence of spectra, each containing the volume of each frequency in the vocal range, from about 30 Hertz to 15 K Hertz.

This series of spectra can be fed into the initial layer of one of the more advanced RNNs (recurrent neural networks), such as the LSTM (long short term memory) networks, its bidirectional version, the B-LSTM, or a GRU (gated recurrent network), which is touted as training equally well with less time or computing resource consumption.

The identity of the speaker is the label. The series of spectra are the features.

Using the PAC (probably approximately correct) learning framework, it may be possible to estimate, in advance of experimentation, the minimum number of words the speaker must speak to produce a particular accuracy and reliability in use of the learned parameters from the network training.

It will take some study to set up the hyper-parameters and design the layers of the network in terms of depth (number of layers) and width sequence (number of cells per layer, which may vary from layer to layer).

The use case limitation of this system is that each speaker must read some text that provides adequate training example sequences of adequate length, so that there are sufficient number of overlapping windows for the FFT to transform into spectra so that the training converges reasonably.

There is no way around the individual user training as there is with recognition of linguistic content, which can be trained across a large set of speakers to recognize content somewhat independent of the speaker. The system can be adjusted and improved to minimize the amount of speech required, but information theory constraints keep that quantity from ever approaching zero.

No network, whether artificial or biological, can learn something from nothing. Claude Shannon and John von Neumann realized decades ago that there is a kind of conservation of information, just as there is a conservation of matter and energy in space below nuclear reaction thresholds. This led to the definition of a bit and the formulation of information as a quantity of bits corresponding to a narrowing of probability that the information provides.

$$ b_i = - \log_2 {\frac {P(x|i)} {P(x)}} $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.