4
$\begingroup$

If I train a speech recognition model using data collected from N different microphones, but deploy it on an unseen (test) microphone - does it impact the accuracy of the model?

While I understand that theoretically an accuracy loss is likely, does anyone have any practical experience with this problem?

$\endgroup$
  • $\begingroup$ Yes it should. I would guess that if you normalize the entire data set then it should alleviate things somewhat but I have no evidence for that belief. $\endgroup$ – k.c. sayz 'k.c sayz' Feb 2 '18 at 16:05
  • $\begingroup$ Welcome to AI! Great practical question. I don't have practical experience with this re: AI, but from years of recording experience, I can tell you that microphone quality can greatly affect the quality of the input. As an analogy, at my office we have a PDF-to-Word function on the copier for scanned documents. Comparing the output to Adobe's also poor conversion capability, I noticed that the scanning introduced significantly more errors to the same document, no doubt a function of the extra noise and loss of fidelity in the scan. $\endgroup$ – DukeZhou Feb 2 '18 at 19:35
3
$\begingroup$

Yes it can. However, other differences between training and test data with audio could have greater effect:

  • Identity of the speaker (including effects from gender, age, physical build, local accent, amongst others)

  • Acoustics of the recording environment (including proximity to the microphone, size of space, presence of hard surfaces, background noise)

If any of these may vary from your training data, then it becomes harder to predict your generalised accuracy during training and early model selection.

One possibility is to ensure your cross-validation set (which you absolutely should have) also separates data out by things that will vary from training to test. So instead of random train/cv split, you split by data that is key for generalisation. This is sometimes called a stratified train/test split.

If your only concern is variation in microphone, then split your train/cv sets by microphone type. You will get a better assessment early on in the model selection process how well the training is generalising, and can focus your search on models that do well despite this expected difference.

$\endgroup$
1
$\begingroup$

The most usual differences in signal records caused by different microphones will have small if not null impact in the recognition accuracy, in particular if we are talking about changes one mic by another of same model and manufactor:

  • Differences in bandwidth: voice is in a very common (central) bandwidth, it is not expected these differences impacts, even for low quality microphones.
  • Microphone distortions: same as previous, they will not impact because they are smaller than, by example, a change in the speaker.

However, if we talk about a general recognition system to be used with very different types of mics, there are some microphone issues that can cause your system completely fail:

  • mic sensitivity: small sensitivity differences will have no effect because they are solved in the same way than differences in speaker volume/intonation. However, if the microphone is not enough sensible the S/N can be below the minimum need, in particular when speaker increase the distance to the mic.
  • lack of beam-forming: if your system is prepared to use an array of microphones to filter noise and/or secondary sources, usage of a normal phone will decrease accuracy.
  • changes in sample ratio and/or sample bits: if the microphone and its A/D has a low sampling speed or size (i.e. Bluetooth mics, phone lines, ...), the accuracy can fail.

By example, for IOT applications, the first two of this list are the more challenging ones.

$\endgroup$
  • $\begingroup$ You know this has very less to do with what he is actually asking..he is asking for the accuracy of Machine Learning model not the device and how it will impact the final result...not exactly how are microphones different and how we can improve its accuracy. $\endgroup$ – DuttaA Feb 10 '18 at 18:23
  • $\begingroup$ @DuttaA, OP ask for practical experience: after years working with IOT microphones and voice recognizers, I think I known a few the most critical points about them. However, answer updated with some clarifications. $\endgroup$ – pasaba por aqui Feb 10 '18 at 18:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.