If I train a speech recognition model using data collected from N different microphones, but deploy it on an unseen (test) microphone - does it impact the accuracy of the model?
While I understand that theoretically an accuracy loss is likely, does anyone have any practical experience with this problem?
Yes it can. However, other differences between training and test data with audio could have greater effect:
Identity of the speaker (including effects from gender, age, physical build, local accent, amongst others)
Acoustics of the recording environment (including proximity to the microphone, size of space, presence of hard surfaces, background noise)
If any of these may vary from your training data, then it becomes harder to predict your generalised accuracy during training and early model selection.
One possibility is to ensure your cross-validation set (which you absolutely should have) also separates data out by things that will vary from training to test. So instead of random train/cv split, you split by data that is key for generalisation. This is sometimes called a stratified train/test split.
If your only concern is variation in microphone, then split your train/cv sets by microphone type. You will get a better assessment early on in the model selection process how well the training is generalising, and can focus your search on models that do well despite this expected difference.
The most usual differences in signal records caused by different microphones will have small if not null impact in the recognition accuracy, in particular if we are talking about changes one mic by another of same model and manufactor:
However, if we talk about a general recognition system to be used with very different types of mics, there are some microphone issues that can cause your system completely fail:
By example, for IOT applications, the first two of this list are the more challenging ones.
