1
$\begingroup$

When recording audio for screencasts or similar, very often the keyboard is clearly visible and can start to annoy listeners after a while.

NN are quiet good at recognizing patterns. Image classification is all over the place these days. There is also some work on audio, so that seems to work as well. Could the following approach therefore work to eliminate (or greatly reduce) the sounds of the keyboard in a recording whilst leaving the voice quality largely untouched?

  1. Train a NN to recognize the clicking sounds of the keys. Lots of labeled data can be created by just recording and tracking key clicks in the millisecond range. That way markers can be placed on the recording automatically that "label" clicks from non clicks. Let's say a click has on average a 10ms range in the audio, the audio feed could be cut into snippets of 10ms and those that have a click sound in it are labelled as such.
  2. A adversarial network is trained to modify an input stream so as to fool the first one into thinking there are no clicks while also being punished for large changes in the stream data. So the better it removes the clicks sounds the better but if it just gives out nothing (technically no clicks then), it's of course bad so there needs to be some reward for being "close to input"

Would this be a good approach? Are there other ways to filter this? I know there is an "ehm detector" that uses MDP to warn speakers whenever they are likely to say "ehm". This wouldn't apply to this though, because it's not that I want to guess when the next click comes but rather I want to manipulate the input stream without running a constant filter on the entire stream such as a lowpass filter for removing unwanted constant noise. So ideally the algorithm would learn to apply a "correction stamp" whenever a click is detected to remove a range of frequencies during small windows in the overall recording but leaving most of it untouched.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.