7
votes
Is it possible to clean up an audio recording of a lecture using some type of AI system?
Yes, it is possible. Usually, the noise reduction is done using regular signal processing methods, such as spectral subtraction due to demand for low latency. But, of course, modern methods of deep ...
- 171
3
votes
Accepted
Is it realistic to train a transformer-based model (e.g. GPT) in a self-supervised way directly on the Mel spectrogram?
The reason most music-generation models use discrete representations is because the long-term structures of music are very challenging to model. Note that the MIDI data in MAESTRO (used in the two ...
- 204
2
votes
How can I find a specific word in an audio file?
After some research on the internet, I realized that using VOSK toolkit in python, it can be found (detect) any particular word in audio file or real time audio streaming.
https://alphacephei.com/vosk/...
- 111
2
votes
I want to determine how similar a given song is to Queen's songs. Am I headed in the right direction?
You are heading in the right direction to make an audio-based classifier. This is not quite the same as providing a similarity metric between two pieces of audio, but it may do as a first attempt. You ...
- 30.3k
2
votes
How to get more accuracy of the logistic regression model?
Try Rectification
Improve the features available to your model, Remove some of the NOISE present in the data.
In audio data, a common way to do this is to smooth the data and then rectify it so that ...
- 1,094
2
votes
How is speech recognition software able to distinguish between different speakers and yet still understand them all?
I would slightly disagree with Ryan's answer: the fundamental frequency is mainly specific to a speaker. Sounds are defined by other frequency patterns.
Vowels, for example, have two bands of energy ...
- 5,362
1
vote
Accepted
How to combine input from different types of data sources?
Handling Multiple Input Types
Multiple input types are common as learning technology moves from academic labs and open source examples into the real world. The design process includes the below four ...
- 7,443
1
vote
Difficulty understanding Keras LSTM fitting data
Yes, intuition says that RNNs like LSTM or GRU will work better in your case, because predicted values might depend on input patterns corresponding to much earlier time intervals.
There is no reason ...
- 21
1
vote
How is speech recognition software able to distinguish between different speakers and yet still understand them all?
A sound byte can be decomposed into a set of features which are distinguishable by a classifier. The most important feature extracted is the fundamental frequency. This is the lowest frequency ...
- 111
1
vote
Accepted
What are the differences between RVC and SO-VITS-SVC models?
From what I've seen, these two models have similar architectures since they took the architecture of SoftVC and combine it with the design of VITS. The RVC is a succession of SoVITS and it has some ...
- 26
1
vote
What type of neural network architecture allows filtering out of unwanted sounds?
Since you say that you "believe that a neural network architecture is needed here...", I am assuming that you are open to other options. This approach doesn't utilize a neural networks, but ...
- 922
1
vote
Accepted
How to define a loss function for multi-label problem?
Given your response in the comments, you are faced with a semi-supervised learning problem where you have a small set of data with ground truth labels, and a large set of data without ground truth ...
1
vote
How to define a loss function for multi-label problem?
If I understand correctly, the training data are voice messages which contain one or more suggested labels for the class that the voice message belongs to. Even though there are multiple suggestions, ...
- 21
1
vote
Is it realistic to train a transformer-based model (e.g. GPT) in a self-supervised way directly on the Mel spectrogram?
These papers are also very close to what I meant in the question (too long for a comment).
The following references come mostly from work on speech recognition.
Mockingjay In this work, they use an ...
- 432
1
vote
Accepted
Can we use transformers for audio classification tasks?
Yes, Transformers can be used to work with audio data, such as audio processing (audio classification, speaker identification, etc) (Audio ALBERT), speech-to-text (Streaming Automatic Speech ...
- 1,510
1
vote
How to combine specific CNN models that work better at slightly different tasks?
What you're describing are "Ensemble Models" -- where multiple models are trained in parallel, and then combined at inference time to squeeze out better performance.
This article gives a ...
- 204
1
vote
How to use AI for language recognition?
Google has an API you can use. https://cloud.google.com/translate/. Their API can translate audio to text. They also have an API for converting speech to text. The language detection feature should ...
- 182
1
vote
Deep audio fingerprinting for word search
You would need to perform some kind of speech-to-text to get the audio transcription with the corresponding synchronization wrt the audio. Then search in the ...
1
vote
Can AI be used to reverse engineer a black box?
Yes this is entirely possible. As was previously mentioned, complex connectionist systems are often thought of as black boxes(despite us being able to "look in" the box given enough computation and ...
- 1,235
1
vote
What are the current open source text-to-audio libraries?
voice-builder is an opensource text-to-speech (TTS) voice building tool from Google.
- 339
1
vote
Isolate the speech of two people in an audio record with two people only
The task of isolating 2 or more speakers is called speaker diarization,
here a list of softwares and useful resources.
Once you have the 2 or more audio files ...
1
vote
Can I filter barking sounds on the television?
OK, here is one approach.
Acquire a data set of 'clean' audio samples without barking dogs and an data set of barking dogs sounds.
Generate a training set by mixing random selections of clean audio ...
- 593
Only top scored, non community-wiki answers of a minimum length are eligible