7 votes

Is it possible to clean up an audio recording of a lecture using some type of AI system?

Yes, it is possible. Usually, the noise reduction is done using regular signal processing methods, such as spectral subtraction due to demand for low latency. But, of course, modern methods of deep ...
igrinis's user avatar
  • 171
3 votes
Accepted

Is it realistic to train a transformer-based model (e.g. GPT) in a self-supervised way directly on the Mel spectrogram?

The reason most music-generation models use discrete representations is because the long-term structures of music are very challenging to model. Note that the MIDI data in MAESTRO (used in the two ...
Robz's user avatar
  • 204
2 votes

How can I find a specific word in an audio file?

After some research on the internet, I realized that using VOSK toolkit in python, it can be found (detect) any particular word in audio file or real time audio streaming. https://alphacephei.com/vosk/...
Ali.kavari76's user avatar
2 votes

I want to determine how similar a given song is to Queen's songs. Am I headed in the right direction?

You are heading in the right direction to make an audio-based classifier. This is not quite the same as providing a similarity metric between two pieces of audio, but it may do as a first attempt. You ...
Neil Slater's user avatar
  • 30.3k
2 votes

How to get more accuracy of the logistic regression model?

Try Rectification Improve the features available to your model, Remove some of the NOISE present in the data. In audio data, a common way to do this is to smooth the data and then rectify it so that ...
Faizy's user avatar
  • 1,094
2 votes

How is speech recognition software able to distinguish between different speakers and yet still understand them all?

I would slightly disagree with Ryan's answer: the fundamental frequency is mainly specific to a speaker. Sounds are defined by other frequency patterns. Vowels, for example, have two bands of energy ...
Oliver Mason's user avatar
  • 5,362
1 vote
Accepted

How to combine input from different types of data sources?

Handling Multiple Input Types Multiple input types are common as learning technology moves from academic labs and open source examples into the real world. The design process includes the below four ...
Douglas Daseeco's user avatar
1 vote

Difficulty understanding Keras LSTM fitting data

Yes, intuition says that RNNs like LSTM or GRU will work better in your case, because predicted values might depend on input patterns corresponding to much earlier time intervals. There is no reason ...
Roman's user avatar
  • 21
1 vote

How is speech recognition software able to distinguish between different speakers and yet still understand them all?

A sound byte can be decomposed into a set of features which are distinguishable by a classifier. The most important feature extracted is the fundamental frequency. This is the lowest frequency ...
Ryan's user avatar
  • 111
1 vote
Accepted

What are the differences between RVC and SO-VITS-SVC models?

From what I've seen, these two models have similar architectures since they took the architecture of SoftVC and combine it with the design of VITS. The RVC is a succession of SoVITS and it has some ...
Cat ALog's user avatar
1 vote

What type of neural network architecture allows filtering out of unwanted sounds?

Since you say that you "believe that a neural network architecture is needed here...", I am assuming that you are open to other options. This approach doesn't utilize a neural networks, but ...
Snehal Patel's user avatar
1 vote
Accepted

How to define a loss function for multi-label problem?

Given your response in the comments, you are faced with a semi-supervised learning problem where you have a small set of data with ground truth labels, and a large set of data without ground truth ...
Raphael Lopez Kaufman's user avatar
1 vote

How to define a loss function for multi-label problem?

If I understand correctly, the training data are voice messages which contain one or more suggested labels for the class that the voice message belongs to. Even though there are multiple suggestions, ...
dmc-au's user avatar
  • 21
1 vote

Is it realistic to train a transformer-based model (e.g. GPT) in a self-supervised way directly on the Mel spectrogram?

These papers are also very close to what I meant in the question (too long for a comment). The following references come mostly from work on speech recognition. Mockingjay In this work, they use an ...
Peter Franek's user avatar
1 vote
Accepted

Can we use transformers for audio classification tasks?

Yes, Transformers can be used to work with audio data, such as audio processing (audio classification, speaker identification, etc) (Audio ALBERT), speech-to-text (Streaming Automatic Speech ...
user3667125's user avatar
  • 1,510
1 vote

How to combine specific CNN models that work better at slightly different tasks?

What you're describing are "Ensemble Models" -- where multiple models are trained in parallel, and then combined at inference time to squeeze out better performance. This article gives a ...
Robz's user avatar
  • 204
1 vote

How to use AI for language recognition?

Google has an API you can use. https://cloud.google.com/translate/. Their API can translate audio to text. They also have an API for converting speech to text. The language detection feature should ...
jgleoj23's user avatar
  • 182
1 vote

Deep audio fingerprinting for word search

You would need to perform some kind of speech-to-text to get the audio transcription with the corresponding synchronization wrt the audio. Then search in the ...
shoegazerstella's user avatar
1 vote

Can AI be used to reverse engineer a black box?

Yes this is entirely possible. As was previously mentioned, complex connectionist systems are often thought of as black boxes(despite us being able to "look in" the box given enough computation and ...
hisairnessag3's user avatar
1 vote

What are the current open source text-to-audio libraries?

voice-builder is an opensource text-to-speech (TTS) voice building tool from Google.
Balraj Ashwath's user avatar
1 vote

Isolate the speech of two people in an audio record with two people only

The task of isolating 2 or more speakers is called speaker diarization, here a list of softwares and useful resources. Once you have the 2 or more audio files ...
shoegazerstella's user avatar
1 vote

Can I filter barking sounds on the television?

OK, here is one approach. Acquire a data set of 'clean' audio samples without barking dogs and an data set of barking dogs sounds. Generate a training set by mixing random selections of clean audio ...
DrMcCleod's user avatar
  • 593

Only top scored, non community-wiki answers of a minimum length are eligible