Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [audio-processing]

For questions about audio processing tasks in the context of artificial intelligence.

Filter by
Sorted by
Tagged with
0 votes
0 answers
23 views

Segment a spectrogram into a series of images by (constant) beats per minute to train a Deep Neural Network

I have a .csv file with information about a soundtrack and it is divided into beats (per minute), which are ordered by row. As in: the index corresponds to each beat, and the columns have info about ...
Johnathan Smitherton's user avatar
0 votes
0 answers
10 views

How to construct source padding mask for embedded audio?

I'm attempting a music transcription task - similar to speech recognition but with music and notes (string representations) instead of speech audio and sentences. The model consists of a CNN audio ...
jy99's user avatar
  • 1
0 votes
0 answers
52 views

Drum sound classification using RNN issues - help needed

I am new to the field of machine learning, even tho I have solid background in semi-related fields (am control system engineer by trade) and as a hobby project I wanted to work a bit with sound ...
APasagic's user avatar
0 votes
0 answers
25 views

Audio-2-face AI to create a talking head from audio and a picture

I want to convert an audio track and a cartoon, or a picture of a puppet, to a video of that face uttering the audio. I found and tried several options, none of them satisfactory: NVIDIA Omniverse ...
emonigma's user avatar
0 votes
0 answers
27 views

What is the input size and sequence length for lstm when the input is audio data where each audio file is sampled at the rate of 3000hz?

I have music data which contains raw vocals of some indian classical and devotional songs and I have segmented each file into 20 seconds files. I have used librosa.load function with a sample rate of ...
Sai Abhishek Bhyri's user avatar
0 votes
0 answers
22 views

Does it make sense to use static masks to improve the quality of a zero-shot source separation model

I'm thinking about ways to fine-tune a neural network for source separation, specifically AudioSep It is known that in nlp tasks, RAG is often used as an alternative to fine tuning to provide LLM with ...
araxal's user avatar
  • 1
0 votes
0 answers
12 views

Looking for an audio classification approach

I am working on a deepfake audio classification project with a dataset consisting of only 3000 samples. I have made several attempts to address this challenge. Firstly, I extracted melspectrograms and ...
user21456801's user avatar
0 votes
2 answers
81 views

How is speech recognition software able to distinguish between different speakers and yet still understand them all?

Some background: I'm an EE major and data science minor, so I have a basic understanding of machine learning - I had a one semester course on it where we covered some of the most commonly used ...
Mikayla Eckel Cifrese's user avatar
2 votes
1 answer
6k views

What are the differences between RVC and SO-VITS-SVC models?

I'm trying to decide which one to use for my project but I can't find anywhere specific differences or comparisons of the models.
Hiperfly's user avatar
0 votes
0 answers
54 views

Feature extraction from log-mel spectrograms using CNNs

I am currently working on an ASR-related project in which I would like to combine Convolutional Neural Network with GRU Network and CTC loss function. The idea is to use the CNN to extract ...
conyai's user avatar
  • 1
2 votes
0 answers
42 views

How to prepare audio data for deep learning?

Audio data is typically an array with the waveform represented by values from -1 to 1. There are two issues with that: if all values are inverted, e.g. -1 becomes 1 and 1 becomes -1, the audio doesn'...
nikishev.'s user avatar
  • 121
1 vote
1 answer
65 views

What type of neural network architecture allows filtering out of unwanted sounds?

I have a use case where I will be inputting audio to a model, and the output of the model will be the same audio except with certain sounds removed (volume set to zero). The dataset is generated by ...
HonestMath's user avatar
0 votes
0 answers
83 views

Autoencoder make spectrogram important parts more pronounced with a "log loss"

Hi I want to create a neural network that essentially picks out the most pronounced parts of a spectrogram. Assume this is the True spectrogram: ...
GILO's user avatar
  • 101
0 votes
2 answers
70 views

How to define a loss function for multi-label problem?

I have voice recordings which are labelled by not only a single label but multiple labels. Each voice recording corresponds to one of class labels within a set. In other words, the training instance ...
MilTom's user avatar
  • 113
0 votes
0 answers
44 views

Could I cluster the audio clips in order to improve the speed of their classification?

I have a neural network which is very resource intensive and is used to classify audio clips. The classification is done in batches, where I record for a set period of time and then go through and ...
GILO's user avatar
  • 101
1 vote
1 answer
165 views

How does using complex weights in a neural network affect performance?

If you switch a neural network from real weights to complex weights, you're roughly doubling the size of the network, and increasing the computational load by a factor of 2 to 4. My question is, in ...
chausies's user avatar
  • 150
2 votes
0 answers
282 views

Understanding gumbel-softmax backpropagation in Wav2Vec papers

I'm studying the series of Wav2Vec papers, in particular, the vq-wav2vec and wav2vec 2.0, and have a problem understanding some details about the quantization procedure. The broader context is this: ...
Peter Franek's user avatar
2 votes
2 answers
410 views

Is it realistic to train a transformer-based model (e.g. GPT) in a self-supervised way directly on the Mel spectrogram?

In music information retrieval, one usually converts an audio signal into some kind "sequence of frequency-vectors", such as STFT or Mel-spectrogram. I'm wondering if it is a good idea to ...
Peter Franek's user avatar
2 votes
0 answers
54 views

Model for direct audio-to-audio speech re-encoding

There are many resources available for text-to-audio (or vice versa) synthesis, for example Google's 'Wavenet'. These tools do not allow the finer degree of control that may be required regarding the ...
NeverWasMyRealName's user avatar
0 votes
0 answers
45 views

Speech diarization for a conversation detector: A good idea or not?

I am trying to write a program in which an ai can detect whether a conversation is occurring or not. The ai does not need to transcribe words or have any meaning about the conversation, simply if one ...
Arnav Kartikeya's user avatar
2 votes
1 answer
47 views

I want to determine how similar a given song is to Queen's songs. Am I headed in the right direction?

I've asked this question before (@ Reddit) and people suggested CNNs on a mel spectrogram more than anything else. This is great. But I'm sort of stuck at: label some music data as "queen" ...
Mike Johnson Jr's user avatar
2 votes
1 answer
734 views

How to get more accuracy of the logistic regression model?

I am working on a Baby Crying Detection model using logistic regression. Out of $581$ audios, $222$ are of a baby crying. Each audio is of $5$ seconds. what I have done is convert each audio into ...
Muhammad Waqar Anwar's user avatar
1 vote
0 answers
64 views

Is it likely that a sentient AI experience synesthesia?

The reason I ask this question is because we humans tend to compartmentalize our sensory inputs, except in some individuals that experience synesthesia. If an Artificial Intelligence Entity (AIE) can ...
Youvan's user avatar
  • 119
1 vote
0 answers
38 views

Can Machine Learning be used to synthesize engine sounds?

I'm working on a project to equip model locomotives with sound boards. I'm in the process of designing the board at the moment, and the idea is to allow users to load their own sound files onto an SD ...
David Cutting's user avatar
1 vote
1 answer
1k views

Can we use transformers for audio classification tasks?

Since transformers are good at processing sequential data, can we also use them for audio classification problems (same as RNNs)?
Isuru Sachitha's user avatar
0 votes
1 answer
210 views

How to combine specific CNN models that work better at slightly different tasks?

I'm not sure how to describe this in the most accurate way but I'll give it a shot. I've developed a Inception-Resnet V2 model for detecting audio signals via spectrogram. It does a pretty good job ...
Mecho Engineer's user avatar
1 vote
0 answers
39 views

Is there a problem for "Sound Source Identification in Video Footage"?

I've been considering starting a project for some time on sound source identification. To be more specific, my goal is to be able to identify the "sources" for sound in videos. Moving parts ...
madprogramer's user avatar
5 votes
1 answer
2k views

How can I find a specific word in an audio file?

I'm trying to train and use a neural network to detect a specific word in an audio file. The input of the neural network is an audio of 2-3 seconds duration, and the neural network must determine ...
Ali.kavari76's user avatar
1 vote
0 answers
56 views

What is the most compressed audio that I can feed an AI?

The problem I currently have is that I want to train an AI to produce music, like music that contains voices etc... However, the problem is that with a WAV file, one second of audio can be up to 48,...
KrystosTheOverlord's user avatar
1 vote
0 answers
28 views

What are the best datasets available for music information retrieval?

I am interested in doing some work in classification problems in music information retrieval. I know that there are some formats of datasets (such as MIDI, Spectrogram, Piano-roll, MusicXML, etc.) for ...
Babaji's user avatar
  • 111
2 votes
0 answers
43 views

Is there an AI that can complete Deezer Spleeter work?

I have used Deezer Spleeter but it produces echoes aside the stems, so I wonder if there is already an AI that remove echoes noises.
Mohammed Mehdi TBER's user avatar
1 vote
0 answers
84 views

Noise Cancellation on live audio stream

I want to build an application which takes a live audio from source (mic) and filtering the noise (unwanted sounds like chattering, traffic noises) and fetch into an application for further processing....
roy-bankesh's user avatar
2 votes
0 answers
586 views

How do I train a multiple-speaker model (speech synthesis) based on Tacotron 2 and espnet?

I'm new to Speech Synthesis & Deep Learning. Recently, I got a task as described below: I have problem in training a multi-speaker model which should be created by Tacotron2. And I was told I can ...
Envelo Lee's user avatar
2 votes
0 answers
42 views

State of the art in voice recognition

In the media there's lot of talk about face recognition, mainly with respect to identifying faces (= assigning to persons). Less attention is paid to the recognition of facially expressed emotions but ...
Hans-Peter Stricker's user avatar
2 votes
1 answer
513 views

How to use AI for language recognition?

Given an audio track, I'm trying to find a way to recognize the audio language. Only within a small set (e.g. English vs Spanish). Is there a simple solution to detect the language in a speech?
Mary's user avatar
  • 973
1 vote
1 answer
91 views

Deep audio fingerprinting for word search

Simply speaking, I'm trying to somehow search an audio clip for a list of words, and if found, I mark the time stamps. My use-case is profanity check with a list of pre-defined profane words. Is ...
Mary's user avatar
  • 973
3 votes
2 answers
736 views

Can AI be used to reverse engineer a black box?

A while back I posted on the Reverse Engineering site about an audio DSP system whose designer had passed away and whose manufacturer no longer had source code (but the question was deleted). ...
chmedly's user avatar
  • 131
1 vote
0 answers
66 views

Speech to text models [closed]

I'm currently using google speech to text api to transcribe audio in real time (police scanner audio dispatches). The audio quality isn't great and I've been putting in key words to try to help train ...
Bill's user avatar
  • 119
0 votes
2 answers
141 views

What are the current open source text-to-audio libraries? [closed]

I am not new to AI and did some work for few months but completely new to text to audio. Yes I used text to audio tools a decade back... but I would like to know where exactly we stand in terms of ...
Mathematics's user avatar
1 vote
1 answer
149 views

Isolate the speech of two people in an audio record with two people only

I would like to find a way to isolate the speech of each of the people in an audio record so I can create a file of that form : ...
secavfr's user avatar
  • 111
2 votes
1 answer
297 views

Can I filter barking sounds on the television? [closed]

My dog goes bonkers every time the sound of a barking dog is heard on a television program. I never noticed this before but literally every movie or show with an outdoors setting eventually includes ...
AlanD's user avatar
  • 21
9 votes
1 answer
14k views

Is it possible to clean up an audio recording of a lecture using some type of AI system?

Is it possible to clean up an audio recording of a lecture from a smartphone (i.e. remove the background noise) using some type of AI system?
Thibault Molleman's user avatar
1 vote
1 answer
4k views

How to combine input from different types of data sources?

I've to train a neural network using microphone data (wav files), accelerometer sensor data and light sensor data. Right now the approach I thought was to convert all data into images and combine ...
Aravind's user avatar
  • 113
1 vote
1 answer
128 views

Difficulty understanding Keras LSTM fitting data

I'm try to train a RNN with a chunk of audio data, where X and Y are two audio channels loaded into numpy arrays. The objective is to experiment with different NN designs to train them to transform ...
Dmitry's user avatar
  • 19