0
$\begingroup$

To create language flashcards I would like to split an audio course into many single audio clips. They're basically a man and a woman speaking after each other. The intervals aren't regular so it's not possible to split it after time intervals. Furthermore silence detection is not possible since some sentences also include pauses. I have already tried diarization using LIUM but the timings were completely wrong. Additionally the audio course includes a transcript which would certainly be machine-readable consisting of the English sentence and the Japanese sentence as well as its Romaji version (Japanese words using English letters).

I'm not experienced in AI, so I'm looking for a solution which isn't to difficult (like constructing and training my own neural network). I have some programming experience, so a mathematical approach would be fine.

Links:

audio course: http://www.japaneseaudiolessons.com/download-japanese-lessons/

LIUM: http://lium3.univ-lemans.fr/diarization/doku.php/welcome

$\endgroup$
  • $\begingroup$ If this is for a specific task, you'll be much better off adapting it for and using something like Mechanical Turk to get it done rather than trying to assemble a reliable fully-automated solution $\endgroup$ – antlersoft Mar 14 '17 at 22:28
  • 1
    $\begingroup$ Yeah, given that you will listen to these sentences many times anyway, you are probably best off just going through them once and cutting them by hand. $\endgroup$ – BlindKungFuMaster Mar 15 '17 at 11:45
  • $\begingroup$ I've considered this already, but splitting 45 hours of audio certainly is a full time job, so I'm looking for an automatic solution. $\endgroup$ – yxyx136 Mar 15 '17 at 12:50
1
$\begingroup$

Edit: It's not clear what exactly you're trying to accomplish... My answer assumes you wanted to split the man and the woman's audio, but re-reading your question make me think otherwise.

Note: Don't expect this to be a perfect answer. I'm not an expert in the field, just an interested student. I can't comment on this site yet, so I'm submitting this as an answer.

This sounds less like a problem suited for machine learning and more of a... statistical analysis(terminology?) problem. Why? You have an extremely small sample space to train on, and the natural cost function would require you to go through the entire recording and manually classify the track anyway.

Your problem is almost well suited for an Independent Component Analysis approach, except for the fact that you only have one audio source. ICA requires one source for each feature you want to extract. It might be a good start though to familiarize yourself with the techniques and terminology. See the Wikipedia page for ICA, an article called "ICA for dummies" by Arnaud Delorme, and this paper: ICA: a tutorial. I would link the first two, but apparently I need more site rep.

I also found a paper which seems to perfectly match your problem, from Mitsubishi Electrical Research Labs: Separation of Mixed Sources by Independent Subspace Analysis. It's a bit a of a heavy read, to say the least, but that might be your best shot. I'll quote the abstract, emphasis mine:

We propose the method of independent subspace analysis (ISA) for separating individual audio sources from a single-channel mixture. ISA is based on independent component analysis (ICA) but relaxes the constraint that requires at least as many mixture observation signals as sources. A second extension to ICA is the use of dynamic components to represent non-stationary signals. Sources are tracked by similarity of dynamic components over small time steps. We propose a method for grouping components by partitioning a matrix of independent component crossentropies that we call an ixegram. The ixegram measures the mutual similarities of components in an audio segment and clustering the ixegram yields the source subspaces and time trajectories. To demonstrate the techniques we give examples of ISA applied to separation of musical and speech sources from single-channel mixtures.

Good luck!

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you very much for your detailed response! Unfortunately that's currently beyond my level of understanding. I'm just looking for a ready-to-use approach $\endgroup$ – yxyx136 Apr 7 '17 at 14:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.