2
$\begingroup$

I'm trying to decide which one to use for my project but I can't find anywhere specific differences or comparisons of the models.

$\endgroup$
1
  • $\begingroup$ You should provide more details about your project and what you want to achieve, and why you even considered these models. $\endgroup$
    – nbro
    Commented Jun 22, 2023 at 21:56

1 Answer 1

2
$\begingroup$

From what I've seen, these two models have similar architectures since they took the architecture of SoftVC and combine it with the design of VITS. The RVC is a succession of SoVITS and it has some improvements.

Firstly, the RVC used ContentVec as the content encoder rather than HuBERT. ContentVec is an improved version of HuBERT, and it can ignore speaker information and only focus on content.

Secondly, the RVC used top1 retrieval to reduce tone leakage. It is just like the codebook used in VQ-VAE, mapping the unseen input into known input in the training dataset.

But according to the 4.1 version update in the SoVITS repo, they replaced HuBERT with ContentVec and also added feature retrieval functionality, so their performance should be the same now.

7/7 Update:

I just talked to their developers and confirmed that their architectures are nearly the same, except that in SoVITS you can select which content encoder to use (HuBERT or ContentVec). I drew a graph to illustrate the model of RVC: RVC Arch

$\endgroup$
3
  • $\begingroup$ Beautiful answer, thank you $\endgroup$
    – Hiperfly
    Commented Jul 3, 2023 at 8:04
  • $\begingroup$ when you say ContentVec, do you mean vec-768-layer-12? So is that better than hubert_base.pt? thanks! $\endgroup$ Commented Mar 18 at 10:32
  • $\begingroup$ @user17952421 I didn't find vec-768-layer-12 but I guess it's a checkpoint from ContentVec, which I would say is not "better" but rather "different". ContentVec is an effort to remove speaker information from voice. It adopts a similar model as Hubert but uses Hubert as a Teacher model and uses some voice conversion on input so that the model learned how to ignore the tone from different speakers. $\endgroup$
    – Cat ALog
    Commented Mar 20 at 11:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .