How do you perform voice cloning with this webui?

[wisplite]

I was looking through the Wiki and the various readme files and none of them seem to detail how you actually do text-to-audio voice cloning with this webui. Is there a way to do this?

[gitmylo]

Text-to-audio voice cloning can be achieved with bark on the text to speech tab, using the clone option in the voice select, then you'll just optionally name and upload your audio file, your next generation will then use the voice of the target speaker.

If it's not like your speaker voice enough, you can usually still improve it with RVC after, but this will require a model of the target speaker.

[dangerweenie]

Text-to-audio voice cloning can be achieved with bark on the text to speech tab, using the clone option in the voice select, then you'll just optionally name and upload your audio file, your next generation will then use the voice of the target speaker.

If it's not like your speaker voice enough, you can usually still improve it with RVC after, but this will require a model of the target speaker.

Thanks for this info - very helpful for someone just getting started.

Do you know if there is a list of what filetype is used for what? I just downloaded a bunch of .index, .pth, .npy files from an AI voice repo, but after putting them in the models/RVC folder, they're still not showing up in audio-webui.

[wisplite]

Now I'm running into a different issue. There seems to be a potential memory leak? When cloning it loads the Hubert model and tokenizer into memory just fine, but the moment it starts extracting semantics it fills up 20gb of system memory (the rest of my 32gb of memory after bark is loaded), causing the Linux kernel to kill tons of processes to prevent a crash, including the webui.

[gitmylo]

how big is your audio file? you're supposed to use at most 15 seconds

[wisplite]

Oh, I was not aware of that, I was training it with 28 minutes of audio lmao. I assumed it would train similar to how RVC does. Thanks for all your help with this!

[gitmylo]

Yeah, it doesn't train, it actually extracts the speech and translates it so bark can continue it

[wisplite]

That explains a lot of the issues I was having. I was under the assumption it was training, but that's my bad. Again, thanks for helping with this!

[gitmylo]

Please don't advertise paid software, thanks.