Size of training dataset, and is it possible to finetune?

[S-T-K]

Thank you for this amazing repo and for sharing your pretrained models!
I have a few questions:

• In your experience, how large should the dataset be to achieve reasonable quality?

• Can the pretrained models be fine-tuned instead of training from scratch?

• Would training a model solely on the voice audio of one specific speaker improve inference results for that speaker?

[yxlu-0102]

Thank you for your questions!

1. Based on our experience, having several to tens of hours of data is sufficient to achieve good quality with this model. However, for out-of-domain data, there might still be some mismatches during inference, especially if the data is quite different from the training set. To improve performance, we highly recommend fine-tuning the model on domain-specific data, which can significantly enhance the model’s adaptability and accuracy.

2. Sorry, the pretrained models can not be fine-tuned as we didn't provide the weights for the discriminators, so you may have to retrain the models on the VCTK dataset.

3. Using only one speaker’s data might not be enough. Why not training the model on a large dataset and fine-tuning it on the specific speaker’s data.

We hope this helps, and feel free to reach out if you have more questions!

[yukiarimo]

1. Can you show logs for all your 500k steps? I'm currently at step 1000 and generation loss (or something like that) is currently 380.
2. Can't you just convert generator into discriminator and continue like that?
3. I see. Can you please explain a bit params in the config and why specifically those where used like hop and window length (they are small), simply please!

[nahue-passano]

Hi @yxlu-0102 is there any chance you could upload the discriminator weights so that it can be fine-tuned? If not, how long did the training take and what hardware did you use (number of GPUs, GPU models, etc.)?

Thanks in advance!