FreeSVC Open-source Singing Voice Conversion Model - Free and Supports Multiple Languages, Zero-shot Cross-lingual Singing Voice Conversion

Free Svc

Developed by alefiury

FreeSVC is a multilingual zero-shot singing voice conversion model capable of cross-language voice conversion without requiring extensive training for specific languages.

Speech Synthesis Supports Multiple Languages#Zero-shot singing voice conversion #Multilingual support #Cross-language timbre preservation

Downloads 27

Release Time : 1/7/2025

Model Overview

FreeSVC employs an enhanced VITS architecture, integrating Speaker-Independent Clustering (SPIN) technology and ECAPA2 speaker encoder, effectively separating speaker characteristics from linguistic content to ensure high-quality natural timbre conversion across multiple languages.

Model Features

Multilingual Support

Supports zero-shot conversion for 11 languages including Chinese, English, Japanese and other major languages

Cross-language Conversion Capability

Enables cross-language singing voice conversion without requiring language-specific training

High-quality Timbre Preservation

Effectively separates speaker characteristics from linguistic content through SPIN technology and ECAPA2 encoder

Diverse Training Data

Trained using over 12 different datasets covering both speech and singing data

Model Capabilities

Voice conversion

Singing voice conversion

Voice cloning

Singing voice cloning

Cross-language conversion

Zero-shot learning

Use Cases

Music Creation

Multilingual Song Covers

Convert songs from one language to another while preserving the original singer's timbre characteristics

Achieves natural and smooth cross-language song conversion

Virtual Singer Timbre Customization

Create unique timbre features for virtual singers

Generates singing voices with specific timbre characteristics

Speech Technology Research

Cross-language Voice Conversion Research

Used to study voice feature conversion between different languages

Provides a research platform for voice conversion technology

🚀 FreeSVC: Zero-shot Multilingual Singing Voice Conversion

FreeSVC is a promising multilingual zero-shot singing voice conversion model. It enables the conversion of singing voices across languages without the need for extensive language-specific training. Check out its GitHub repository and Paper arXiv pre-print.

🚀 Quick Start

FreeSVC offers a seamless way to perform multilingual zero - shot singing voice conversion. You can explore the model through its GitHub repository.

✨ Features

Multilingual Support: FreeSVC supports multiple languages, allowing for cross - language singing voice conversion.
Zero - shot Conversion: It can convert singing voices without extensive language - specific training.

📦 Supported Languages

Language	ID	Status	Speech Data	Singing Data
Chinese	0	✅ Full	255h	70h
Dutch	1	✅ Full	Part of CML	-
English	2	✅ Full	921h	47h
French	3	✅ Full	Part of CML	-
German	4	✅ Full	Part of CML	-
Italian	5	✅ Full	Part of CML	-
Japanese	6	✅ Full	30h	-
Other*	7	⚠️ Partial	-	10h
Polish	8	✅ Full	Part of CML	-
Portuguese	9	✅ Full	Part of CML	-
Spanish	10	✅ Full	Part of CML	-

*Note: The "Other" category is used for vocal techniques without content.

🔧 Technical Details

Model Overview

FreeSVC leverages an enhanced VITS architecture integrated with Speaker - invariant Clustering (SPIN) and the ECAPA2 speaker encoder. This combination effectively separates speaker characteristics from linguistic content, ensuring high - quality and natural - sounding voice conversions across multiple languages.

📚 Training Datasets

FreeSVC was trained on a diverse set of speech and singing datasets covering multiple languages:

Property	Details
Dataset	AISHELL - 1, AISHELL - 3, CML - TTS, HiFiTTS, JVS, LibriTTS - R, NUS (NHSS), OpenSinger, Opencpop, PopBuTFy, POPCS, VCTK, VocalSet
Hours	170h (AISHELL - 1), 85h (AISHELL - 3), 3.1k (CML - TTS), 292h (HiFiTTS), 30h (JVS), 585h (LibriTTS - R), 7h (NUS (NHSS)), 50h (OpenSinger), 5h (Opencpop), 10h, 40h (PopBuTFy), 5h (POPCS), 44h (VCTK), 10h (VocalSet)
Language	Chinese, English, Japanese, 7 Languages (CML - TTS), Other
Type	Speech, Singing

📄 License

FreeSVC is released under the Creative Commons Attribution - NonCommercial - ShareAlike 4.0 International (CC BY - NC - SA 4.0) license. This means:

⚠️ Important Note

The model can only be used for research and non - commercial purposes. Any commercial use is strictly prohibited.

Any derivative works must be shared under the same license.

Proper attribution must be given when using the model.

Users must also comply with the licenses of the original datasets used for training. Some datasets may have additional restrictions beyond CC BY - NC - SA 4.0. Ensure you review and adhere to their terms before using the model.

For full details, refer to the [CC BY - NC - SA 4.0 License](https://creativecommons.org/licenses/by - nc - sa/4.0/).

📚 Citation

@INPROCEEDINGS{10890068,
  author={Ferreira, Alef Iury and Gris, Lucas Rafael and Da Rosa, Augusto and Oliveira, Frederico and Casanova, Edresson and Sousa, Rafael and Junior, Arnaldo and Soares, Anderson and Filho, Arlindo Galvão},
  booktitle={ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, 
  title={FreeSVC: Towards Zero - shot Multilingual Singing Voice Conversion}, 
  year={2025},
  volume={},
  number={},
  pages={1 - 5},
  keywords={Training;Source coding;Zero shot learning;Refining;Signal processing;Data models;Acoustics;Multilingual;Data mining;Speech synthesis;Singing Voice Conversion;Synthesis of Singing Voices;Cross - lingual and multilingual aspects in speech synthesis},
  doi={10.1109/ICASSP49660.2025.10890068}
}

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご