Fastspeech2 Conformer With Hifigan

🚀 FastSpeech2ConformerWithHifiGan

This model combines FastSpeech2Conformer and FastSpeech2ConformerHifiGan into one, offering a simpler and more convenient way to generate high - quality speech from text.

🚀 Quick Start

This model integrates FastSpeech2Conformer and FastSpeech2ConformerHifiGan. FastSpeech2Conformer is a non - autoregressive text - to - speech (TTS) model. It combines the advantages of FastSpeech2 and the conformer architecture, enabling fast and efficient generation of high - quality speech from text. The HiFi - GAN vocoder is then used to convert the generated mel - spectrograms into speech waveforms.

📦 Installation

You can run FastSpeech2Conformer locally with the 🤗 Transformers library. First, you need to install the necessary libraries:

pip install --upgrade pip
pip install --upgrade transformers g2p-en

💻 Usage Examples

Basic Usage

from transformers import FastSpeech2ConformerTokenizer, FastSpeech2ConformerWithHifiGan
import soundfile as sf

tokenizer = FastSpeech2ConformerTokenizer.from_pretrained("espnet/fastspeech2_conformer")
inputs = tokenizer("Hello, my dog is cute.", return_tensors="pt")
input_ids = inputs["input_ids"]

model = FastSpeech2ConformerWithHifiGan.from_pretrained("espnet/fastspeech2_conformer_with_hifigan")
output_dict = model(input_ids, return_dict=True)
waveform = output_dict["waveform"]

sf.write("speech.wav", waveform.squeeze().detach().numpy(), samplerate=22050)

📄 License

This project is licensed under the Apache - 2.0 license.