Torchaudio Tacotron2 Kss

🚀 Torchaudio_Tacotron2_kss

This is a torchaudio Tacotron2 model, trained on the kss dataset. It enables text - to - speech conversion, offering a practical solution for related applications.

🚀 Quick Start

To get started with this model, you need to follow the installation steps and then you can use the provided code examples to perform text - to - speech tasks.

✨ Features

• Based on the torchaudio Tacotron2 model, which is well - known in the field of text - to - speech.
• Trained on the kss dataset, potentially providing high - quality speech synthesis.

📦 Installation

Requirements

First, install the necessary Python packages:

pip install torch torchaudio transformers phonemizer

You also have to install espeak-ng.

⚠️ Important Note

If you are using Windows, you need to set additional environment variables. See: https://github.com/bootphon/phonemizer/issues/44

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoModel, AutoTokenizer

repo = "Bingsu/torchaudio_tacotron2_kss"
model = AutoModel.from_pretrained(
    repo,
    trust_remote_code=True,
    revision="589d6557e8b4bb347f49de74270541063ba9c2bc"
    )
tokenizer = AutoTokenizer.from_pretrained(repo)
model.eval()

vocoder = torch.hub.load("seungwonpark/melgan:aca59909f6dd028ec808f987b154535a7ca3400c", "melgan", trust_repo=True, pretrained=False)
url = "https://huggingface.co/Bingsu/torchaudio_tacotron2_kss/resolve/main/melgan.pt"
state_dict = torch.hub.load_state_dict_from_url(url)
vocoder.load_state_dict(state_dict)

The vocoder is the same as the original seungwonpark/melgan, but the weights are on the cuda, so they are loaded separately.

text = "반갑습니다 타코트론2입니다."
inp = tokenizer(text, return_tensors="pt", return_length=True, return_attention_mask=False)

with torch.inference_mode():
    out = model(**inp)
    audio = vocoder(out[0])

import IPython.display as ipd

ipd.Audio(audio[0].numpy(), rate=22050)

You can also play an example audio:

📄 License

• Code: MIT License
• pytorch_model.bin weights: CC BY - NC - SA 4.0 (license of the kss dataset)