🚀 BigVGAN: A Universal Neural Vocoder with Large-Scale Training
BigVGAN is a universal neural vocoder trained on a large scale, which can effectively generate high - quality synthesized waveforms.
Sang - gil Lee, Wei Ping, Boris Ginsburg, Bryan Catanzaro, Sungroh Yoon
[Paper] - [Code] - [[Showcase]](https://bigvgan - demo.github.io/) - [Project Page] - [[Weights]](https://huggingface.co/collections/nvidia/bigvgan - 66959df3d97fd7d98d97dc9a) - [Demo]
[
](https://paperswithcode.com/sota/speech - synthesis - on - libritts?p=bigvgan - a - universal - neural - vocoder - with - large)
🚀 Quick Start
This README provides information on the installation, usage, and pre - trained models of BigVGAN. You can quickly start using BigVGAN by following the steps below.
✨ Features
- General Refactoring and Code Improvement: In July 2024 (v2.3), the code was refactored and improved for better readability.
- Interactive Local Demo: In July 2024 (v2.2), an interactive local demo using gradio was added to the repository.
- Integration with Hugging Face Hub: In July 2024 (v2.1), BigVGAN was integrated with Hugging Face Hub, making it easy to use pre - trained checkpoints for inference. An interactive demo is also provided on Hugging Face Spaces.
- Release of BigVGAN - v2: In July 2024 (v2), BigVGAN - v2 was released with pre - trained checkpoints. It includes a custom CUDA kernel for faster inference, an improved discriminator and loss, and is trained on larger and more diverse training data.
📦 Installation
This repository contains pretrained BigVGAN checkpoints with easy access to inference and additional huggingface_hub
support.
If you are interested in training the model and additional functionalities, please visit the official GitHub repository for more information: https://github.com/NVIDIA/BigVGAN
git lfs install
git clone https://huggingface.co/nvidia/bigvgan_22khz_80band
💻 Usage Examples
Basic Usage
The following example shows how to use BigVGAN: load the pre - trained BigVGAN generator from Hugging Face Hub, compute the mel spectrogram from the input waveform, and generate the synthesized waveform using the mel spectrogram as the model's input.
device = 'cuda'
import torch
import bigvgan
import librosa
from meldataset import get_mel_spectrogram
model = bigvgan.BigVGAN.from_pretrained('nvidia/bigvgan_22khz_80band', use_cuda_kernel=False)
model.remove_weight_norm()
model = model.eval().to(device)
wav_path = '/path/to/your/audio.wav'
wav, sr = librosa.load(wav_path, sr=model.h.sampling_rate, mono=True)
wav = torch.FloatTensor(wav).unsqueeze(0)
mel = get_mel_spectrogram(wav, model.h).to(device)
with torch.inference_mode():
wav_gen = model(mel)
wav_gen_float = wav_gen.squeeze(0).cpu()
wav_gen_int16 = (wav_gen_float * 32767.0).numpy().astype('int16')
Advanced Usage
You can apply the fast CUDA inference kernel by using a parameter use_cuda_kernel
when instantiating BigVGAN:
import bigvgan
model = bigvgan.BigVGAN.from_pretrained('nvidia/bigvgan_22khz_80band', use_cuda_kernel=True)
When applied for the first time, it builds the kernel using nvcc
and ninja
. If the build succeeds, the kernel is saved to alias_free_activation/cuda/build
and the model automatically loads the kernel. The codebase has been tested using CUDA 12.1
.
Please make sure that both are installed in your system and nvcc
installed in your system matches the version your PyTorch build is using.
For detail, see the official GitHub repository: https://github.com/NVIDIA/BigVGAN?tab=readme - ov - file#using - custom - cuda - kernel - for - synthesis
📚 Documentation
Pretrained Models
We provide the [pretrained models on Hugging Face Collections](https://huggingface.co/collections/nvidia/bigvgan - 66959df3d97fd7d98d97dc9a). One can download the checkpoints of the generator weight (named bigvgan_generator.pt
) and its discriminator/optimizer states (named bigvgan_discriminator_optimizer.pt
) within the listed model repositories.
Property |
Details |
Model Name |
bigvgan_v2_44khz_128band_512x, bigvgan_v2_44khz_128band_256x, etc. |
Sampling Rate |
44 kHz, 24 kHz, 22 kHz |
Mel band |
128, 100, 80 |
fmax |
22050, 12000, 8000 |
Upsampling Ratio |
512, 256 |
Params |
122M, 112M, 14M |
Dataset |
Large - scale Compilation, LibriTTS, LibriTTS + VCTK + LJSpeech |
Steps |
3M, 5M |
Fine - Tuned |
No |
📄 License
This project is licensed under the MIT license. License Link