Bigvgan V2 22khz 80band 256x

🚀 BigVGAN: A Universal Neural Vocoder with Large-Scale Training

BigVGAN is a universal neural vocoder that enables high-quality audio generation through large-scale training.

Sang-gil Lee, Wei Ping, Boris Ginsburg, Bryan Catanzaro, Sungroh Yoon

[Paper] - [Code] - [Showcase] - [Project Page] - [Weights] - [Demo]

🚀 Quick Start

This repository provides pretrained BigVGAN checkpoints, allowing for easy inference and additional huggingface_hub support. If you're interested in model training and other features, visit the official GitHub repository: https://github.com/NVIDIA/BigVGAN.

✨ Features

News

• Jul 2024 (v2.3):
  • General refactoring and code improvements for better readability.
  • A fully fused CUDA kernel for anti-aliased activation (upsampling + activation + downsampling) with an inference speed benchmark.
• Jul 2024 (v2.2): The repository now includes an interactive local demo using Gradio.
• Jul 2024 (v2.1): BigVGAN is now integrated with 🤗 Hugging Face Hub, enabling easy inference with pretrained checkpoints. An interactive demo is also available on Hugging Face Spaces.
• Jul 2024 (v2): We released BigVGAN-v2 along with pretrained checkpoints. Highlights include:
  • A custom CUDA kernel for inference: A fused upsampling + activation kernel written in CUDA for accelerated inference speed. Tests show 1.5 - 3x faster speed on a single A100 GPU.
  • An improved discriminator and loss: BigVGAN-v2 is trained using a multi-scale sub-band CQT discriminator and a multi-scale mel spectrogram loss.
  • Larger training data: BigVGAN-v2 is trained on datasets with diverse audio types, including multi-language speech, environmental sounds, and instruments.
  • We offer pretrained checkpoints of BigVGAN-v2 with diverse audio configurations, supporting up to 44 kHz sampling rate and 512x upsampling ratio.

📦 Installation

git lfs install
git clone https://huggingface.co/nvidia/bigvgan_v2_22khz_80band_256x

💻 Usage Examples

Basic Usage

The following example demonstrates how to use BigVGAN: load the pretrained BigVGAN generator from Hugging Face Hub, compute the mel spectrogram from the input waveform, and generate a synthesized waveform using the mel spectrogram as the model input.

device = 'cuda'

import torch
import bigvgan
import librosa
from meldataset import get_mel_spectrogram

# instantiate the model. You can optionally set use_cuda_kernel=True for faster inference.
model = bigvgan.BigVGAN.from_pretrained('nvidia/bigvgan_v2_22khz_80band_256x', use_cuda_kernel=False)

# remove weight norm in the model and set to eval mode
model.remove_weight_norm()
model = model.eval().to(device)

# load wav file and compute mel spectrogram
wav_path = '/path/to/your/audio.wav'
wav, sr = librosa.load(wav_path, sr=model.h.sampling_rate, mono=True) # wav is np.ndarray with shape [T_time] and values in [-1, 1]
wav = torch.FloatTensor(wav).unsqueeze(0) # wav is FloatTensor with shape [B(1), T_time]

# compute mel spectrogram from the ground truth audio
mel = get_mel_spectrogram(wav, model.h).to(device) # mel is FloatTensor with shape [B(1), C_mel, T_frame]

# generate waveform from mel
with torch.inference_mode():
    wav_gen = model(mel) # wav_gen is FloatTensor with shape [B(1), 1, T_time] and values in [-1, 1]
wav_gen_float = wav_gen.squeeze(0).cpu() # wav_gen is FloatTensor with shape [1, T_time]

# you can convert the generated waveform to 16 bit linear PCM
wav_gen_int16 = (wav_gen_float * 32767.0).numpy().astype('int16') # wav_gen is now np.ndarray with shape [1, T_time] and int16 dtype

Advanced Usage

You can apply the fast CUDA inference kernel by using the parameter use_cuda_kernel when instantiating BigVGAN:

import bigvgan
model = bigvgan.BigVGAN.from_pretrained('nvidia/bigvgan_v2_22khz_80band_256x', use_cuda_kernel=True)

When applied for the first time, it builds the kernel using nvcc and ninja. If the build succeeds, the kernel is saved to alias_free_activation/cuda/build and the model automatically loads the kernel. The codebase has been tested using CUDA 12.1.

Please ensure that both are installed in your system and that the nvcc version in your system matches the version used by your PyTorch build.

For more details, see the official GitHub repository: https://github.com/NVIDIA/BigVGAN?tab=readme-ov-file#using-custom-cuda-kernel-for-synthesis

📚 Documentation

Pretrained Models

We offer pretrained models on Hugging Face Collections. You can download the generator weight checkpoints (named bigvgan_generator.pt) and their discriminator/optimizer states (named bigvgan_discriminator_optimizer.pt) from the listed model repositories.

📄 License

This project is licensed under the MIT License.