Bigvgan V2 44khz 128band 512x

🚀 BigVGAN: A Universal Neural Vocoder with Large-Scale Training

BigVGAN is a universal neural vocoder trained on a large scale, which can effectively generate high - quality audio.

🚀 Quick Start

BigVGAN is a powerful neural vocoder. You can quickly start using it by following the steps below. First, install the necessary dependencies, then load the pre - trained model, and finally generate the desired audio.

✨ Features

• Code Refactoring: In July 2024 (v2.3), the code was generally refactored to improve readability.
• Fused CUDA Kernel: A fully fused CUDA kernel of anti - alised activation (upsampling + activation + downsampling) was introduced with an inference speed benchmark.
• Interactive Demo: In July 2024 (v2.2), an interactive local demo using gradio was added to the repository.
• Hugging Face Integration: In July 2024 (v2.1), BigVGAN was integrated with 🤗 Hugging Face Hub, allowing easy access to inference using pretrained checkpoints, and an interactive demo was provided on Hugging Face Spaces.
• BigVGAN - v2 Release: In July 2024 (v2), BigVGAN - v2 was released with the following improvements:
  • Custom CUDA Kernel: A fused upsampling + activation kernel written in CUDA was provided for accelerated inference speed, achieving 1.5 - 3x faster speed on a single A100 GPU.
  • Improved Discriminator and Loss: It was trained using a multi - scale sub - band CQT discriminator and a multi - scale mel spectrogram loss.
  • Larger Training Data: Trained using datasets containing diverse audio types, including speech in multiple languages, environmental sounds, and instruments.
  • Pretrained Checkpoints: Pretrained checkpoints of BigVGAN - v2 with diverse audio configurations were provided, supporting up to 44 kHz sampling rate and 512x upsampling ratio.

📦 Installation

This repository contains pretrained BigVGAN checkpoints with easy access to inference and additional huggingface_hub support.

If you are interested in training the model and additional functionalities, please visit the official GitHub repository for more information: https://github.com/NVIDIA/BigVGAN

git lfs install
git clone https://huggingface.co/nvidia/bigvgan_v2_44khz_128band_512x

💻 Usage Examples

Basic Usage

device = 'cuda'

import torch
import bigvgan
import librosa
from meldataset import get_mel_spectrogram

# instantiate the model. You can optionally set use_cuda_kernel=True for faster inference.
model = bigvgan.BigVGAN.from_pretrained('nvidia/bigvgan_v2_44khz_128band_512x', use_cuda_kernel=False)

# remove weight norm in the model and set to eval mode
model.remove_weight_norm()
model = model.eval().to(device)

# load wav file and compute mel spectrogram
wav_path = '/path/to/your/audio.wav'
wav, sr = librosa.load(wav_path, sr=model.h.sampling_rate, mono=True) # wav is np.ndarray with shape [T_time] and values in [-1, 1]
wav = torch.FloatTensor(wav).unsqueeze(0) # wav is FloatTensor with shape [B(1), T_time]

# compute mel spectrogram from the ground truth audio
mel = get_mel_spectrogram(wav, model.h).to(device) # mel is FloatTensor with shape [B(1), C_mel, T_frame]

# generate waveform from mel
with torch.inference_mode():
    wav_gen = model(mel) # wav_gen is FloatTensor with shape [B(1), 1, T_time] and values in [-1, 1]
wav_gen_float = wav_gen.squeeze(0).cpu() # wav_gen is FloatTensor with shape [1, T_time]

# you can convert the generated waveform to 16 bit linear PCM
wav_gen_int16 = (wav_gen_float * 32767.0).numpy().astype('int16') # wav_gen is now np.ndarray with shape [1, T_time] and int16 dtype

Advanced Usage

import bigvgan
model = bigvgan.BigVGAN.from_pretrained('nvidia/bigvgan_v2_44khz_128band_512x', use_cuda_kernel=True)

When applied for the first time, it builds the kernel using nvcc and ninja. If the build succeeds, the kernel is saved to alias_free_activation/cuda/build and the model automatically loads the kernel. The codebase has been tested using CUDA 12.1.

Please make sure that both are installed in your system and nvcc installed in your system matches the version your PyTorch build is using.

For detail, see the official GitHub repository: https://github.com/NVIDIA/BigVGAN?tab=readme-ov-file#using-custom-cuda-kernel-for-synthesis

📚 Documentation

Pretrained Models

We provide the pretrained models on Hugging Face Collections. One can download the checkpoints of the generator weight (named bigvgan_generator.pt) and its discriminator/optimizer states (named bigvgan_discriminator_optimizer.pt) within the listed model repositories.

📄 License

This project is licensed under the MIT License. License Link