Wav2vec2 Large Superb Sid

🚀 Wav2Vec2-Large for Speaker Identification

This model is designed for speaker identification, leveraging the power of Wav2Vec2 technology to accurately classify speaker identities.

🚀 Quick Start

You can quickly start using this model through the Audio Classification pipeline or by using the model directly. For detailed code examples, please refer to the "Usage Examples" section below.

✨ Features

• Ported Version: This is a ported version of S3PRL's Wav2Vec2 for the SUPERB Speaker Identification task.
• Pretrained Base Model: The base model is wav2vec2-large-lv60, pretrained on 16kHz sampled speech audio.
• Applicable Dataset: Adopts the widely used VoxCeleb1 dataset for speaker identification.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

You can use the model via the Audio Classification pipeline:

from datasets import load_dataset
from transformers import pipeline

dataset = load_dataset("anton-l/superb_demo", "si", split="test")

classifier = pipeline("audio-classification", model="superb/wav2vec2-large-superb-sid")
labels = classifier(dataset[0]["file"], top_k=5)

Advanced Usage

Or use the model directly:

import torch
import librosa
from datasets import load_dataset
from transformers import Wav2Vec2ForSequenceClassification, Wav2Vec2FeatureExtractor

def map_to_array(example):
    speech, _ = librosa.load(example["file"], sr=16000, mono=True)
    example["speech"] = speech
    return example

# load a demo dataset and read audio files
dataset = load_dataset("anton-l/superb_demo", "si", split="test")
dataset = dataset.map(map_to_array)

model = Wav2Vec2ForSequenceClassification.from_pretrained("superb/wav2vec2-large-superb-sid")
feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained("superb/wav2vec2-large-superb-sid")

# compute attention masks and normalize the waveform if needed
inputs = feature_extractor(dataset[:2]["speech"], sampling_rate=16000, padding=True, return_tensors="pt")

logits = model(**inputs).logits
predicted_ids = torch.argmax(logits, dim=-1)
labels = [model.config.id2label[_id] for _id in predicted_ids.tolist()]

📚 Documentation

Model description

This is a ported version of S3PRL's Wav2Vec2 for the SUPERB Speaker Identification task.

The base model is wav2vec2-large-lv60, which is pretrained on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz.

For more information refer to SUPERB: Speech processing Universal PERformance Benchmark

Task and dataset description

Speaker Identification (SI) classifies each utterance for its speaker identity as a multi-class classification, where speakers are in the same predefined set for both training and testing. The widely used VoxCeleb1 dataset is adopted

For the original model's training and evaluation instructions refer to the S3PRL downstream task README.

🔧 Technical Details

No specific technical details are provided in the original document, so this section is skipped.

📄 License

This model is licensed under the Apache-2.0 license.

Additional Information

⚠️ Important Note

When using the model, make sure that your speech input is sampled at 16Khz.

💡 Usage Tip

You can use the Audio Classification pipeline for quick usage, or use the model directly for more customized operations.

BibTeX entry and citation info

@article{yang2021superb,
  title={SUPERB: Speech processing Universal PERformance Benchmark},
  author={Yang, Shu-wen and Chi, Po-Han and Chuang, Yung-Sung and Lai, Cheng-I Jeff and Lakhotia, Kushal and Lin, Yist Y and Liu, Andy T and Shi, Jiatong and Chang, Xuankai and Lin, Guan-Ting and others},
  journal={arXiv preprint arXiv:2105.01051},
  year={2021}
}