Hubert Large Superb Sid

🚀 Hubert-Large for Speaker Identification

A model for speaker identification based on the Hubert architecture, ported for the SUPERB Speaker Identification task.

🚀 Quick Start

This model is a ported version for the SUPERB Speaker Identification task. The base model is hubert-large-ll60k, pretrained on 16kHz sampled speech audio. Ensure your speech input is also sampled at 16Khz when using the model.

✨ Features

• Ported from S3PRL's Hubert for the SUPERB Speaker Identification task.
• Based on the hubert-large-ll60k base model.
• Suitable for multi - class speaker identification tasks using the VoxCeleb1 dataset.

📚 Documentation

Model description

This is a ported version of S3PRL's Hubert for the SUPERB Speaker Identification task. The base model is hubert-large-ll60k, which is pretrained on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz. For more information refer to SUPERB: Speech processing Universal PERformance Benchmark.

Task and dataset description

Speaker Identification (SI) classifies each utterance for its speaker identity as a multi - class classification, where speakers are in the same predefined set for both training and testing. The widely used VoxCeleb1 dataset is adopted. For the original model's training and evaluation instructions refer to the S3PRL downstream task README.

💻 Usage Examples

Basic Usage

from datasets import load_dataset
from transformers import pipeline

dataset = load_dataset("anton-l/superb_demo", "si", split="test")

classifier = pipeline("audio-classification", model="superb/hubert-large-superb-sid")
labels = classifier(dataset[0]["file"], top_k=5)

Advanced Usage

import torch
import librosa
from datasets import load_dataset
from transformers import HubertForSequenceClassification, Wav2Vec2FeatureExtractor

def map_to_array(example):
    speech, _ = librosa.load(example["file"], sr=16000, mono=True)
    example["speech"] = speech
    return example

# load a demo dataset and read audio files
dataset = load_dataset("anton-l/superb_demo", "si", split="test")
dataset = dataset.map(map_to_array)

model = HubertForSequenceClassification.from_pretrained("superb/hubert-large-superb-sid")
feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained("superb/hubert-large-superb-sid")

# compute attention masks and normalize the waveform if needed
inputs = feature_extractor(dataset[:2]["speech"], sampling_rate=16000, padding=True, return_tensors="pt")

logits = model(**inputs).logits
predicted_ids = torch.argmax(logits, dim=-1)
labels = [model.config.id2label[_id] for _id in predicted_ids.tolist()]

📄 License

This model is licensed under the apache-2.0 license.

📊 Eval results

The evaluation metric is accuracy.

BibTeX entry and citation info

@article{yang2021superb,
  title={SUPERB: Speech processing Universal PERformance Benchmark},
  author={Yang, Shu - wen and Chi, Po - Han and Chuang, Yung - Sung and Lai, Cheng - I Jeff and Lakhotia, Kushal and Lin, Yist Y and Liu, Andy T and Shi, Jiatong and Chang, Xuankai and Lin, Guan - Ting and others},
  journal={arXiv preprint arXiv:2105.01051},
  year={2021}
}