đ Hubert-Base for Keyword Spotting
This is a ported model for keyword spotting, based on the Hubert architecture and trained on the SUPERB dataset.
đ Quick Start
This model is designed for keyword spotting. It's based on the hubert-base-ls960
model, pretrained on 16kHz sampled speech audio. Ensure your speech input is also sampled at 16kHz when using this model.
⨠Features
đĻ Installation
No specific installation steps are provided in the original README. If you want to use this model, you need to have the datasets
and transformers
libraries installed. You can install them using the following commands:
pip install datasets transformers
đģ Usage Examples
Basic Usage
You can use the model via the Audio Classification pipeline:
from datasets import load_dataset
from transformers import pipeline
dataset = load_dataset("anton-l/superb_demo", "ks", split="test")
classifier = pipeline("audio-classification", model="superb/hubert-base-superb-ks")
labels = classifier(dataset[0]["file"], top_k=5)
Advanced Usage
Or use the model directly:
import torch
from datasets import load_dataset
from transformers import HubertForSequenceClassification, Wav2Vec2FeatureExtractor
from torchaudio.sox_effects import apply_effects_file
effects = [["channels", "1"], ["rate", "16000"], ["gain", "-3.0"]]
def map_to_array(example):
speech, _ = apply_effects_file(example["file"], effects)
example["speech"] = speech.squeeze(0).numpy()
return example
dataset = load_dataset("anton-l/superb_demo", "ks", split="test")
dataset = dataset.map(map_to_array)
model = HubertForSequenceClassification.from_pretrained("superb/hubert-base-superb-ks")
feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained("superb/hubert-base-superb-ks")
inputs = feature_extractor(dataset[:4]["speech"], sampling_rate=16000, padding=True, return_tensors="pt")
logits = model(**inputs).logits
predicted_ids = torch.argmax(logits, dim=-1)
labels = [model.config.id2label[_id] for _id in predicted_ids.tolist()]
đ Documentation
Model description
This is a ported version of S3PRL's Hubert for the SUPERB Keyword Spotting task. The base model is hubert-base-ls960, which is pretrained on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz. For more information refer to SUPERB: Speech processing Universal PERformance Benchmark
Task and dataset description
Keyword Spotting (KS) detects preregistered keywords by classifying utterances into a predefined set of words. The task is usually performed on - device for the fast response time. Thus, accuracy, model size, and inference time are all crucial. SUPERB uses the widely used Speech Commands dataset v1.0 for the task. The dataset consists of ten classes of keywords, a class for silence, and an unknown class to include the false positive. For the original model's training and evaluation instructions refer to the S3PRL downstream task README.
đ§ Technical Details
The model is based on the Hubert architecture. The base model hubert-base-ls960
is pretrained on large - scale speech data. For the keyword spotting task, it classifies utterances into a set of predefined keywords. The task is optimized for on - device performance, considering factors like accuracy, model size, and inference time.
đ License
This model is licensed under the Apache 2.0 license.
BibTeX entry and citation info
@article{yang2021superb,
title={SUPERB: Speech processing Universal PERformance Benchmark},
author={Yang, Shu - wen and Chi, Po - Han and Chuang, Yung - Sung and Lai, Cheng - I Jeff and Lakhotia, Kushal and Lin, Yist Y and Liu, Andy T and Shi, Jiatong and Chang, Xuankai and Lin, Guan - Ting and others},
journal={arXiv preprint arXiv:2105.01051},
year={2021}
}
đ Eval results
The evaluation metric is accuracy.
|
s3prl |
transformers |
test |
0.9630 |
0.9672 |