Ced Small

🚀 CED-Small Model

CED are simple ViT-Transformer-based models for audio tagging, achieving state-of-the-art performance on Audioset.

🚀 Quick Start

CED models offer a novel approach to audio tagging, leveraging the power of ViT-Transformer architectures. They are designed to provide high performance on the Audioset dataset while addressing several limitations of existing models.

✨ Features

• Simplification for finetuning: Batchnormalization of Mel - Spectrograms. This eliminates the need to compute mean/variance over the dataset during finetuning, unlike AST.
• Support for variable length inputs: Most other models use static time - frequency position embeddings, which can limit generalization to shorter segments. CED models handle variable length inputs more effectively.
• Training/Inference speedup: Utilizes 64 - dimensional mel - filterbanks and 16x16 non - overlapping patches, resulting in fewer patches compared to AST, thus speeding up training and inference.
• Performance: CED models with relatively fewer parameters (e.g., 10M) outperform many previous approaches with much larger parameter counts (~80M).

Model Performance

Model Sources

• Original Repository: CED Original
• Repository: CED on Hugging Face
• Paper: CED: Consistent ensemble distillation for audio tagging
• Demo: CED Demo

📦 Installation

pip install git+https://github.com/jimbozhang/hf_transformers_custom_model_ced.git

💻 Usage Examples

Basic Usage

>>> from ced_model.feature_extraction_ced import CedFeatureExtractor
>>> from ced_model.modeling_ced import CedForAudioClassification

>>> model_name = "mispeech/ced-small"
>>> feature_extractor = CedFeatureExtractor.from_pretrained(model_name)
>>> model = CedForAudioClassification.from_pretrained(model_name)

>>> import torchaudio
>>> audio, sampling_rate = torchaudio.load("resources/JeD5V5aaaoI_931_932.wav")
>>> assert sampling_rate == 16000
>>> inputs = feature_extractor(audio, sampling_rate=sampling_rate, return_tensors="pt")

>>> import torch
>>> with torch.no_grad():
...     logits = model(**inputs).logits

>>> predicted_class_id = torch.argmax(logits, dim=-1).item()
>>> model.config.id2label[predicted_class_id]
'Finger snapping'

Fine - tuning

You can refer to the example_finetune_esc50.ipynb notebook to learn how to train a linear head on the ESC - 50 dataset with the CED encoder frozen.

📄 License

This project is licensed under the Apache - 2.0 license.