SER Odyssey Baseline WavLM Dominance

🚀 音频情感支配力识别模型

本模型用于音频情感支配力识别，基于MSP - Podcast数据集训练，可预测音频中情感的支配力程度，预测范围约为0到1，为Odyssey 2024情感识别竞赛提供了基线模型。

🚀 快速开始

本模型主要用于音频情感支配力的识别，通过输入音频文件，可预测其情感支配力程度。以下是使用示例：

from transformers import AutoModelForAudioClassification
import librosa, torch

#load model
model = AutoModelForAudioClassification.from_pretrained("3loi/SER-Odyssey-Baseline-WavLM-Dominance", trust_remote_code=True)

#get mean/std
mean = model.config.mean
std = model.config.std

#load an audio file
audio_path = "/path/to/audio.wav"
raw_wav, _ = librosa.load(audio_path, sr=model.config.sampling_rate)

#normalize the audio by mean/std
norm_wav = (raw_wav - mean) / (std+0.000001)

#generate the mask
mask = torch.ones(1, len(norm_wav))

#batch it (add dim)
wavs = torch.tensor(norm_wav).unsqueeze(0)

#predict
with torch.no_grad():
    pred = model(wavs, mask)

print(model.config.id2label) 
print(pred)
#{0: 'dominance'}
#tensor([[0.3670]])

✨ 主要特性

• 单任务专业模型：专门用于预测音频情感的支配力程度，预测范围约为0到1。
• 基于特定数据集训练：在[MSP - Podcast](https://ecs.utdallas.edu/research/researchlabs/msp-lab/MSP - Podcast.html)数据集上进行训练，为Odyssey 2024情感识别竞赛提供基线。

📚 详细文档

模型基准测试

基于Odyssey竞赛的Test3和开发集进行CCC（一致性相关系数）评估，结果如下：

更多详情

• [演示](https://huggingface.co/spaces/3loi/WavLM - SER - Multi - Baseline - Odyssey2024)
• [论文](https://ecs.utdallas.edu/research/researchlabs/msp - lab/publications/Goncalves_2024.pdf)
• [GitHub](https://github.com/MSP - UTD/MSP - Podcast_Challenge/tree/main)

引用信息

@InProceedings{Goncalves_2024,
            author={L. Goncalves and A. N. Salman and A. {Reddy Naini} and L. Moro - Velazquez and T. Thebaud and L. {Paola Garcia} and N. Dehak and B. Sisman and C. Busso},
            title={Odyssey2024 - Speech Emotion Recognition Challenge: Dataset, Baseline Framework, and Results},
            booktitle={Odyssey 2024: The Speaker and Language Recognition Workshop)},
            volume={To appear},
            year={2024},
            month={June},
            address =  {Quebec, Canada},
}

📄 许可证

本模型采用MIT许可证。