SER Odyssey Baseline WavLM Categorical

🚀 语音情感识别模型

本模型是为2024年奥德赛情感识别竞赛基线而设计，在 MSP-Podcast 数据集上进行训练。该模型基于分类方式，能够预测以下情感类别：“愤怒”、“悲伤”、“快乐”、“惊讶”、“恐惧”、“厌恶”、“轻蔑”和“中立”。

✨ 主要特性

• 基于 MSP-Podcast 数据集训练，用于2024年奥德赛情感识别竞赛基线。
• 采用分类方式，可预测多种情感类别。

📚 详细文档

基准测试

基于奥德赛竞赛的Test3和开发集的F1分数：

以下是具体的F1分数表格：

更多详细信息可参考：演示、论文 和 GitHub。

引用信息

@InProceedings{Goncalves_2024,
            author={L. Goncalves and A. N. Salman and A. {Reddy Naini} and L. Moro-Velazquez and T. Thebaud and L. {Paola Garcia} and N. Dehak and B. Sisman and C. Busso},
            title={Odyssey2024 - Speech Emotion Recognition Challenge: Dataset, Baseline Framework, and Results},
            booktitle={Odyssey 2024: The Speaker and Language Recognition Workshop)},
            volume={To appear},
            year={2024},
            month={June},
            address =  {Quebec, Canada},
}

💻 使用示例

基础用法

from transformers import AutoModelForAudioClassification
import librosa, torch

#load model
model = AutoModelForAudioClassification.from_pretrained("3loi/SER-Odyssey-Baseline-WavLM-Categorical-Attributes", trust_remote_code=True)

#get mean/std
mean = model.config.mean
std = model.config.std

#load an audio file
audio_path = "/path/to/audio.wav"
raw_wav, _ = librosa.load(audio_path, sr=model.config.sampling_rate)

#normalize the audio by mean/std
norm_wav = (raw_wav - mean) / (std+0.000001)

#generate the mask
mask = torch.ones(1, len(norm_wav))

#batch it (add dim)
wavs = torch.tensor(norm_wav).unsqueeze(0)

#predict
with torch.no_grad():
    pred = model(wavs, mask)

print(model.config.id2label)  
print(pred)
#{0: 'Angry', 1: 'Sad', 2: 'Happy', 3: 'Surprise', 4: 'Fear', 5: 'Disgust', 6: 'Contempt', 7: 'Neutral'}
#tensor([[0.0015, 0.3651, 0.0593, 0.0315, 0.0600, 0.0125, 0.0319, 0.4382]])

#convert logits to probability
probabilities = torch.nn.functional.softmax(pred, dim=1)
print(probabilities)
#[[0.0015, 0.3651, 0.0593, 0.0315, 0.0600, 0.0125, 0.0319, 0.4382]]

📄 许可证

本模型采用MIT许可证。