Wav2vec English Speech Emotion Recognition

🚀 Speech Emotion Recognition By Fine-Tuning Wav2Vec 2.0

This model is a fine-tuned version of jonatasgrosman/wav2vec2-large-xlsr-53-english designed for the Speech Emotion Recognition (SER) task. It leverages multiple datasets for fine - tuning and demonstrates high accuracy in emotion classification.

🚀 Quick Start

Installation

pip install transformers librosa torch

Usage Example

from transformers import *
import librosa
import torch

feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained("r-f/wav2vec-english-speech-emotion-recognition")
model = Wav2Vec2ForCTC.from_pretrained("r-f/wav2vec-english-speech-emotion-recognition")

def predict_emotion(audio_path):
    audio, rate = librosa.load(audio_path, sr=16000)
    inputs = feature_extractor(audio, sampling_rate=rate, return_tensors="pt", padding=True)
    
    with torch.no_grad():
        outputs = model(inputs.input_values)
        predictions = torch.nn.functional.softmax(outputs.logits.mean(dim=1), dim=-1)  # Average over sequence length
        predicted_label = torch.argmax(predictions, dim=-1)
        emotion = model.config.id2label[predicted_label.item()]
    return emotion

emotion = predict_emotion("example_audio.wav")
print(f"Predicted emotion: {emotion}")
>> Predicted emotion: angry

✨ Features

• Multi - dataset fine - tuning: Utilizes several datasets including SAVEE, RAVDESS, and TESS for fine - tuning.
• High accuracy: Achieves an accuracy of 0.97463 on the evaluation set.
• 7 - emotion classification: Can classify 7 different emotions: angry, disgust, fear, happy, neutral, sad, and surprise.

📦 Installation

pip install transformers librosa torch

💻 Usage Examples

Basic Usage

from transformers import *
import librosa
import torch

feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained("r-f/wav2vec-english-speech-emotion-recognition")
model = Wav2Vec2ForCTC.from_pretrained("r-f/wav2vec-english-speech-emotion-recognition")

def predict_emotion(audio_path):
    audio, rate = librosa.load(audio_path, sr=16000)
    inputs = feature_extractor(audio, sampling_rate=rate, return_tensors="pt", padding=True)
    
    with torch.no_grad():
        outputs = model(inputs.input_values)
        predictions = torch.nn.functional.softmax(outputs.logits.mean(dim=1), dim=-1)  # Average over sequence length
        predicted_label = torch.argmax(predictions, dim=-1)
        emotion = model.config.id2label[predicted_label.item()]
    return emotion

emotion = predict_emotion("example_audio.wav")
print(f"Predicted emotion: {emotion}")

📚 Documentation

Datasets Used for Fine - Tuning

• Surrey Audio - Visual Expressed Emotion (SAVEE) - 480 audio files from 4 male actors
• Ryerson Audio - Visual Database of Emotional Speech and Song (RAVDESS) - 1440 audio files from 24 professional actors (12 female, 12 male)
• Toronto emotional speech set (TESS) - 2800 audio files from 2 female actors

Classification Labels

emotions = ['angry' 'disgust' 'fear' 'happy' 'neutral' 'sad' 'surprise']

Evaluation Results

• Loss: 0.104075
• Accuracy: 0.97463

🔧 Technical Details

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• eval_steps: 500
• seed: 42
• gradient_accumulation_steps: 2
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e - 08
• num_epochs: 4
• max_steps=7500
• save_steps: 1500

Training results

📄 License

This project is licensed under the Apache - 2.0 license.