Ast Finetuned Model

🚀 AST Fine-Tuned Model for Emotion Classification

This is a fine - tuned Audio Spectrogram Transformer (AST) model designed for classifying emotions in speech audio. It was fine - tuned on the CREMA - D dataset, focusing on six emotional categories. The base model is from MIT's pre - trained AST model.

🚀 Quick Start

To use this model for emotion classification, you can follow these steps:

from transformers import AutoModelForAudioClassification, AutoProcessor

# Load the model and processor
model = AutoModelForAudioClassification.from_pretrained("forwarder1121/ast-finetuned-model")
processor = AutoProcessor.from_pretrained("forwarder1121/ast-finetuned-model")

# Prepare input audio (e.g., waveform) as log-mel spectrogram
inputs = processor("path_to_audio.wav", sampling_rate=16000, return_tensors="pt")

# Make predictions
outputs = model(**inputs)
predicted_class = outputs.logits.argmax(-1).item()

print(f"Predicted emotion: {model.config.id2label[str(predicted_class)]}")

✨ Features

• Emotion Classification: Specifically designed to classify six different emotions in speech audio.
• Fine - Tuned on CREMA - D: Trained on a well - known emotion - labeled speech dataset.
• Based on AST Architecture: Utilizes the Audio Spectrogram Transformer architecture.

📦 Installation

Since this model is based on the transformers library, you can install it using the following command:

pip install transformers

📚 Documentation

Model Details

Model Configuration

Training Details

Metrics

Validation Results

• Best Validation Accuracy: 60.71%
• Validation Loss: 1.1126

Evaluation Details

• Eval Dataset: CREMA - D test split
• Batch Size: 16
• Number of Steps: 94

🔧 Technical Details

The model is fine - tuned on the CREMA - D dataset using data augmentation techniques such as noise injection, time shifting, and speed perturbation. It uses the Adam optimizer with a learning rate of 1e - 4 and a linear decay learning rate scheduler. The model architecture is an Audio Spectrogram Transformer (AST) with specific configuration parameters like a hidden size of 768, 12 attention heads, and 12 hidden layers.

📄 License

The model is shared under the MIT License. Refer to the licensing details in the repository.

📖 Citation

If you use this model in your work, please cite:

@misc{ast-finetuned-model,
  author = {forwarder1121},
  title = {Fine-Tuned Audio Spectrogram Transformer for Emotion Classification},
  year = {2024},
  url = {https://huggingface.co/forwarder1121/ast-finetuned-model},
}

📞 Contact

For questions, reach out to forwarder1121@naver.com.