Wav2vec2 Animal Sounds Finetuned Hubert Finetuned Animals

🚀 hubert-finetuned-animals

A fine - tuned version of facebook/hubert-base-ls960 for animal sound classification.

🚀 Quick Start

Try the model here: Animal Sound Classification Spaces

✨ Features

This model, hubert-finetuned-animals, is a fine - tuned version of facebook/hubert-base-ls960 specifically for the task of animal sound classification. It can recognize distinct animal sounds, such as those of dogs, roosters, pigs, cows, frogs, cats, hens, insects, sheeps, and crows. This can be useful in bioacoustic monitoring, educational tools, and wildlife conservation efforts.

📦 Installation

No specific installation steps are provided in the original README.

💻 Usage Examples

Basic Usage

import librosa
import torch
from transformers import HubertForSequenceClassification, Wav2Vec2FeatureExtractor

# Load the fine-tuned model and feature extractor
model_name = "ardneebwar/wav2vec2-animal-sounds-finetuned-hubert-finetuned-animals"
feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained(model_name)
model = HubertForSequenceClassification.from_pretrained(model_name)

# Prepare the device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
model.eval()  # Set the model to evaluation mode

# Function to predict the class of an audio file
def predict_audio_class(audio_file, feature_extractor, model, device):
    # Load and preprocess the audio file
    speech, sr = librosa.load(audio_file, sr=16000)
    input_values = feature_extractor(speech, return_tensors="pt", sampling_rate=16000).input_values
    input_values = input_values.to(device)

    # Predict
    with torch.no_grad():
        logits = model(input_values).logits

    # Get the predicted class ID
    predicted_id = torch.argmax(logits, dim=-1)
    # Convert the predicted ID to the class name
    predicted_class = model.config.id2label[predicted_id.item()]
    
    return predicted_class

# Replace 'path_to_your_new_audio_file.wav' with the actual path to the new audio file
audio_file_path = "path_to_audio_file.wav"
predicted_class = predict_audio_class(audio_file_path, feature_extractor, model, device)
print(f"Predicted class: {predicted_class}")

📚 Documentation

Model description

The HuBERT model, originally trained on large amounts of unlabelled audio data, has been fine - tuned here for a downstream task of animal sound classification. This fine - tuning allows the model to specialize in recognizing distinct animal sounds, which can be particularly useful in applications such as bioacoustic monitoring, educational tools, and more interactive forms of wildlife conservation efforts.

Intended uses & limitations

This model is intended for the classification of specific animal sounds within audio clips. It can be used in software applications related to wildlife research, educational content related to animals, or for entertainment purposes where animal sound recognition is needed.

Limitations

While the model shows high accuracy, it is trained on a limited set of categories from the ESC - 50 dataset, which may not cover all possible animal sounds. The performance can vary significantly with audio quality, background noise, and animal sound variations not represented in the training data.

Training and evaluation data

The model was fine - tuned on a subset of the ESC - 50 dataset, which is a publicly available collection designed for environmental sound classification tasks. This subset specifically includes only the categories relevant to animal sounds. Each category in the dataset contains 40 examples, providing a diverse set of samples for model training and evaluation.

Training procedure

The model was fine - tuned using the following procedure:

1. Preprocessing: Audio files were converted into spectrograms.
2. Data Split: The data was split into 70% training, 20% testing sets and 10% validation sets.
3. Fine - tuning: The model was fine - tuned for 10 epochs on the training set.
4. Evaluation: The model's performance was evaluated on the validation set after each epoch to monitor improvement and prevent overfitting.

Training hyperparameters

The following hyperparameters were used during training:

Training results

Framework versions

• Transformers 4.33.2
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3

Github Repository

Animal Sound Classification

📄 License

This model is licensed under the Apache 2.0 license.