Wav2vec En Finetuned On Cryptocurrency

🚀 Wav2Vec2 Fine-tuned on Cryptocurrency Audio

This project fine-tunes the facebook/wav2vec2-large-960h model on cryptocurrency-related audio clips, achieving better performance in automatic speech recognition for this domain.

🚀 Quick Start

We fine-tuned the facebook/wav2vec2-large-960h model using 1400 audio clips (each around 10 - 15 seconds) from various cryptocurrency-related podcasts. To label the data, we downloaded cryptocurrency podcasts from YouTube along with their subtitle data and split the clips by sentence. We then compared the YouTube transcriptions with facebook/wav2vec2-large-960h to correct many mistakes in the YouTube transcriptions. Better results may be achieved with more data cleaning.

On our dataset, we achieved a Word Error Rate (WER) of 13.1%, while the original facebook/wav2vec2-large-960h model reached a WER of 27%.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
from datasets import load_dataset
import soundfile as sf
import torch


# load model and tokenizer
processor = Wav2Vec2Processor.from_pretrained("distractedm1nd/wav2vec-en-finetuned-on-cryptocurrency")
model = Wav2Vec2ForCTC.from_pretrained("distractedm1nd/wav2vec-en-finetuned-on-cryptocurrency")

filename = "INSERT_FILENAME"
audio, sampling_rate = sf.read(filename)

input_values = processor(audio, return_tensors="pt", padding="longest", sampling_rate=sampling_rate).input_values  # Batch size 1


# retrieve logits
logits = model(input_values).logits

# take argmax and decode
predicted_ids = torch.argmax(logits, dim=-1)
tokenizer.batch_decode(predicted_ids)

📄 License

This project is licensed under the MIT license.