Wav2vec2 2 Bart Large

🚀 wav2vec2-2-bart-large

This model is a fine - tuned version of pre - trained models, aiming to solve the problem of automatic speech recognition. By leveraging the power of facebook/wav2vec2-large-lv60 and bart-large, it provides high - quality speech recognition capabilities on the librispeech_asr - clean dataset.

🚀 Quick Start

To rerun this experiment, please clone this directory and run the following commands:

python create_model.py

Then, run:

./run_librispeech.sh

✨ Features

This model is a fine - tuned version of facebook/wav2vec2-large-lv60 and bart-large on the librispeech_asr - clean dataset. It achieves the following results on the evaluation set:

• Loss: 0.3204
• Wer: 0.0486

🔧 Technical Details

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• gradient_accumulation_steps: 2
• seed: 42
• distributed_type: multi - GPU
• num_devices: 8
• total_train_batch_size: 64
• total_eval_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e - 08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 5
• mixed_precision_training: Native AMP

Training results

See Training Metrics Tab.

Framework versions

• Transformers 4.15.0.dev0
• Pytorch 1.9.0+cu111
• Datasets 1.16.2.dev0
• Tokenizers 0.10.3

Model Information

Interactive Widgets

You can try out the model with the following audio samples:

• Librispeech sample 1
• Librispeech sample 2
• Common Voice sample