Wav2vec2 Xls R 300m Japanese

🚀 XLS-R-300M - Japanese

This model is designed to transcribe audio into Hiragana, a format of the Japanese language. It's a fine - tuned version of facebook/wav2vec2-xls-r-300m on the mozilla - foundation/common_voice_8_0 dataset, offering valuable capabilities in automatic speech recognition.

🚀 Quick Start

This model is for transcribing audio into Hiragana, one format of Japanese language.

This model is a fine - tuned version of facebook/wav2vec2-xls-r-300m on the mozilla - foundation/common_voice_8_0 dataset. Note that the following results are achieved by:

• Modify eval.py to suit the use case.
• Since kanji and katakana shares the same sound as hiragana, we convert all texts to hiragana using pykakasi and tokenize them using fugashi.

It achieves the following results on the evaluation set:

• Loss: 0.7751
• Cer: 0.2227

📚 Documentation

Evaluation results (Running ./eval.py)

Model Information

Model Index

• Name: XLS - R - 300M - Japanese
  • Results:
    • Task:
      • Name: Automatic Speech Recognition
      • Type: automatic - speech - recognition
    • Dataset:
      • Name: Common Voice 8
      • Type: mozilla - foundation/common_voice_8_0
      • Args: ja
    • Metrics:
      • Name: Test WER
      • Type: wer
      • Value: 54.05
      • Name: Test CER
      • Type: cer
      • Value: 27.54
    • Task:
      • Name: Automatic Speech Recognition
      • Type: automatic - speech - recognition
    • Dataset:
      • Name: Robust Speech Event - Dev Data
      • Type: speech - recognition - community - v2/dev_data
      • Args: ja
    • Metrics:
      • Name: Validation WER
      • Type: wer
      • Value: 48.77
      • Name: Validation CER
      • Type: cer
      • Value: 24.87
    • Task:
      • Name: Automatic Speech Recognition
      • Type: automatic - speech - recognition
    • Dataset:
      • Name: Robust Speech Event - Test Data
      • Type: speech - recognition - community - v2/eval_data
      • Args: ja
    • Metrics:
      • Name: Test CER
      • Type: cer
      • Value: 27.36

Training Procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e - 05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Framework versions

• Transformers 4.17.0.dev0
• Pytorch 1.10.2+cu102
• Datasets 1.18.2.dev0
• Tokenizers 0.11.0

📄 License

This project is licensed under the Apache - 2.0 license.