Wav2vec2 Large Xlsr 53 German Gpt2

🚀 Wav2Vec2-Large-XLSR-53-German-GPT2

This is an encoder-decoder model for automatic speech recognition, trained on the MOZILLA-FOUNDATION/COMMON_VOICE_7_0 - DE dataset, aiming to provide high - quality speech recognition capabilities.

🚀 Quick Start

This is an encoder-decoder model for automatic speech recognition trained on the MOZILLA-FOUNDATION/COMMON_VOICE_7_0 - DE dataset. The encoder was initialized from jonatasgrosman/wav2vec2-large-xlsr-53-german and the decoder from dbmdz/german-gpt2.

✨ Features

• Two - step training process:
  • Fine - tuning cross - attention and decoder: Fine - tune only the cross - attention weights and the decoder using the pre - computed outputs of the Wav2Vec model. It offers relatively fast training, can work on small GPUs (e.g., 8 GB), but may require a large amount of disk space and should already yield decent results.
  • End - to - end fine - tuning: Fine - tune the model end - to - end. It is much slower and needs a bigger GPU.
• Performance improvement trick: Adding position embeddings to the encoder outputs and initializing them with the pre - trained position embeddings of the GPT2 model (See eval.py).

📚 Documentation

Model Information

Results

Training Notes

The training notebooks are still early drafts. Also, results can probably be improved a lot by using, for example, a learning rate schedule.

📄 License

This project is licensed under the Apache - 2.0 license.