XLSR-WithLM-Malayalam Open-Source Model - Easily Achieve Automatic Speech Recognition for Malayalam

XLSR WithLM Malayalam

Developed by kavyamanohar

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the IMaSC, Indic TTS Malayalam, and OpenSLR Malayalam training datasets, supporting automatic speech recognition for Malayalam.

Speech Recognition

Transformers

Open Source License:Apache-2.0 #Malayalam speech recognition #Low-resource language optimization #KENLM language model enhancement

Downloads 19

Release Time : 7/22/2024

Model Overview

This is an optimized automatic speech recognition model for Malayalam, enhanced with a trigram language model trained using the KENLM library, demonstrating excellent performance across multiple Malayalam datasets.

Model Features

Multi-dataset fine-tuning

Fine-tuned on multiple Malayalam datasets including IMaSC, Indic TTS Malayalam, and OpenSLR Malayalam, improving recognition accuracy.

Language model enhancement

Post-processed with a trigram language model trained on the ml-sentences dataset using the KENLM library, significantly improving recognition performance.

Efficient training

Utilized techniques such as gradient accumulation and mixed-precision training to achieve efficient training with limited resources.

Model Capabilities

Malayalam speech recognition

Speech-to-text

Use Cases

Speech transcription

Malayalam speech transcription

Convert Malayalam speech content into text

Achieved a WER of 27.3 on the OpenSLR Malayalam test set

Voice assistants

Malayalam voice assistant

Used to build voice assistant applications supporting Malayalam

🚀 XLSR-WithLM-Malayalam

This model is a fine - tuned version of facebook/wav2vec2-xls-r-300m on the IMASC, Indic TTS Malayalam, OpenSLR Malayalam Train split datasets. It aims to provide high - quality automatic speech recognition for Malayalam language, achieving significant performance improvements on relevant evaluation sets.

✨ Features

Fine - tuned from a well - known base model for better adaptability to Malayalam speech.
Utilizes a trigram language model trained on a specific dataset to enhance recognition accuracy.

📚 Documentation

Evaluation Results

It achieves the following results on the evaluation set:

Loss: 0.1395
Wer: 0.2952

Trigram Language Model

A trigram language model was trained using the KENLM Library on the [kavyamanohar/ml - sentences](https://huggingface.co/datasets/kavyamanohar/ml - sentences) dataset.

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.00024
train_batch_size: 1
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 4
optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 800
num_epochs: 1
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1.4912	0.1165	1000	0.5497	0.7011
0.5377	0.2330	2000	0.3292	0.5364
0.4343	0.3494	3000	0.2475	0.4424
0.3678	0.4659	4000	0.2145	0.4014
0.3345	0.5824	5000	0.1898	0.3774
0.3029	0.6989	6000	0.1718	0.3441
0.2685	0.8153	7000	0.1517	0.3135
0.2385	0.9318	8000	0.1395	0.2952

Framework versions

Transformers 4.42.4
Pytorch 2.3.1+cu121
Datasets 2.20.0
Tokenizers 0.19.1

📄 License

This project is licensed under the Apache - 2.0 license.

🔧 Technical Details

Model Information

Property	Details
Model Type	Fine - tuned version of facebook/wav2vec2 - xls - r - 300m
Training Data	IMASC, Indic TTS Malayalam, OpenSLR Malayalam Train split, [kavyamanohar/ml - sentences](https://huggingface.co/datasets/kavyamanohar/ml - sentences)

Model Performance on Different Datasets

Task	Dataset	WER
Automatic Speech Recognition	OpenSLR Malayalam - Test (vrclc/openslr63, ml split)	27.3
Automatic Speech Recognition	Goole Fleurs (google/fleurs, ml split)	37.2
Automatic Speech Recognition	MSC (smcproject/msc, ml split)	52.9

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご