Muril-adapted-local Open-source Model - A Free Tool Supporting Multilingual Representations in 17 Indian Languages

Muril Adapted Local

Developed by monsoon-nlp

MuRIL is an open-source BERT model pretrained on 17 Indian languages and their transliterated versions by Google, supporting multilingual representations.

Large Language Model Supports Multiple LanguagesOpen Source License:Apache-2.0 #Indian Multilingual Processing #Whole Word Masking Pretraining #17 Language Support

Downloads 24

Release Time : 3/2/2022

Model Overview

This model is trained through self-supervised masked language modeling tasks using a whole word masking strategy, predicting up to 80 words. Suitable for handling text tasks in multiple Indian languages.

Model Features

Multilingual Support

Supports 17 Indian languages and their transliterated versions, covering a wide range of linguistic needs.

Whole Word Masking Strategy

Trained using a whole word masking strategy, predicting up to 80 words to enhance model comprehension.

High Compatibility

Converted to PyTorch format to ensure compatibility with the HuggingFace Transformers library.

Model Capabilities

Text Understanding

Multilingual Text Processing

Masked Language Modeling

Use Cases

Natural Language Processing

Multilingual Text Classification

Perform classification tasks on texts in multiple Indian languages.

Translation Assistance

Serves as a pretraining base for translation models to improve translation quality.

🚀 MuRIL - Unofficial

Multilingual Representations for Indian Languages: Google open-sourced this BERT model pre-trained on 17 Indian languages and their transliterated counterparts.

🚀 Quick Start

MuRIL is a BERT model pre - trained on 17 Indian languages and their transliterated versions. It was trained using a self - supervised masked language modeling task. The model uses whole word masking with a maximum of 80 predictions. It was trained for 1000K steps, with a batch size of 4096, and a max sequence length of 512.

✨ Features

Multilingual Support: Supports languages like English (en), Hindi (hi), Bengali (bn), Tamil (ta), Assamese (as), Gujarati (gu), Kannada (kn), Kashmiri (ks), Malayalam (ml), Marathi (mr), Nepali (ne), Oriya (or), Punjabi (pa), Sanskrit (sa), Sindhi (sd), Telugu (te), Urdu (ur), and multilingual scenarios.
Self - supervised Training: Trained with a masked language modeling task.
Whole Word Masking: Performs whole word masking with a limit of 80 predictions.

🔧 Technical Details

The model was trained using a self - supervised masked language modeling task. Whole word masking is applied with a maximum of 80 predictions. The training was carried out for 1000K steps, with a batch size of 4096 and a maximum sequence length of 512.

📚 Documentation

Original TFHub Model: You can access the original model on TFHub at https://tfhub.dev/google/MuRIL/1.
Official HuggingFace Release: The official release on HuggingFace (March 2021) can be found at https://huggingface.co/google/muril-base-cased.

📄 License

This project is licensed under the Apache 2.0 license.

About this upload

I ported the TFHub .pb model to .h5 and then pytorch_model.bin for compatibility with Transformers.

Property	Details
Model Type	MuRIL (Multilingual Representations for Indian Languages)
Training Data	17 Indian languages and their transliterated counterparts
Training Task	Self - supervised masked language modeling
Training Steps	1000K
Batch Size	4096
Max Sequence Length	512

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご