Xlm Roberta Comet Small
mMiniLM-L12xH384 XLM-R is a lightweight multilingual pre-trained model based on the MiniLMv2 architecture, compressed from the traditional XLM-RoBERTa model through relational distillation technology.
Downloads 45
Release Time : 3/2/2022
Model Overview
This model is a lightweight version of XLM-RoBERTa, compressed via multi-head self-attention relational distillation, suitable for multilingual natural language processing tasks.
Model Features
Lightweight Design
More lightweight compared to traditional XLM-RoBERTa base and large versions, suitable for resource-constrained environments
Multilingual Support
Based on the XLM-R architecture, it possesses strong multilingual processing capabilities
Relational Distillation Technology
Utilizes MiniLMv2's multi-head self-attention relational distillation method to reduce model size while maintaining performance
Machine Translation Optimization
Fine-tuned using WMT direct evaluation annotated data, making it particularly suitable for machine translation-related tasks
Model Capabilities
Multilingual Text Understanding
Cross-Lingual Representation Learning
Machine Translation Evaluation
Text Classification
Semantic Similarity Calculation
Use Cases
Machine Translation
Translation Quality Evaluation
Uses the fine-tuned model to evaluate the quality of machine translation results
Performs well on WMT datasets
Multilingual Pre-training
Serves as a lightweight multilingual pre-trained model for downstream tasks
Cross-Lingual Applications
Cross-Lingual Information Retrieval
Used for similarity calculation and information retrieval between documents in different languages
Multilingual Text Classification
Supports text classification tasks in multiple languages
Featured Recommended AI Models