MT0-XXL Open-Source Multilingual Large Model - Free Deployment, Supports Cross-Lingual Task Processing in 46 Languages

Mt0 Xxl

Developed by bigscience

mt0-xxl is a multilingual large model from the BLOOMZ series, supporting cross-lingual task processing in 46 languages, fine-tuned on the xP3 dataset, and excels in zero-shot cross-lingual instruction execution

Large Language Model

Transformers

Supports Multiple LanguagesOpen Source License:Apache-2.0 #Multilingual Zero-shot Learning #Cross-lingual Task Generalization #High-precision Coreference Resolution

Downloads 1,914

Release Time : 10/19/2022

Model Overview

A multilingual large model based on the mT5 architecture, achieving cross-lingual task generalization through fine-tuning on the xP3 dataset, supporting various tasks such as text generation, translation, and question answering

Model Features

Multilingual Zero-shot Learning

Can handle instruction tasks in 46 languages without language-specific training

Cross-task Generalization

Fine-tuned on the xP3 mixed-task dataset, capable of generalizing to unseen task types

Unified Multi-task Processing

A unified framework for handling diverse tasks such as translation, question answering, and text generation

Model Capabilities

Multilingual text generation

Cross-lingual translation

Coreference resolution

Natural language inference

Sentiment analysis

Question answering system

Keyword extraction

Use Cases

Language Services

Multilingual Translation

Supports mutual translation between 46 languages (e.g., Mongolian-English)

Cross-lingual conversion with semantic accuracy

Cross-lingual Question Answering

Ask questions in any supported language and receive answers in multiple languages

Achieves 76-81% accuracy on the XWinograd test set

Content Generation

Multilingual Story Creation

Generates coherent text based on specified language and theme (e.g., Spanish fairy tales)

Creative output with cultural adaptability

🚀 Project Introduction

This project involves a set of datasets and models for text - to - text generation. It supports a wide range of languages and tasks, enabling cross - lingual generalization and zero - shot following of human instructions.

📦 Datasets

Property	Details
Datasets	- bigscience/xP3 - mc4
Languages Supported	af, am, ar, az, be, bg, bn, ca, ceb, co, cs, cy, da, de, el, en, eo, es, et, eu, fa, fi, fil, fr, fy, ga, gd, gl, gu, ha, haw, hi, hmn, ht, hu, hy, ig, is, it, iw, ja, jv, ka, kk, km, kn, ko, ku, ky, la, lb, lo, lt, lv, mg, mi, mk, ml, mn, mr, ms, mt, my, ne, nl, 'no', ny, pa, pl, ps, pt, ro, ru, sd, si, sk, sl, sm, sn, so, sq, sr, st, su, sv, sw, ta, te, tg, th, tr, uk, und, ur, uz, vi, xh, yi, yo, zh, zu
Tags	text2text - generation

Dataset Explanation

Name	Explanation	Example models
xP3	Mixture of 13 training tasks in 46 languages with English prompts	bloomz & mt0-xxl
xP3mt	Mixture of 13 training tasks in 46 languages with prompts in 20 languages (machine - translated from English)	bloomz - mt & mt0 - xxl - mt
xP3all	xP3 + our evaluation datasets adding an additional 3 tasks for a total of 16 tasks in 46 languages with English prompts
xP3megds	Megatron - DeepSpeed processed version of xP3	bloomz
P3	Repreprocessed version of the English - only P3 with 8 training tasks	bloomz - p3 & mt0 - xxl - p3

💻 Widget Examples

Example 1: en - en struct - to - text

Text:

Name	Explanation	Example models
xP3	Mixture of 13 training tasks in 46 languages with English prompts	bloomz & mt0-xxl
xP3mt	Mixture of 13 training tasks in 46 languages with prompts in 20 languages (machine - translated from English)	bloomz-mt & mt0-xxl-mt
xP3all	xP3 + our evaluation datasets adding an additional 3 tasks for a total of 16 tasks in 46 languages with English prompts
xP3megds	Megatron-DeepSpeed processed version of xP3	bloomz
P3	Repreprocessed version of the English - only P3 with 8 training tasks	bloomz-p3 & mt0-xxl-p3

Which dataset has the most tasks?

Example 2: mn - en translation

Text: Life is beautiful! Translate to Mongolian.

Example 3: jp - or - fr translation

Text: Le mot japonais «憂鬱» veut dire quoi en Odia?

... (Other examples follow the same format as above)

📚 Model Index

Model: mt0 - xxl

Task Type	Dataset	Accuracy
Coreference resolution	Winogrande XL (xl)	63.38
Coreference resolution	XWinograd (en)	81.29
Coreference resolution	XWinograd (fr)	78.31
Coreference resolution	XWinograd (jp)	78.62
Coreference resolution	XWinograd (pt)	77.95
Coreference resolution	XWinograd (ru)	76.51
Coreference resolution	XWinograd (zh)	77.38
Natural language inference	ANLI (r1)	49.5
Natural language inference	ANLI (r2)	43
Natural language inference	ANLI (r3)	46.08
Natural language inference	SuperGLUE (cb)	85.71
Natural language inference	SuperGLUE (rte)	85.56
Natural language inference	XNLI (ar)	57.91
Natural language inference	XNLI (bg)	59.88
Natural language inference	XNLI (de)	60.64
Natural language inference	XNLI (el)	59
Natural language inference	XNLI (en)	62.01
Natural language inference	XNLI (es)	60.8
Natural language inference	XNLI (fr)	59.88
Natural language inference	XNLI (hi)	57.23
Natural language inference	XNLI (ru)	58.88
Natural language inference	XNLI (sw)	55.66
Natural language inference	XNLI (th)	57.43
Natural language inference	XNLI (tr)	57.59
Natural language inference	XNLI (ur)	55.42
Natural language inference	XNLI (vi)	58.51
Natural language inference	XNLI (zh)	59.12
Sentence completion	StoryCloze (2016)	96.04
Sentence completion	SuperGLUE (copa)	93
Sentence completion	XCOPA (et)	79
Sentence completion	XCOPA (ht)	81
Sentence completion	XCOPA (id)	92
Sentence completion	XCOPA (it)	90
Sentence completion	XCOPA (qu)	... (Data incomplete in the original, continue as available)

📄 License

The project uses the apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご