🚀 九种语言的多语言毒性分类器 (2024)
本项目是基于 [xlm - roberta - large](https://huggingface.co/FacebookAI/xlm - roberta - large) 模型微调得到的，针对二元毒性分类任务，在我们整理的数据集 textdetox/multilingual_toxicity_dataset 上进行了训练。该项目能够有效识别九种语言文本中的毒性内容，为多语言环境下的文本安全提供了有力支持。
📦 模型信息
| 属性 |
详情 |
| 许可证 |
OpenRail++ |
| 训练数据集 |
textdetox/multilingual_toxicity_dataset |
| 支持语言 |
英语、俄语、乌克兰语、西班牙语、德语、阿姆哈拉语、阿拉伯语、中文、印地语 |
| 评估指标 |
F1 |
| 基础模型 |
FacebookAI/xlm - roberta - large |
| 标签 |
毒性 |
| 新版本 |
textdetox/xlmr - large - toxicity - classifier - v2 |
📊 模型训练与评估
首先，我们划分出 20% 的平衡测试集来评估模型的适用性。然后，在全量数据上对模型进行微调。以下是模型在测试集上的评估结果：
| 语言 |
精确率 |
召回率 |
F1值 |
| 所有语言 |
0.8713 |
0.8710 |
0.8710 |
| 英语 |
0.9650 |
0.9650 |
0.9650 |
| 俄语 |
0.9791 |
0.9790 |
0.9790 |
| 乌克兰语 |
0.9267 |
0.9250 |
0.9251 |
| 德语 |
0.8791 |
0.8760 |
0.8758 |
| 西班牙语 |
0.8700 |
0.8700 |
0.8700 |
| 阿拉伯语 |
0.7787 |
0.7780 |
0.7780 |
| 阿姆哈拉语 |
0.7781 |
0.7780 |
0.7780 |
| 印地语 |
0.9360 |
0.9360 |
0.9360 |
| 中文 |
0.7318 |
0.7320 |
0.7315 |
📖 引用信息
如果您想引用我们的工作，请参考以下文献：
@inproceedings{dementieva2024overview,
title={Overview of the Multilingual Text Detoxification Task at PAN 2024},
author={Dementieva, Daryna and Moskovskiy, Daniil and Babakov, Nikolay and Ayele, Abinew Ali and Rizwan, Naquee and Schneider, Frolian and Wang, Xintog and Yimam, Seid Muhie and Ustalov, Dmitry and Stakovskii, Elisei and Smirnova, Alisa and Elnagar, Ashraf and Mukherjee, Animesh and Panchenko, Alexander},
booktitle={Working Notes of CLEF 2024 - Conference and Labs of the Evaluation Forum},
editor={Guglielmo Faggioli and Nicola Ferro and Petra Galu{\v{s}}{\v{c}}{\'a}kov{\'a} and Alba Garc{\'i}a Seco de Herrera},
year={2024},
organization={CEUR-WS.org}
}
@inproceedings{DBLP:conf/ecir/BevendorffCCDEFFKMMPPRRSSSTUWZ24,
author = {Janek Bevendorff and
Xavier Bonet Casals and
Berta Chulvi and
Daryna Dementieva and
Ashaf Elnagar and
Dayne Freitag and
Maik Fr{\"{o}}be and
Damir Korencic and
Maximilian Mayerl and
Animesh Mukherjee and
Alexander Panchenko and
Martin Potthast and
Francisco Rangel and
Paolo Rosso and
Alisa Smirnova and
Efstathios Stamatatos and
Benno Stein and
Mariona Taul{\'{e}} and
Dmitry Ustalov and
Matti Wiegmann and
Eva Zangerle},
editor = {Nazli Goharian and
Nicola Tonellotto and
Yulan He and
Aldo Lipani and
Graham McDonald and
Craig Macdonald and
Iadh Ounis},
title = {Overview of {PAN} 2024: Multi-author Writing Style Analysis, Multilingual
Text Detoxification, Oppositional Thinking Analysis, and Generative
{AI} Authorship Verification - Extended Abstract},
booktitle = {Advances in Information Retrieval - 46th European Conference on Information
Retrieval, {ECIR} 2024, Glasgow, UK, March 24-28, 2024, Proceedings,
Part {VI}},
series = {Lecture Notes in Computer Science},
volume = {14613},
pages = {3--10},
publisher = {Springer},
year = {2024},
url = {https://doi.org/10.1007/978-3-031-56072-9\_1},
doi = {10.1007/978-3-031-56072-9\_1},
timestamp = {Fri, 29 Mar 2024 23:01:36 +0100},
biburl = {https://dblp.org/rec/conf/ecir/BevendorffCCDEFFKMMPPRRSSSTUWZ24.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
Transformers 支持多种语言