All MiniLM L6 Multilingual V2 En Es Pt Pt Br V2

🚀 基于 sentence-transformers/paraphrase-MiniLM-L6-v2 的句子转换器

本模型是基于 sentence-transformers 框架，在 en-pt-br、en-es 和 en-pt 数据集上对 sentence-transformers/paraphrase-MiniLM-L6-v2 进行微调得到的。它能够将句子和段落映射到一个 384 维的密集向量空间，可用于语义文本相似度计算、语义搜索、释义挖掘、文本分类、聚类等任务。

✨ 主要特性

• 多语言支持：支持英语（en）、多语言（multilingual）、西班牙语（es）和葡萄牙语（pt）。
• 高效映射：能够将句子和段落快速准确地映射到 384 维的密集向量空间。
• 广泛应用：可用于多种自然语言处理任务，如语义文本相似度计算、语义搜索、释义挖掘、文本分类、聚类等。

📦 安装指南

首先，安装 Sentence Transformers 库：

pip install -U sentence-transformers

💻 使用示例

基础用法

from sentence_transformers import SentenceTransformer

# 从 🤗 Hub 下载模型
model = SentenceTransformer("jvanhoof/all-MiniLM-L6-multilingual-v2-en-es-pt-pt-br")
# 运行推理
sentences = [
    'We now call this place home.',
    'Moramos ali. Agora é aqui a nossa casa.',
    'É mais fácil do que se possa imaginar.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# 获取嵌入向量的相似度分数
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

📚 详细文档

模型详情

模型描述

模型来源

• 文档：Sentence Transformers 文档
• 仓库：GitHub 上的 Sentence Transformers
• Hugging Face：Hugging Face 上的 Sentence Transformers

完整模型架构

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

评估指标

知识蒸馏

• 数据集：en-pt-br、en-es 和 en-pt
• 评估方法：使用 MSEEvaluator 进行评估

翻译

• 数据集：en-pt-br、en-es 和 en-pt
• 评估方法：使用 TranslationEvaluator 进行评估

语义相似度

• 数据集：sts17-es-en-test
• 评估方法：使用 EmbeddingSimilarityEvaluator 进行评估

训练详情

训练数据集

en-pt-br

• 数据集：en-pt-br（版本 0c70bc6）
• 大小：405,807 个训练样本
• 列信息：english、non_english 和 label
• 近似统计信息（基于前 1000 个样本）： | | 英语 | 非英语 | 标签 | |------|------|------|------| | 类型 | 字符串 | 字符串 | 列表 | | 详情 |
  • 最小：4 个词元
  • 平均：23.98 个词元
  • 最大：128 个词元
  |
  • 最小：6 个词元
  • 平均：36.86 个词元
  • 最大：128 个词元
  |
  • 大小：384 个元素
  |
• 样本示例： | 英语 | 非英语 | 标签 | |------|------|------| | And then there are certain conceptual things that can also benefit from hand calculating, but I think they're relatively small in number. | E também existem alguns aspectos conceituais que também podem se beneficiar do cálculo manual, mas eu acho que eles são relativamente poucos. | [-0.2655501961708069, 0.2715710997581482, 0.13977409899234772, 0.007375418208539486, -0.09395705163478851, ...] | | One thing I often ask about is ancient Greek and how this relates. | Uma coisa sobre a qual eu pergunto com frequencia é grego antigo e como ele se relaciona a isto. | [0.34961527585983276, -0.01806497573852539, 0.06103038787841797, 0.11750973761081696, -0.34720802307128906, ...] | | See, the thing we're doing right now is we're forcing people to learn mathematics. | Vejam, o que estamos fazendo agora, é que estamos forçando as pessoas a aprender matemática. | [0.031645823270082474, -0.1787087768316269, -0.30170342326164246, 0.1304805874824524, -0.29176947474479675, ...] |
• 损失函数：MSELoss

en-es

• 数据集：en-es
• 大小：6,889,042 个训练样本
• 列信息：english、non_english 和 label
• 近似统计信息（基于前 1000 个样本）： | | 英语 | 非英语 | 标签 | |------|------|------|------| | 类型 | 字符串 | 字符串 | 列表 | | 详情 |
  • 最小：4 个词元
  • 平均：24.04 个词元
  • 最大：128 个词元
  |
  • 最小：5 个词元
  • 平均：35.11 个词元
  • 最大：128 个词元
  |
  • 大小：384 个元素
  |
• 样本示例： | 英语 | 非英语 | 标签 | |------|------|------| | And then there are certain conceptual things that can also benefit from hand calculating, but I think they're relatively small in number. | Y luego hay ciertas aspectos conceptuales que pueden beneficiarse del cálculo a mano pero creo que son relativamente pocos. | [-0.2655501961708069, 0.2715710997581482, 0.13977409899234772, 0.007375418208539486, -0.09395705163478851, ...] | | One thing I often ask about is ancient Greek and how this relates. | Algo que pregunto a menudo es sobre el griego antiguo y cómo se relaciona. | [0.34961527585983276, -0.01806497573852539, 0.06103038787841797, 0.11750973761081696, -0.34720802307128906, ...] | | See, the thing we're doing right now is we're forcing people to learn mathematics. | Vean, lo que estamos haciendo ahora es forzar a la gente a aprender matemáticas. | [0.031645823270082474, -0.1787087768316269, -0.30170342326164246, 0.1304805874824524, -0.29176947474479675, ...] |
• 损失函数：MSELoss

en-pt

• 数据集：en-pt
• 大小：6,636,095 个训练样本
• 列信息：english、non_english 和 label
• 近似统计信息（基于前 1000 个样本）： | | 英语 | 非英语 | 标签 | |------|------|------|------| | 类型 | 字符串 | 字符串 | 列表 | | 详情 |
  • 最小：4 个词元
  • 平均：23.5 个词元
  • 最大：128 个词元
  |
  • 最小：5 个词元
  • 平均：35.23 个词元
  • 最大：128 个词元
  |
  • 大小：384 个元素
  |
• 样本示例： | 英语 | 非英语 | 标签 | |------|------|------| | And the country that does this first will, in my view, leapfrog others in achieving a new economy even, an improved economy, an improved outlook. | E o país que fizer isto primeiro vai, na minha opinião, ultrapassar outros em alcançar uma nova economia até uma economia melhorada, uma visão melhorada. | [-0.1395619511604309, -0.1703503578901291, 0.21396367251873016, -0.29212212562561035, 0.2718254327774048, ...] | | In fact, I even talk about us moving from what we often call now the "knowledge economy" to what we might call a "computational knowledge economy," where high-level math is integral to what everyone does in the way that knowledge currently is. | De facto, eu até falo de mudarmos do que chamamos hoje a economia do conhecimento para o que poderemos chamar a economia do conhecimento computacional, onde a matemática de alto nível está integrada no que toda a gente faz da forma que o conhecimento actualmente está. | [-0.002996142255142331, -0.34310653805732727, -0.09672430157661438, 0.23709852993488312, -0.013354267925024033, ...] | | We can engage so many more students with this, and they can have a better time doing it. | Podemos cativar tantos mais estudantes com isto, e eles podem divertir-se mais a fazê-lo. | [0.2670706808567047, 0.09549400955438614, -0.17057836055755615, -0.2152799665927887, -0.2832679748535156, ...] |
• 损失函数：MSELoss

评估数据集

en-pt-br

• 数据集：en-pt-br（版本 0c70bc6）
• 大小：992 个评估样本
• 列信息：english、non_english 和 label
• 近似统计信息（基于前 992 个样本）： | | 英语 | 非英语 | 标签 | |------|------|------|------| | 类型 | 字符串 | 字符串 | 列表 | | 详情 |
  • 最小：4 个词元
  • 平均：24.37 个词元
  • 最大：128 个词元
  |
  • 最小：5 个词元
  • 平均：38.6 个词元
  • 最大：128 个词元
  |
  • 大小：384 个元素
  |
• 样本示例： | 英语 | 非英语 | 标签 | |------|------|------| | Thank you so much, Chris. | Muito obrigado, Chris. | [-0.1929965764284134, 0.051721055060625076, 0.3780047297477722, -0.20386895537376404, -0.2625442445278168, ...] | | And it's truly a great honor to have the opportunity to come to this stage twice; I'm extremely grateful. | É realmente uma grande honra ter a oportunidade de estar neste palco pela segunda vez. Estou muito agradecido. | [0.04667849838733673, 0.16640479862689972, 0.05405835807323456, -0.2507464587688446, -0.5305444002151489, ...] | | I have been blown away by this conference, and I want to thank all of you for the many nice comments about what I had to say the other night. | Eu fui muito aplaudido por esta conferência e quero agradecer a todos pelos muitos comentários delicados sobre o que eu tinha a dizer naquela noite. | [0.04410325363278389, 0.2660813629627228, -0.013608227483928204, 0.08376947790384293, 0.22691071033477783, ...] |
• 损失函数：MSELoss

en-es

• 数据集：en-es
• 大小：9,990 个评估样本
• 列信息：english、non_english 和 label
• 近似统计信息（基于前 1000 个样本）： | | 英语 | 非英语 | 标签 | |------|------|------|------| | 类型 | 字符串 | 字符串 | 列表 | | 详情 |
  • 最小：4 个词元
  • 平均：24.39 个词元
  • 最大：128 个词元
  |
  • 最小：4 个词元
  • 平均：36.38 个词元
  • 最大：128 个词元
  |
  • 大小：384 个元素
  |
• 样本示例： | 英语 | 非英语 | 标签 | |------|------|------| | Thank you so much, Chris. | Muchas gracias Chris. | [-0.19299663603305817, 0.051721103489398956, 0.37800467014312744, -0.20386885106563568, -0.2625444531440735, ...] | | And it's truly a great honor to have the opportunity to come to this stage twice; I'm extremely grateful. | Y es en verdad un gran honor tener la oportunidad de venir a este escenario por segunda vez. Estoy extremadamente agradecido. | [0.04667845368385315, 0.16640479862689972, 0.05405828729271889, -0.25074639916419983, -0.5305443406105042, ...] | | I have been blown away by this conference, and I want to thank all of you for the many nice comments about what I had to say the other night. | He quedado conmovido por esta conferencia, y deseo agradecer a todos ustedes sus amables comentarios acerca de lo que tenía que decir la otra noche. | [0.04410335421562195, 0.2660813629627228, -0.01360794436186552, 0.08376938849687576, 0.22691065073013306, ...] |
• 损失函数：MSELoss

en-pt

• 数据集：en-pt
• 大小：9,992 个评估样本
• 列信息：english、non_english 和 label
• 近似统计信息（基于前 1000 个样本）： | | 英语 | 非英语 | 标签 | |------|------|------|------| | 类型 | 字符串 | 字符串 | 列表 | | 详情 |
  • 最小：4 个词元
  • 平均：23.82 个词元
  • 最大：128 个词元
  |
  • 最小：5 个词元
  • 平均：36.7 个词元
  • 最大：128 个词元
  |
  • 大小：384 个元素
  |
• 样本示例： | 英语 | 非英语 | 标签 | |------|------|------| | Thank you so much, Chris. | Muito obrigado, Chris. | [-0.19299663603305817, 0.051721103489398956, 0.37800467014312744, -0.20386885106563568, -0.2625444531440735, ...] | | And it's truly a great honor to have the opportunity to come to this stage twice; I'm extremely grateful. | É realmente uma grande honra ter a oportunidade de pisar este palco pela segunda vez. Estou muito agradecido. | [0.04667849838733673, 0.16640479862689972, 0.05405835807323456, -0.2507464587688446, -0.5305444002151489, ...] | | I have been blown away by this conference, and I want to thank all of you for the many nice comments about what I had to say the other night. | Fiquei muito impressionado com esta conferência e quero agradecer a todos os imensos comentários simpáticos sobre o que eu tinha a dizer naquela noite. | [0.04410335421562195, 0.2660813629627228, -0.01360794436186552, 0.08376938849687576, 0.22691065073013306, ...] |
• 损失函数：MSELoss

训练超参数

非默认超参数

• eval_strategy：steps
• per_device_train_batch_size：128
• per_device_eval_batch_size：128
• gradient_accumulation_steps：8
• num_train_epochs：6
• warmup_ratio：0.15
• bf16：True

所有超参数

训练日志