Tags Allnli GroNLP Bert Base Dutch Cased

🚀 tags - allnli - GroNLP - bert - base - dutch - cased

这是一个句子转换器模型，它能将句子和段落映射到一个768维的密集向量空间，可用于聚类或语义搜索等任务。

🚀 快速开始

本模型可通过两种方式使用，分别是使用sentence - transformers库和直接使用HuggingFace Transformers库。下面为你详细介绍。

📦 安装指南

若要使用sentence - transformers库，你需要先安装它：

pip install -U sentence-transformers

💻 使用示例

基础用法（Sentence - Transformers）

使用sentence - transformers库时，你可以这样使用该模型：

from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)

高级用法（HuggingFace Transformers）

若不使用sentence - transformers库，你可以按以下步骤使用模型：首先，将输入传递给Transformer模型，然后对上下文词嵌入应用正确的池化操作。

from transformers import AutoTokenizer, AutoModel
import torch


#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)


# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained(textgain/tags-allnli-GroNLP-bert-base-dutch-cased)
model = AutoModel.from_pretrained(textgain/tags-allnli-GroNLP-bert-base-dutch-cased)

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)

📚 详细文档

评估结果

若要对该模型进行自动化评估，请参考句子嵌入基准测试：https://seb.sbert.net

训练信息

该模型使用以下参数进行训练：

数据加载器

sentence_transformers.datasets.NoDuplicatesDataLoader.NoDuplicatesDataLoader，长度为4687，参数如下：

{'batch_size': 128}

损失函数

sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss，参数如下：

{'scale': 20.0, 'similarity_fct': 'cos_sim'}

fit()方法的参数

{
    "epochs": 1,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'torch.optim.adamw.AdamW'>",
    "optimizer_params": {
        "lr": 5e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": 3000,
    "warmup_steps": 300.0,
    "weight_decay": 0.01
}

完整模型架构

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)

BibTeX引用

@inproceedings{kosar-etal-2023-advancing,
    title = "Advancing Topical Text Classification: A Novel Distance-Based Method with Contextual Embeddings",
    author = "Kosar, Andriy  and
      De Pauw, Guy  and
      Daelemans, Walter",
    editor = "Mitkov, Ruslan  and
      Angelova, Galia",
    booktitle = "Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing",
    month = sep,
    year = "2023",
    address = "Varna, Bulgaria",
    publisher = "INCOMA Ltd., Shoumen, Bulgaria",
    url = "https://aclanthology.org/2023.ranlp-1.64",
    pages = "586--597",
}

其他信息

小部件示例

源句子：“In Spanje en Portugal zijn dit weekend door branden duizenden hectares bos verwoest, meldt persbureau DPA. In het westen van Portugal was volgens de autoriteiten vanochtend 6200 hectare afgebrand.” 候选句子：

• "kunst, cultuur, entertainment en media"
• "conflict, oorlog en vrede"
• "misdaad, recht en gerechtigheid"
• "rampen, ongevallen en noodgevallen"
• "economie, handel en financiën"
• "onderwijs"
• "milieu"
• "gezondheid"
• "menselijke interesse"
• "arbeid"
• "levensstijl en vrije tijd"
• "politiek"
• "religie en geloof"
• "wetenschap en technologie"
• "maatschappij"
• "sport"
• "weer" 示例标题：“IPTC媒体主题”