Tags Allnli GroNLP Bert Base Dutch Cased

🚀 tags - allnli - GroNLP - bert - base - dutch - cased

這是一個句子轉換器模型，它能將句子和段落映射到一個768維的密集向量空間，可用於聚類或語義搜索等任務。

🚀 快速開始

本模型可通過兩種方式使用，分別是使用sentence - transformers庫和直接使用HuggingFace Transformers庫。下面為你詳細介紹。

📦 安裝指南

若要使用sentence - transformers庫，你需要先安裝它：

pip install -U sentence-transformers

💻 使用示例

基礎用法（Sentence - Transformers）

使用sentence - transformers庫時，你可以這樣使用該模型：

from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('{MODEL_NAME}')
embeddings = model.encode(sentences)
print(embeddings)

高級用法（HuggingFace Transformers）

若不使用sentence - transformers庫，你可以按以下步驟使用模型：首先，將輸入傳遞給Transformer模型，然後對上下文詞嵌入應用正確的池化操作。

from transformers import AutoTokenizer, AutoModel
import torch


#Mean Pooling - Take attention mask into account for correct averaging
def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)


# Sentences we want sentence embeddings for
sentences = ['This is an example sentence', 'Each sentence is converted']

# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained(textgain/tags-allnli-GroNLP-bert-base-dutch-cased)
model = AutoModel.from_pretrained(textgain/tags-allnli-GroNLP-bert-base-dutch-cased)

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, mean pooling.
sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)

📚 詳細文檔

評估結果

若要對該模型進行自動化評估，請參考句子嵌入基準測試：https://seb.sbert.net

訓練信息

該模型使用以下參數進行訓練：

數據加載器

sentence_transformers.datasets.NoDuplicatesDataLoader.NoDuplicatesDataLoader，長度為4687，參數如下：

{'batch_size': 128}

損失函數

sentence_transformers.losses.MultipleNegativesRankingLoss.MultipleNegativesRankingLoss，參數如下：

{'scale': 20.0, 'similarity_fct': 'cos_sim'}

fit()方法的參數

{
    "epochs": 1,
    "evaluation_steps": 0,
    "evaluator": "NoneType",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'torch.optim.adamw.AdamW'>",
    "optimizer_params": {
        "lr": 5e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": 3000,
    "warmup_steps": 300.0,
    "weight_decay": 0.01
}

完整模型架構

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)

BibTeX引用

@inproceedings{kosar-etal-2023-advancing,
    title = "Advancing Topical Text Classification: A Novel Distance-Based Method with Contextual Embeddings",
    author = "Kosar, Andriy  and
      De Pauw, Guy  and
      Daelemans, Walter",
    editor = "Mitkov, Ruslan  and
      Angelova, Galia",
    booktitle = "Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing",
    month = sep,
    year = "2023",
    address = "Varna, Bulgaria",
    publisher = "INCOMA Ltd., Shoumen, Bulgaria",
    url = "https://aclanthology.org/2023.ranlp-1.64",
    pages = "586--597",
}

其他信息

小部件示例

源句子：“In Spanje en Portugal zijn dit weekend door branden duizenden hectares bos verwoest, meldt persbureau DPA. In het westen van Portugal was volgens de autoriteiten vanochtend 6200 hectare afgebrand.” 候選句子：

• "kunst, cultuur, entertainment en media"
• "conflict, oorlog en vrede"
• "misdaad, recht en gerechtigheid"
• "rampen, ongevallen en noodgevallen"
• "economie, handel en financiën"
• "onderwijs"
• "milieu"
• "gezondheid"
• "menselijke interesse"
• "arbeid"
• "levensstijl en vrije tijd"
• "politiek"
• "religie en geloof"
• "wetenschap en technologie"
• "maatschappij"
• "sport"
• "weer" 示例標題：“IPTC媒體主題”