Gliner Model Merge Large V1.0

🚀 xomad/gliner-model-merge-large-v1.0模型

xomad/gliner-model-merge-large-v1.0 模型基于预训练模型 knowledgator/gliner-multitask-large-v0.5 开发，旨在探索模型融合技术的能力。通过该技术，模型性能显著提升了3.25个百分点，F1分数从0.6276提高到了0.6601。

该模型仅在具有商业友好许可的数据集上进行训练，以确保在Apache - 2.0许可下具有广泛的适用性。训练过程中使用了以下数据集：

• knowledgator/GLINER-multi-task-synthetic-data
• EmergentMethods/AskNews-NER-v0
• urchade/pile-mistral-v0.1
• MultiCoNER/multiconer_v2
• DFKI-SLT/few-nerd

🚀 快速开始

本模型基于预训练模型 knowledgator/gliner-multitask-large-v0.5 开发，通过模型融合技术显著提升了性能。下面将介绍如何安装和使用该模型。

✨ 主要特性

• 性能提升：通过模型融合技术，F1分数显著提高，从0.6276提升到0.6601。
• 商业友好：仅在具有商业友好许可的数据集上训练，适用于Apache - 2.0许可。
• 多数据集训练：使用多个公开数据集进行训练，包括 knowledgator/GLINER-multi-task-synthetic-data、EmergentMethods/AskNews-NER-v0 等。

📦 安装指南

要使用此模型，你必须安装 GLiNER Python库：

pip install gliner

下载GLiNER库后，你可以导入GLiNER类，然后使用 GLiNER.from_pretrained 加载此模型。

💻 使用示例

基础用法

from gliner import GLiNER

model = GLiNER.from_pretrained("xomad/gliner-model-merge-large-v1.0")

text = """
Microsoft was founded by Bill Gates and Paul Allen on April 4, 1975 to develop and sell BASIC interpreters for the Altair 8800. During his career at Microsoft, Gates held the positions of chairman, chief executive officer, president and chief software architect, while also being the largest individual shareholder until May 2014.
"""

labels = ["founder", "computer", "software", "position", "date", "company"]

entities = model.predict_entities(text, labels)

for entity in entities:
    print(entity["text"], "=>", entity["label"])

输出：

Microsoft => company
Bill Gates => founder
Paul Allen => founder
April 4, 1975 => date
BASIC => software
Altair 8800 => computer
Microsoft => company
chairman => position
chief executive officer => position
president => position
chief software architect => position
May 2014 => date

📚 详细文档

⚙️ 微调过程

该过程从基础模型 knowledgator/gliner-multitask-large-v0.5 开始。我们的模型 xomad/gliner-model-merge-large-v1.0 在上述每个数据集上分别进行微调，并在微调过程中保存多个检查点。我们将所有这些检查点放入一个池中，然后应用 Model soups 技术来生成不同的融合模型：

• uniform_merged
• greedy_on_random
• greedy_on_sorted

随后，我们对从上述3个模型和原始模型中选出的模型对应用 WiSE - FT 融合技术，生成 wise_ft_merged 模型。这结束了第一阶段的微调。

在第二阶段的微调中，以 wise_ft_merged 作为新的起点重复该过程，以生成最终模型。整个微调流程如下图所示：

微调模型池和融合模型的性能在 CrossNER、TwitterNER基准测试中进行评估，并绘制在以下两个图中（分别为 crossner_f1 和 other_f1）。

第一阶段微调图：

第二阶段微调图：

📊 基准测试

不同零样本NER基准测试（CrossNER、mit - movie和mit - restaurant）的性能，数据来源于 https://huggingface.co/knowledgator/gliner-multitask-large-v0.5：

不同数据集上的详细性能：

作者

Hoan Nguyen，来自xomad.com

引用

@misc{wortsman2022modelsoupsaveragingweights,
      title={Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time}, 
      author={Mitchell Wortsman and Gabriel Ilharco and Samir Yitzhak Gadre and Rebecca Roelofs and Raphael Gontijo-Lopes and Ari S. Morcos and Hongseok Namkoong and Ali Farhadi and Yair Carmon and Simon Kornblith and Ludwig Schmidt},
      year={2022},
      eprint={2203.05482},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2203.05482}, 
}

@InProceedings{Wortsman_2022_CVPR,
    author    = {Wortsman, Mitchell and Ilharco, Gabriel and Kim, Jong Wook and Li, Mike and Kornblith, Simon and Roelofs, Rebecca and Lopes, Raphael Gontijo and Hajishirzi, Hannaneh and Farhadi, Ali and Namkoong, Hongseok and Schmidt, Ludwig},
    title     = {Robust Fine-Tuning of Zero-Shot Models},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {7959-7971}
}

@misc{stepanov2024gliner,
      title={GLiNER multi-task: Generalist Lightweight Model for Various Information Extraction Tasks}, 
      author={Ihor Stepanov and Mykhailo Shtopko},
      year={2024},
      eprint={2406.12925},
      archivePrefix={arXiv},
      primaryClass={id='cs.LG' full_name='Machine Learning' is_active=True alt_name=None in_archive='cs' is_general=False description='Papers on all aspects of machine learning research (supervised, unsupervised, reinforcement learning, bandit problems, and so on) including also robustness, explanation, fairness, and methodology. cs.LG is also an appropriate primary category for applications of machine learning methods.'}
}

@misc{zaratiana2023gliner,
      title={GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer}, 
      author={Urchade Zaratiana and Nadi Tomeh and Pierre Holat and Thierry Charnois},
      year={2023},
      eprint={2311.08526},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

📄 许可证

本模型采用Apache - 2.0许可证。