Gliner Model Merge Large V1.0

🚀 xomad/gliner-model-merge-large-v1.0

The xomad/gliner-model-merge-large-v1.0 model is developed from the pretrained model knowledgator/gliner-multitask-large-v0.5. It explores the capabilities of model merging techniques, resulting in a significant performance boost of 3.25 points, elevating the model's capability from 0.6276 to 0.6601 F1-score.

🚀 Quick Start

The xomad/gliner-model-merge-large-v1.0 model is designed for token - classification tasks, specifically in the field of NER. It is trained on datasets with commercial - friendly licenses, ensuring broad applicability under the Apache - 2.0 license.

✨ Features

• Performance Boost: Achieved a 3.25 - point increase in F1 - score through model merging techniques.
• Commercial - Friendly: Trained on datasets with commercial - friendly licenses, suitable for a wide range of applications.
• Multi - Dataset Training: Utilized multiple datasets during the training process, enhancing the model's generalization ability.

📦 Installation

To use this model, you must install the GLiNER Python library:

pip install gliner

Once you've downloaded the GLiNER library, you can import the GLiNER class. You can then load this model using GLiNER.from_pretrained.

💻 Usage Examples

Basic Usage

from gliner import GLiNER

model = GLiNER.from_pretrained("xomad/gliner-model-merge-large-v1.0")

text = """
Microsoft was founded by Bill Gates and Paul Allen on April 4, 1975 to develop and sell BASIC interpreters for the Altair 8800. During his career at Microsoft, Gates held the positions of chairman, chief executive officer, president and chief software architect, while also being the largest individual shareholder until May 2014.
"""

labels = ["founder", "computer", "software", "position", "date", "company"]

entities = model.predict_entities(text, labels)

for entity in entities:
    print(entity["text"], "=>", entity["label"])

Output:

Microsoft => company
Bill Gates => founder
Paul Allen => founder
April 4, 1975 => date
BASIC => software
Altair 8800 => computer
Microsoft => company
chairman => position
chief executive officer => position
president => position
chief software architect => position
May 2014 => date

📚 Documentation

⚙️ Finetuning process

The process begins with the base model knowledgator/gliner-multitask-large-v0.5. Our model xomad/gliner-model-merge-large-v1.0 is fine - tuned separately on each of the following datasets:

• knowledgator/GLINER-multi-task-synthetic-data
• EmergentMethods/AskNews-NER-v0
• urchade/pile-mistral-v0.1
• MultiCoNER/multiconer_v2
• DFKI-SLT/few-nerd

We save multiple checkpoints along the fine - tuning process. We put all these checkpoints together into a pool and then we apply the Model soups technique to produce different merged models:

• uniform_merged
• greedy_on_random
• greedy_on_sorted

Following this, we apply WiSE - FT merging technique to pairs of models selected from a group of the above 3 models and the original model to produce the wise_ft_merged model. This concludes the 1st finetuning phase.

The process is then repeated in the 2nd finetuning phase, using the wise_ft_merged as the new starting point, to produce the final model. The whole finetuning flow is illustrated in the following figure:

The performance of the pool of fine - tuned models and the merged models are evaluated on the CrossNER, TwitterNER benchmarks, and plotted in the following 2 figures (as crossner_f1 and other_f1 respectively).

The 1st finetuning phase plot:

The 2nd finetuning phase plot:

📊 Benchmarks

Performance on different zero - shot NER benchmarks (CrossNER, mit - movie and mit - restaurant), numbers reported from https://huggingface.co/knowledgator/gliner-multitask-large-v0.5:

Detailed performance on different datasets:

🔧 Technical Details

The model merging techniques, such as Model soups and WiSE - FT, play a crucial role in improving the model's performance. By combining multiple checkpoints and applying these techniques, the model can achieve better generalization and accuracy.

📄 License

This project is licensed under the Apache - 2.0 license.

Authors

Hoan Nguyen, at xomad.com

Citations

@misc{wortsman2022modelsoupsaveragingweights,
      title={Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time}, 
      author={Mitchell Wortsman and Gabriel Ilharco and Samir Yitzhak Gadre and Rebecca Roelofs and Raphael Gontijo-Lopes and Ari S. Morcos and Hongseok Namkoong and Ali Farhadi and Yair Carmon and Simon Kornblith and Ludwig Schmidt},
      year={2022},
      eprint={2203.05482},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2203.05482}, 
}

@InProceedings{Wortsman_2022_CVPR,
    author    = {Wortsman, Mitchell and Ilharco, Gabriel and Kim, Jong Wook and Li, Mike and Kornblith, Simon and Roelofs, Rebecca and Lopes, Raphael Gontijo and Hajishirzi, Hannaneh and Farhadi, Ali and Namkoong, Hongseok and Schmidt, Ludwig},
    title     = {Robust Fine-Tuning of Zero-Shot Models},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {7959-7971}
}

@misc{stepanov2024gliner,
      title={GLiNER multi-task: Generalist Lightweight Model for Various Information Extraction Tasks}, 
      author={Ihor Stepanov and Mykhailo Shtopko},
      year={2024},
      eprint={2406.12925},
      archivePrefix={arXiv},
      primaryClass={id='cs.LG' full_name='Machine Learning' is_active=True alt_name=None in_archive='cs' is_general=False description='Papers on all aspects of machine learning research (supervised, unsupervised, reinforcement learning, bandit problems, and so on) including also robustness, explanation, fairness, and methodology. cs.LG is also an appropriate primary category for applications of machine learning methods.'}
}

@misc{zaratiana2023gliner,
      title={GLiNER: