RelBERT Open-Source Semantic Relation Understanding Model - Free Deployment to Solve Vocabulary Classification and Analogy Problems

Roberta Large Semeval2012 V2 Mask Prompt A Nce

Developed by research-backup

RelBERT is a semantic relation understanding model fine-tuned on RoBERTa-large, specifically designed for lexical relation classification and analogy question tasks.

Text Embedding

Transformers

#Relation Embedding Extraction #Lexical Relation Classification #Analogy Question Answering

Downloads 16

Release Time : 8/17/2022

Model Overview

This model, by fine-tuning the RoBERTa-large architecture, focuses on understanding semantic relations between words and excels in various relation classification and analogy tasks.

Model Features

High-performance Relation Understanding

Achieves high accuracy in various semantic relation tasks, such as F1 score of 0.926 on the BLESS dataset.

Multi-task Adaptation

Capable of handling multiple tasks simultaneously, including relation classification, analogy questions, and relation mapping.

Prompt-based Fine-tuning

Uses specific templates for masked prompt fine-tuning to enhance relation representation capabilities.

Model Capabilities

Lexical Relation Classification

Analogy Question Answering

Semantic Relation Mapping

Relation Embedding Generation

Use Cases

Educational Technology

SAT Analogy Question Answering

Used to answer vocabulary analogy questions in standardized tests.

SAT accuracy rate of 71.8%

Knowledge Graph

Lexical Relation Classification

Entity relation classification when constructing knowledge graphs.

F1 score of 0.959 on the K&H+N dataset

🚀 relbert/roberta-large-semeval2012-v2-mask-prompt-a-nce

RelBERT fine-tuned from roberta-large for relation understanding tasks.

🚀 Quick Start

RelBERT is fine-tuned from roberta-large on relbert/semeval2012_relational_similarity_v2. Fine-tuning is carried out via the RelBERT library (refer to the repository for more details). It achieves the following results on relation understanding tasks:

Analogy Question

Dataset, Full result
- Accuracy on SAT (full): 0.7165775401069518
- Accuracy on SAT: 0.7181008902077152
- Accuracy on BATS: 0.7626459143968871
- Accuracy on U2: 0.6359649122807017
- Accuracy on U4: 0.6435185185185185
- Accuracy on Google: 0.946

Lexical Relation Classification

Dataset, Full result
- Micro F1 score on BLESS: 0.9264728039777008
- Micro F1 score on CogALexV: 0.8760563380281691
- Micro F1 score on EVALution: 0.7085590465872156
- Micro F1 score on K&H+N: 0.9585448981011337
- Micro F1 score on ROOT09: 0.9088060169225948

Relation Mapping

Dataset, Full result
- Accuracy on Relation Mapping: 0.8338095238095238

💻 Usage Examples

Basic Usage

Install the relbert library via pip:

pip install relbert

Activate the model as follows:

from relbert import RelBERT
model = RelBERT("relbert/roberta-large-semeval2012-v2-mask-prompt-a-nce")
vector = model.get_embedding(['Tokyo', 'Japan'])  # shape of (1024, )

📚 Documentation

Training hyperparameters

The following hyperparameters were used during training:

Property	Details
Model	roberta-large
Max Length	64
Mode	mask
Data	relbert/semeval2012_relational_similarity_v2
Template Mode	manual
Template	Today, I finally discovered the relation between and : is the of
Loss Function	nce_logout
Temperature NCE Constant	0.05
Temperature NCE Rank	{'min': 0.01, 'max': 0.05, 'type': 'linear'}
Epoch	29
Batch	128
Learning Rate	5e-06
LR Decay	False
LR Warmup	1
Weight Decay	0
Random Seed	0
Exclude Relation	None
Number of Samples	640
Gradient Accumulation	8

The full configuration can be found at fine-tuning parameter file.

📄 License

Reference

If you use any resource from RelBERT, please consider citing our paper.

@inproceedings{ushio-etal-2021-distilling-relation-embeddings,
    title = "{D}istilling {R}elation {E}mbeddings from {P}re-trained {L}anguage {M}odels",
    author = "Ushio, Asahi  and
      Schockaert, Steven  and
      Camacho-Collados, Jose",
    booktitle = "EMNLP 2021",
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
}

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご