BERTIS

🚀 Model Card for BERTIS Model

This README offers an overview of BERTIS, including its purpose and usage. BERTIS is introduced in the paper "META4: Semantically-Aligned Generation of metaphoric gestures using self-supervised text and speech representations". If you find this work useful, please consider citing our paper.

Bibtex:

@article{fares2023META4,
  title={META4: Semantically-Aligned Generation of metaphoric gestures using self-supervised text and speech representations},
  author={Fares, Mireille and Pelachaud, Catherine and Obin, Nicolas},
  journal={arXiv preprint arXiv:2311.05481},
  year={2023}
}

✨ Features

BERTIS Model Description

BERTIS (BERT-based Image Schema) is a computational model designed to classify input texts into specific Image Schema classes. Image Schemas are cognitive patterns that play a fundamental role in shaping the way humans conceptualize and reason about various concepts present in language. BERTIS is built upon the BERT architecture and fine-tuned using a specialized corpus created by Wachowiak et al. (2022).

BERTIS Image Schema Classes

The main purpose of BERTIS is to automatically categorize input texts into predefined Image Schema classes. BERTIS considers 14 distinct Image Schema classes, each capturing a specific cognitive pattern. These classes and their corresponding examples (taken from Wachowiak et al. (2022)) are as follows:

• CENTER-PERIPHERY: She brushed the thought away.
• CONTACT: That blew me away.
• CONTAINMENT: Keep it in the back of your mind.
• COVERING: His judgement is clouded.
• FORCE: They are attracted to each other.
• LINK: Breaking social ties.
• OBJECT: Seize the opportunity.
• PART-WHOLE: They assembled a theory.
• SCALE: This class is bigger than that one.
• SOURCE_PATH_GOAL: The time for action has arrived.
• SPLITTING: What separates the men from the boys?
• SUBSTANCE: Emotions are tinged with suffuse.
• VERTICALITY: No known spoken language uses the lateral axis for time.
• SUPPORT: The poor in our country need a boost up.

📦 Installation

No installation steps are provided in the original document.

💻 Usage Examples

No code examples are provided in the original document.

📚 Documentation

Training and Testing Details

The data used for training, validating, and testing BERTIS can be found at: https://github.com/lwachowiak/Systematic-Analysis-of-Image-Schemas-through-Explainable-Multilingual-Language-Models/blob/main/Data/Image%20Schemas%20English%20and%20German.csv.

80% of the data was used for training BERTIS, 10% for validation, and 10% for testing.

Evaluation

Metrics

• Precision: measures the proportion of correctly predicted positive instances out of all instances predicted as positive. It focuses on the accuracy of positive predictions.
• Recall: measures the proportion of correctly predicted positive instances out of all actual positive instances. It focuses on the ability to capture positive instances.
• F1-score: the harmonic mean of precision and recall, providing a balanced measure that combines both metrics.

Results

Overall Accuracy: 0.93

For all Image Schema classes, the F1-score ranges from 0.77 to 1, indicating that BERTIS performs well in terms of both accurately predicting the correct Image Schema classes (precision) and capturing as many correct Image Schema classes as possible (recall). These results are also reflected in the precision and recall scores for all Image Schema classes. We observe a high precision (between 0.74 and 1) for all classes, which indicates the number of correctly predicted Image Schema classes. The recall scores are above 0.74 for most classes, with one Image Schema class having a recall score of 0.67. The overall accuracy of BERTIS with respect to all Image Schema classes is 0.93, indicating high accuracy and good performance in classifying the input text into the correct Image Schema classes.

🔧 Technical Details

The model is based on the BERT architecture and fine-tuned using a specialized corpus. The data used for training, validation, and testing is publicly available, and the distribution of the data for different purposes is clearly defined.

📄 License

The model is released under the Academic Free License (AFL-3.0).

Information Table

Developer Information

Developed by: Mireille Fares Language(s) (NLP): English, can generalize to other languages such as German