Hebemo Disgust

🚀 HebEMO - Emotion Recognition Model for Modern Hebrew

HebEMO is a tool designed to detect polarity and extract emotions from modern Hebrew User-Generated Content (UGC). It was trained on a unique Covid-19 related dataset that we collected and annotated. The model achieved a high performance in polarity classification with a weighted average F1-score of 0.96. In emotion detection, it reached an F1-score ranging from 0.78 to 0.97, except for the surprise emotion, which the model had difficulty capturing (F1 = 0.41). These results outperform the best-reported performance, even when compared to models in the English language.

✨ Features

• Polarity and Emotion Detection: HebEMO can accurately detect the polarity (sentiment) and extract emotions from modern Hebrew UGC.
• High Performance: Demonstrates excellent performance in both polarity classification and emotion detection tasks.
• Unique Dataset: Trained on a specially collected and annotated Covid-19 related dataset.

📦 Installation

The installation steps are included in the usage examples section. You need to install the following packages:

# !pip install pyplutchik==0.0.7
# !pip install transformers==4.14.1

And clone the repository:

!git clone https://github.com/avichaychriqui/HeBERT.git

💻 Usage Examples

Basic Usage

# Install necessary packages
# !pip install pyplutchik==0.0.7
# !pip install transformers==4.14.1

# Clone the repository
!git clone https://github.com/avichaychriqui/HeBERT.git

# Import the HebEMO class
from HeBERT.src.HebEMO import *

# Initialize the HebEMO model
HebEMO_model = HebEMO()

# Analyze text from a file
HebEMO_model.hebemo(input_path = 'data/text_example.txt')
# return analyzed pandas.DataFrame  

# Analyze text directly
hebEMO_df = HebEMO_model.hebemo(text='החיים יפים ומאושרים', plot=True)

Advanced Usage - Sentiment Classification Model (Polarity ONLY)

from transformers import AutoTokenizer, AutoModel, pipeline

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("avichr/heBERT_sentiment_analysis") #same as 'avichr/heBERT' tokenizer
model = AutoModel.from_pretrained("avichr/heBERT_sentiment_analysis")

# Create a sentiment analysis pipeline
sentiment_analysis = pipeline(
    "sentiment-analysis",
    model="avichr/heBERT_sentiment_analysis",
    tokenizer="avichr/heBERT_sentiment_analysis",
    return_all_scores = True
)

# Analyze text
sentiment_analysis('אני מתלבט מה לאכול לארוחת צהריים')	
# Output: [[{'label': 'neutral', 'score': 0.9978172183036804},
#           {'label': 'positive', 'score': 0.0014792329166084528},
#           {'label': 'negative', 'score': 0.0007035882445052266}]]

sentiment_analysis('קפה זה טעים')
# Output: [[{'label': 'neutral', 'score': 0.00047328314394690096},
#           {'label': 'possitive', 'score': 0.9994067549705505},
#           {'label': 'negetive', 'score': 0.00011996887042187154}]]

sentiment_analysis('אני לא אוהב את העולם')
# Output: [[{'label': 'neutral', 'score': 9.214012970915064e-05}, 
#           {'label': 'possitive', 'score': 8.876807987689972e-05}, 
#           {'label': 'negetive', 'score': 0.9998190999031067}]]

📚 Documentation

Emotion UGC Data Description

Our UGC data consists of comments posted on news articles collected from 3 major Israeli news sites between January 2020 and August 2020. The total data size is approximately 150 MB, containing over 7 million words and 350K sentences. Around 2000 sentences were annotated by crowd members (3 - 10 annotators per sentence) for overall sentiment (polarity) and eight emotions: anger, disgust, anticipation, fear, joy, sadness, surprise, and trust. The table below shows the percentage of sentences in which each emotion appeared.

Performance

Emotion Recognition

The above metrics are for the positive class (meaning, the emotion is reflected in the text).

Sentiment (Polarity) Analysis

The sentiment (polarity) analysis model is also available on AWS! For more information, visit AWS' git

📄 License

No license information is provided in the original document.

Contact Us

• Avichay Chriqui
• Inbal yahav
• The Coller Semitic Languages AI Lab

Thank you, תודה, شكرا

Citation

If you used this model, please cite us as: Chriqui, A., & Yahav, I. (2022). HeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition. INFORMS Journal on Data Science, forthcoming.

@article{chriqui2021hebert,
  title={HeBERT \& HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition},
  author={Chriqui, Avihay and Yahav, Inbal},
  journal={INFORMS Journal on Data Science},
  year={2022}
}