Flan T5 Tsa Prompt Xl

🚀 Model Card for Model ID

This model card provides detailed information about a fine - tuned Flan - T5 model for sentiment analysis. It includes model details, usage instructions, training information, and evaluation results.

🚀 Quick Start

Direct Use

Here are the following two steps for a quick start with model application:

1. Loading model and tokenizer:

import torch
from transformers import AutoTokenizer, T5ForConditionalGeneration

# Setup model path.
model_path = "nicolay-r/flan-t5-tsa-prompt-base"
# Setup device.
device = "cuda:0"

model = T5ForConditionalGeneration.from_pretrained(model_path, torch_dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(model_path)
model.to(device)

2. Setup ask method for generating LLM responses:

def ask(prompt):
  inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
  inputs.to(device)
  output = model.generate(**inputs, temperature=1)
  return tokenizer.batch_decode(output, skip_special_tokens=True)[0]

Finally, you can infer model results as follows:

# Input sentence.
sentence = "I would support him"
# Input target.
target = "him"
# output response
flant5_response = ask(f"What's the attitude of the sentence '{context}', to the target '{target}'?")
print(f"Author opinion towards `{target}` in `{sentence}` is:\n{flant5_response}")

The response of the model is as follows:

Author opinion towards "him" in "I would support him despite his bad behavior." is: positive

Downstream Use

Please refer to the related section of the Reasoning - for - Sentiment - Analysis Framework.

With this example it applies this model (zero - shot - learning) in the PROMPT mode to the validation data of the RuSentNE - 2023 competition for evaluation.

python thor_finetune.py -m "nicolay-r/flan-t5-tsa-prompt-xl" -r "prompt" \
    -p "What's the attitude of the sentence '{context}', to the target '{target}'?" \
    -d "rusentne2023" -z -bs 4 -f "./config/config.yaml"

Follow the Google Colab Notebook for implementation reproduction.

✨ Features

• BATCHING MODE SUPPORT: As of February 23, 2025, the model supports batching mode. See 🌌 Flan - T5 provider for bulk - chain project. Test is available here.
• Sentiment Analysis: Capable of performing sentiment analysis and target - sentiment analysis.
• Prompt Tuning: Utilizes prompt - tuning for better performance.

📦 Installation

No specific installation steps are provided in the original document.

📚 Documentation

Model Details

Model Description

• Developed by: Reforged by nicolay - r, initial credits for implementation to scofield7419.
• Model type: [Flan - T5](https://huggingface.co/docs/transformers/en/model_doc/flan - t5).
• Language(s) (NLP): English.
• License: [Apache License 2.0](https://github.com/scofield7419/THOR - ISA/blob/main/LICENSE.txt).

Model Sources

• Repository: Reasoning - for - Sentiment - Analysis - Framework.
• Paper: https://arxiv.org/abs/2404.12342.
• Demo: We have a code on Google - Colab for launching the related model.

Uses

Out - of - Scope Use

This model represents a fine - tuned version of the Flan - T5 on RuSentNE - 2023 dataset. Since the dataset represents three - scale output answers (positive, negative, neutral), the behavior in general might be biased to this particular task.

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases, and limitations of the model. More information is needed for further recommendations.

How to Get Started with the Model

Please proceed with the code from the related Three - Hop - Reasoning CoT section. Or follow the related section on Google Colab notebook.

Training Details

Training Data

We utilize train data which was automatically translated into English using GoogleTransAPI. The initial source of the texts written in Russian is from the following repository: https://github.com/dialogue - evaluation/RuSentNE - evaluation

The translated version of the dataset in English can be automatically downloaded via the following script: https://github.com/nicolay-r/Reasoning-for-Sentiment-Analysis-Framework/blob/main/rusentne23_download.py

Training Procedure

This model has been trained using the PROMPT - finetuning. For training procedure accomplishment, use the reforged version of THoR framework.

The Google - colab notebook can be used for reproduction. The overall training process took 3 epochs.


Training Hyperparameters

• Training regime: All the configuration details were highlighted in the related config file.

Evaluation

Testing Data, Factors & Metrics

Testing Data

The direct link to the test evaluation data: https://github.com/dialogue - evaluation/RuSentNE - evaluation/blob/main/final_data.csv

Metrics

For the model evaluation, two metrics were used:

1. F1_PN -- F1 - measure over positive and negative classes;
2. F1_PN0 -- F1 - measure over positive, negative, and neutral classes;

Results

The test evaluation for this model showcases the F1_PN = 60.024.

Below is the log of the training process that showcases the final performance on the RuSentNE - 2023 test set after 4 epochs (lines 5 - 6):

  F1_PN  F1_PN0  default   mode
0  66.678  73.838   73.838  valid
1  68.019  74.816   74.816  valid
2  67.870  74.688   74.688  valid
3  65.090  72.449   72.449   test
4  65.090  72.449   72.449   test

🔧 Technical Details

The model is a fine - tuned version of Flan - T5. It uses PROMPT - finetuning on the RuSentNE - 2023 dataset. The training data was translated from Russian to English using GoogleTransAPI. The overall training process took 3 epochs, and specific training configurations are detailed in the config file.

📄 License

The model is licensed under the [Apache License 2.0](https://github.com/scofield7419/THOR - ISA/blob/main/LICENSE.txt).