Sujet Finance 8B V0.1

🚀 Introducing Sujet Finance 8B v0.1

Sujet Finance 8B v0.1 is a specialized financial language model. It is fine - tuned on the Sujet Finance Instruct - 177k dataset, offering high - quality solutions for various finance - related tasks.

✨ Features

🎯 Fine - Tuning Focus

In the initial fine - tuning, the model focuses on three key financial tasks:

1. ✅❌ Yes/No Questions

   • Description: Answer financial questions with a simple "yes" or "no".
   • Class Distribution:
     • Train Set: 5,265 "yes" examples, 5,302 "no" examples.
     • Eval Set: 1,340 "yes" examples, 1,303 "no" examples.

2. 📂 Topic Classification

   • Description: Classify financial texts into specific finance - related categories like company news, markets, earnings, etc.
   • Class Distribution:
     • Train Set: Balanced across 20 classes, with 29 - 40 examples per class.
     • Eval Set: Varies across classes, ranging from 4 to 15 examples per class.

3. 😊😐😡 Sentiment Analysis

   • Description: Analyze financial texts to categorize sentiments as positive, negative, neutral, bearish, or bullish.
   • Class Distribution:
     • Train Set: 1,160 positive, 1,155 negative, 1,150 neutral, 1,133 bearish, and 1,185 bullish examples.
     • Eval Set: 281 positive, 286 negative, 291 neutral, 308 bearish, and 256 bullish examples.

📦 Installation

This model was finetuned using Unsloth. Please refer to their github repository and make sure you have it installed before using the model: Unsloth

💻 Usage Examples

Basic Usage

from unsloth import FastLanguageModel


max_seq_length = 2048 
dtype = None 
load_in_4bit = False 


alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
{}"""


model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "sujet-ai/Sujet-Finance-8B-v0.1",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
    token = "your hf token here",
)


example = {
'system_prompt': 'You are a financial sentiment analysis expert. Your task is to analyze the sentiment expressed in the given financial text.Only reply with bearish, neutral, or bullish.',
'user_prompt': "Expedia's Problems Run Deeper Than SEO Headwinds",
'answer': 'bearish',
}


inputs = tokenizer(
                [alpaca_prompt.format(
                    example['system_prompt'],  # instruction
                    example['user_prompt'],  # input
                    "",  # output - leave this blank for generation!
                )],
                return_tensors="pt"
            ).to("cuda")
            
outputs = model.generate(**inputs, max_new_tokens=2048, use_cache=True, pad_token_id=tokenizer.eos_token_id)
output = tokenizer.batch_decode(outputs)[0]
response = output.split("### Response:")[1].strip()
print(response)

Advanced Usage

You can find more information about the dataset by clicking on this link: Sujet - Finance - Instruct - 177k Dataset

📚 Documentation

🎓 Training Methodology

To ensure optimal performance, a balanced training approach is employed. The dataset preparation process strategically selects an equal number of examples from each subclass within the three focus tasks. The final balanced training dataset consists of 17,036 examples, and the evaluation dataset contains 4,259 examples.

🔧 Technical Details

📊 Evaluation Results

The model is tested by comparing its performance against the base LLAMA 3 model on the evaluation dataset. A response is considered correct if the true answer appears within the first 10 words generated by the model.

📄 License

This model is licensed under the Apache - 2.0 license.