đ Introducing Sujet Finance 8B v0.1
Sujet Finance 8B v0.1 is a specialized financial language model. It is fine - tuned on the Sujet Finance Instruct - 177k dataset, offering high - quality solutions for various finance - related tasks.
⨠Features
đ¯ Fine - Tuning Focus
In the initial fine - tuning, the model focuses on three key financial tasks:
-
â
â Yes/No Questions
- Description: Answer financial questions with a simple "yes" or "no".
- Class Distribution:
- Train Set: 5,265 "yes" examples, 5,302 "no" examples.
- Eval Set: 1,340 "yes" examples, 1,303 "no" examples.
-
đ Topic Classification
- Description: Classify financial texts into specific finance - related categories like company news, markets, earnings, etc.
- Class Distribution:
- Train Set: Balanced across 20 classes, with 29 - 40 examples per class.
- Eval Set: Varies across classes, ranging from 4 to 15 examples per class.
-
đđđĄ Sentiment Analysis
- Description: Analyze financial texts to categorize sentiments as positive, negative, neutral, bearish, or bullish.
- Class Distribution:
- Train Set: 1,160 positive, 1,155 negative, 1,150 neutral, 1,133 bearish, and 1,185 bullish examples.
- Eval Set: 281 positive, 286 negative, 291 neutral, 308 bearish, and 256 bullish examples.
đĻ Installation
This model was finetuned using Unsloth. Please refer to their github repository and make sure you have it installed before using the model: Unsloth
đģ Usage Examples
Basic Usage
from unsloth import FastLanguageModel
max_seq_length = 2048
dtype = None
load_in_4bit = False
alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
{}
### Input:
{}
### Response:
{}"""
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "sujet-ai/Sujet-Finance-8B-v0.1",
max_seq_length = max_seq_length,
dtype = dtype,
load_in_4bit = load_in_4bit,
token = "your hf token here",
)
example = {
'system_prompt': 'You are a financial sentiment analysis expert. Your task is to analyze the sentiment expressed in the given financial text.Only reply with bearish, neutral, or bullish.',
'user_prompt': "Expedia's Problems Run Deeper Than SEO Headwinds",
'answer': 'bearish',
}
inputs = tokenizer(
[alpaca_prompt.format(
example['system_prompt'],
example['user_prompt'],
"",
)],
return_tensors="pt"
).to("cuda")
outputs = model.generate(**inputs, max_new_tokens=2048, use_cache=True, pad_token_id=tokenizer.eos_token_id)
output = tokenizer.batch_decode(outputs)[0]
response = output.split("### Response:")[1].strip()
print(response)
Advanced Usage
You can find more information about the dataset by clicking on this link: Sujet - Finance - Instruct - 177k Dataset
đ Documentation
đ Training Methodology
To ensure optimal performance, a balanced training approach is employed. The dataset preparation process strategically selects an equal number of examples from each subclass within the three focus tasks. The final balanced training dataset consists of 17,036 examples, and the evaluation dataset contains 4,259 examples.
đ§ Technical Details
Property |
Details |
Base Model |
LLAMA 3 8B đĻ |
Fine - Tuning Technique |
LoRA (Low - Rank Adaptation), r = 16, alpha = 32 |
Learning Rate |
2e - 4 đ |
Weight Decay |
0.01 đī¸ââī¸ |
Epochs |
1 đ |
Quantization |
float16 for VLLM đī¸ |
đ Evaluation Results
The model is tested by comparing its performance against the base LLAMA 3 model on the evaluation dataset. A response is considered correct if the true answer appears within the first 10 words generated by the model.
đ License
This model is licensed under the Apache - 2.0 license.