DistilRoBERTa Financial Text Sentiment Analysis Model - Open - source and Free to Determine Positive, Negative, and Neutral Tendencies of News

Distilroberta Finetuned Financial Text Classification

Developed by nickmuchi

A financial text sentiment analysis model fine-tuned based on distilroberta-base, used to determine the sentiment tendency (negative, neutral, positive) of financial news.

Text Classification

Transformers

EnglishOpen Source License:Apache-2.0 #Financial Sentiment Analysis #High F1 Score #Pandemic Market Impact

Downloads 1,220

Release Time : 3/2/2022

Model Overview

This model is specifically designed for sentiment analysis in the financial domain, capable of identifying bullish, bearish, or neutral sentiments in text. It addresses data imbalance issues by adjusting class weights and incorporates COVID-19-related data to enhance model generalization.

Model Features

Financial Domain Optimization

Fine-tuned specifically for financial texts, accurately identifying market sentiment changes.

Pandemic Data Enhancement

Includes data on the impact of COVID-19 on markets, improving analysis capabilities during special periods.

Class Weight Adjustment

Adjusts weights to address data imbalance, improving recognition accuracy for minority classes.

Efficient and Lightweight

Based on the distilled DistilRoBERTa model, reducing computational resource requirements while maintaining performance.

Model Capabilities

Financial Text Sentiment Classification

Market Sentiment Analysis

News Sentiment Tendency Judgment

Use Cases

Financial Analysis

Market Sentiment Monitoring

Real-time analysis of sentiment tendencies in financial news and market comments.

Accuracy 89%, F1 score 0.8835

Investment Decision Support

Identifies potential market sentiment changes to provide references for investment decisions.

Risk Warning

Detects financial topics with concentrated negative sentiment for early risk warnings.

🚀 distilroberta-finetuned-financial-text-classification

This model is fine - tuned for financial text classification, aiming to accurately determine the sentiment of financial news.

🚀 Quick Start

This model is a fine - tuned version of [distilroberta - base](https://huggingface.co/distilroberta - base) on the sentence_50Agree [financial - phrasebank + Kaggle Dataset](https://huggingface.co/datasets/nickmuchi/financial - classification), a dataset consisting of 4840 Financial News categorised by sentiment (negative, neutral, positive). The Kaggle dataset includes Covid - 19 sentiment data and can be found here: [sentiment - classification - selflabel - dataset](https://www.kaggle.com/percyzheng/sentiment - classification - selflabel - dataset). It achieves the following results on the evaluation set:

Loss: 0.4463
F1: 0.8835

✨ Features

Determines the financial sentiment of given text.
Adjusts weights to handle unbalanced class labels, potentially improving overall performance.
Incorporates a Covid dataset to enrich the model's understanding of the impact of Covid - 19 on earnings or markets.

📚 Documentation

Model description

Model determines the financial sentiment of given text. Given the unbalanced distribution of the class labels, the weights were adjusted to pay attention to the less sampled labels which should increase overall performance. The Covid dataset was added in order to enrich the model, given most models have not been trained on the impact of Covid - 19 on earnings or markets.

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e - 05
train_batch_size: 64
eval_batch_size: 64
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
lr_scheduler_type: linear
num_epochs: 10
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	F1
0.7309	1.0	72	0.3671	0.8441
0.3757	2.0	144	0.3199	0.8709
0.3054	3.0	216	0.3096	0.8678
0.2229	4.0	288	0.3776	0.8390
0.1744	5.0	360	0.3678	0.8723
0.1436	6.0	432	0.3728	0.8758
0.1044	7.0	504	0.4116	0.8744
0.0931	8.0	576	0.4148	0.8761
0.0683	9.0	648	0.4423	0.8837
0.0611	10.0	720	0.4463	0.8835

Framework versions

Transformers 4.15.0
Pytorch 1.10.0+cu111
Datasets 1.18.0
Tokenizers 0.10.3

📄 License

This model is released under the Apache 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご