BGE-M3-Financial-Matryoshka Open-source Financial Sentence Embedding Model - Supports Information Retrieval and Semantic Similarity Calculation

Bge M3 Financial Matryoshka

Developed by haophancs

A financial domain sentence embedding model fine-tuned based on BAAI/bge-m3, supporting 1024-dimensional vector representation, suitable for semantic similarity and information retrieval tasks

Text Embedding

Safetensors

EnglishOpen Source License:Apache-2.0 #Financial Semantic Retrieval #Multi-dimensional Embedding #Long Text Encoding

Downloads 34

Release Time : 6/22/2024

Model Overview

This is a financial domain-specific model based on the sentence-transformers framework, capable of mapping text to high-dimensional vector spaces, primarily used for semantic text similarity calculation, information retrieval, and other tasks. The model is optimized for financial texts and is particularly suitable for processing professional content such as financial reports and business documents.

Model Features

Financial Domain Optimization

Specifically fine-tuned for financial texts, performing better when processing professional content such as financial reports and business documents

High-dimensional Vector Representation

Supports 1024-dimensional dense vector representation, capable of capturing richer semantic information

Multi-dimensional Evaluation

The model has been evaluated across multiple dimensions (1024/768/512/384) to ensure performance in different scenarios

Long Text Support

Maximum sequence length of 8192 tokens, suitable for processing longer financial documents

Model Capabilities

Semantic text similarity calculation

Semantic search

Paraphrase mining

Text classification

Text clustering

Financial document analysis

Use Cases

Financial Document Processing

Financial Report Retrieval

Quickly find the most relevant paragraphs from a large number of financial reports based on query questions

Achieved 71.7% accuracy@1 on the test set

Regulatory Document Analysis

Analyze relevant content of specific clauses in regulatory documents

Achieved 83.1% accuracy@3 on the test set

Business Intelligence

Business Report Similarity Analysis

Compare similar content across different business reports

🚀 BGE-M3 Financial Matryoshka

This model, based on sentence-transformers, is fine - tuned from BAAI/bge-m3. It maps sentences and paragraphs into a 1024 - dimensional dense vector space, which can be applied to semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and other tasks.

📚 Documentation

✨ Features

Semantic Understanding: Effectively maps sentences and paragraphs to a 1024 - dimensional dense vector space, enabling accurate semantic analysis.
Versatile Applications: Can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

📦 Installation

First, install the Sentence Transformers library:

pip install -U sentence-transformers

💻 Usage Examples

Basic Usage

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("haophancs/bge-m3-financial-matryoshka")
# Run inference
sentences = [
    'As of January 28, 2024 the net carrying value of our inventories was $1.3 billion, which included provisions for obsolete and damaged inventory of $139.7 million.',
    "What is the status of the company's inventory as of January 28, 2024, in terms of its valuation and provisions for obsolescence?",
    'What is the relationship between the ESG goals and the long - term growth strategy?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

📚 Model Details

Model Description

Property	Details
Model Type	Sentence Transformer
Base model	BAAI/bge-m3
Maximum Sequence Length	8192 tokens
Output Dimensionality	1024 tokens
Similarity Function	Cosine Similarity
Language	en
License	apache - 2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

🔧 Evaluation

Metrics - Information Retrieval

Dataset: dim_1024
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7171
cosine_accuracy@3	0.8314
cosine_accuracy@5	0.87
cosine_accuracy@10	0.9143
cosine_precision@1	0.7171
cosine_precision@3	0.2771
cosine_precision@5	0.174
cosine_precision@10	0.0914
cosine_recall@1	0.7171
cosine_recall@3	0.8314
cosine_recall@5	0.87
cosine_recall@10	0.9143
cosine_ndcg@10	0.8152
cosine_mrr@10	0.7836
cosine_map@100	0.7867

📄 License

This model is licensed under the apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご