3gpp Embedding Model V0

🚀 BGE_base_3gpp-qa-v2_Matryoshka

這是一個基於 sentence-transformers 的模型，它在 json 數據集上對 BAAI/bge-base-en-v1.5 進行了微調。該模型可以將句子和段落映射到一個 768 維的密集向量空間，可用於語義文本相似度、語義搜索、釋義挖掘、文本分類、聚類等任務。

🚀 快速開始

直接使用（Sentence Transformers）

首先安裝 Sentence Transformers 庫：

pip install -U sentence-transformers

然後，你可以加載這個模型並進行推理。

from sentence_transformers import SentenceTransformer

# 從 🤗 Hub 下載
model = SentenceTransformer("iris49/3gpp-embedding-model-v0")
# 進行推理
sentences = [
    'What types of data structures are supported by the GET request body on the resource described in table 5.2.11.3.4-2, and how do they influence the request?',
    "The data structures supported by the GET request body on the resource are detailed in table 5.2.11.3.4-2. These structures define the format and content of the data that can be sent in the request body. They might include fields such as 'filterCriteria', 'sortOrder', or 'pagination', which influence how the server processes the request and returns the appropriate data.",
    "The specific triggers on the Ro interface that can lead to the termination of the IMS service include: 1) Reception of an unsuccessful Operation Result different from DIAMETER_CREDIT_CONTROL_NOT_APPLICABLE in the Debit/Reserve Units Response message. 2) Reception of an unsuccessful Result Code different from DIAMETER_CREDIT_CONTROL_NOT_APPLICABLE within the multiple units operation in the Debit/Reserve Units Response message when only one instance of the multiple units operation field is used. 3) Execution of the termination action procedure as defined in TS 32.299 when only one instance of the Multiple Unit Operation field is used. 4) Execution of the failure handling procedures when the Failure Action is set to 'Terminate' or 'Retry & Terminate'. 5) Reception in the IMS-GWF of an Abort-Session-Request message from OCS.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# 獲取嵌入向量的相似度得分
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

✨ 主要特性

• 將句子和段落映射到 768 維的密集向量空間。
• 可用於語義文本相似度、語義搜索、釋義挖掘、文本分類、聚類等多種任務。

📦 安裝指南

安裝 Sentence Transformers 庫：

pip install -U sentence-transformers

📚 詳細文檔

模型詳情

模型描述

模型來源

• 文檔：Sentence Transformers 文檔
• 倉庫：GitHub 上的 Sentence Transformers
• Hugging Face：Hugging Face 上的 Sentence Transformers

完整模型架構

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

評估

指標

信息檢索

• 數據集：dim_768、dim_512、dim_256、dim_128 和 dim_64
• 評估方法：使用 InformationRetrievalEvaluator 進行評估

訓練詳情

訓練數據集

json

• 數據集：json
• 大小：56,041 個訓練樣本
• 列：anchor 和 positive
• 基於前 1000 個樣本的近似統計信息： | | anchor | positive | | ---- | ---- | ---- | | 類型 | 字符串 | 字符串 | | 詳情 |
  • 最小值：15 個標記
  • 平均值：30.56 個標記
  • 最大值：66 個標記
  |
  • 最小值：42 個標記
  • 平均值：109.65 個標記
  • 最大值：298 個標記
  |
• 樣本： | anchor | positive | | ---- | ---- | | What does the 'dataStatProps' attribute represent in the 'AnalyticsMetadataInfo' type, and what is its data type? | The 'dataStatProps' attribute in the 'AnalyticsMetadataInfo' type represents a list of dataset statistical properties of the data used to generate the analytics. It is defined as an optional attribute with a data type of 'array(DatasetStatisticalProperty)' and a cardinality of 1..N, meaning it can contain one or more elements. | | Why is it important to have standardized methods for resource management in the Nudm_SubscriberDataManagement Service API? | Standardized methods for resource management in the Nudm_SubscriberDataManagement Service API are important because they ensure uniformity, predictability, and compatibility across different implementations and systems. This standardization facilitates seamless integration, reduces errors, and enhances the efficiency of managing subscriber data, which is critical for maintaining reliable communication services. | | What is the purpose of the Nsmf_PDUSession_SMContextStatusNotify service operation in the context of I-SMF context transfer? | The Nsmf_PDUSession_SMContextStatusNotify service operation is used by the SMF (Session Management Function) to notify its consumers about the status of an SM (Session Management) context related to a PDU (Packet Data Unit) Session. In the context of I-SMF (Intermediate SMF) context transfer, this service operation is used to indicate the transfer of the SM context to a new I-SMF or SMF set. It also allows the SMF to update the SMF-derived CN (Core Network) assisted RAN (Radio Access Network) parameters tuning in the AMF (Access and Mobility Management Function). Additionally, it can report DDN (Downlink Data Notification) failures and provide target DNAI (Data Network Access Identifier) information for the current or next PDU session. |
• 損失函數：MatryoshkaLoss，參數如下：

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

訓練超參數

非默認超參數

• eval_strategy：epoch
• per_device_train_batch_size：32
• per_device_eval_batch_size：16
• gradient_accumulation_steps：16
• learning_rate：2e-05
• num_train_epochs：4
• lr_scheduler_type：cosine
• warmup_ratio：0.1
• fp16：True
• load_best_model_at_end：True
• optim：adamw_torch_fused
• batch_sampler：no_duplicates

所有超參數

訓練日誌

加粗行表示保存的檢查點。

框架版本

• Python：3.11.11
• Sentence Transformers：3.3.1
• Transformers：4.41.2
• PyTorch：2.1.2+cu121
• Accelerate：1.2.1
• Datasets：2.19.1
• Tokenizers：0.19.1

📄 許可證

本模型使用 apache-2.0 許可證。

📖 引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}