Declutr Small

🚀 DeCLUTR-small

DeCLUTR-small是一個用於句子相似度任務的模型，它基於對比學習，可作為通用句子編碼器，為文本提供高質量的語義表示。

🚀 快速開始

本模型可作為通用句子編碼器使用，類似於 Google的通用句子編碼器 或 Sentence Transformers。

完整使用細節請參考 我們的倉庫，下面是簡單的使用示例。

💻 使用示例

基礎用法

使用 SentenceTransformers

from scipy.spatial.distance import cosine
from sentence_transformers import SentenceTransformer

# Load the model
model = SentenceTransformer("johngiorgi/declutr-small")

# Prepare some text to embed
texts = [
    "A smiling costumed woman is holding an umbrella.",
    "A happy woman in a fairy costume holds an umbrella.",
]

# Embed the text
embeddings = model.encode(texts)

# Compute a semantic similarity via the cosine distance
semantic_sim = 1 - cosine(embeddings[0], embeddings[1])

使用 🤗 Transformers

import torch
from scipy.spatial.distance import cosine
from transformers import AutoModel, AutoTokenizer

# Load the model
tokenizer = AutoTokenizer.from_pretrained("johngiorgi/declutr-small")
model = AutoModel.from_pretrained("johngiorgi/declutr-small")

# Prepare some text to embed
text = [
    "A smiling costumed woman is holding an umbrella.",
    "A happy woman in a fairy costume holds an umbrella.",
]
inputs = tokenizer(text, padding=True, truncation=True, return_tensors="pt")

# Embed the text
with torch.no_grad():
    sequence_output = model(**inputs)[0]

# Mean pool the token-level embeddings to get sentence-level embeddings
embeddings = torch.sum(
    sequence_output * inputs["attention_mask"].unsqueeze(-1), dim=1
) / torch.clamp(torch.sum(inputs["attention_mask"], dim=1, keepdims=True), min=1e-9)

# Compute a semantic similarity via the cosine distance
semantic_sim = 1 - cosine(embeddings[0], embeddings[1])

📚 詳細文檔

模型描述

“DeCLUTR-small” 模型來自我們的論文：DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations。

預期用途和限制

該模型旨在用作通用句子編碼器，類似於 Google的通用句子編碼器 或 Sentence Transformers。

BibTeX引用和引用信息

@inproceedings{giorgi-etal-2021-declutr,
	title        = {{D}e{CLUTR}: Deep Contrastive Learning for Unsupervised Textual Representations},
	author       = {Giorgi, John  and Nitski, Osvald  and Wang, Bo  and Bader, Gary},
	year         = 2021,
	month        = aug,
	booktitle    = {Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
	publisher    = {Association for Computational Linguistics},
	address      = {Online},
	pages        = {879--895},
	doi          = {10.18653/v1/2021.acl-long.72},
	url          = {https://aclanthology.org/2021.acl-long.72}
}

📄 許可證

本模型使用的許可證為 Apache-2.0。

📦 模型信息