SBERT_XtremeDistil Open-source Lightweight Sentence Embedding Model - Freely Perform Semantic Computation and Text Clustering

Sbert Xtremedistil L6 H256 Uncased Mean Cosine H32

Developed by ABrinkmann

This is a lightweight sentence embedding model based on sentence-transformers, capable of mapping text to a 32-dimensional vector space, suitable for semantic similarity calculation and text clustering tasks.

Text Embedding

PyTorch

#Minimalist Vectorization #Semantic Similarity #Lightweight Embedding

Downloads 17

Release Time : 4/13/2022

Model Overview

This model is an extremely distilled version of a sentence transformer, designed for efficient computation, converting sentences and paragraphs into 32-dimensional dense vector representations, suitable for tasks such as semantic search, clustering, and information retrieval.

Model Features

Efficient and Lightweight

Utilizes extreme distillation technology, resulting in a small model size and high computational efficiency

Low-Dimensional Representation

Maps text to a 32-dimensional vector space, balancing computational efficiency and representational capability

Semantic Encoding

Capable of capturing the semantic information of sentences, suitable for similarity calculation

Model Capabilities

Sentence vectorization

Semantic similarity calculation

Text clustering

Information retrieval

Use Cases

Information Retrieval

Semantic Search

Building search systems based on semantics rather than keywords

Improves the relevance of search results

Text Analysis

Document Clustering

Automatically grouping similar documents

Achieves unsupervised document organization

🚀 ABrinkmann/sbert_xtremedistil-l6-h256-uncased-mean-cosine-h32

This is a sentence-transformers model that maps sentences and paragraphs to a 32-dimensional dense vector space, useful for tasks such as clustering or semantic search.

🚀 Quick Start

This model simplifies usage once you have sentence-transformers installed. First, install the necessary library:

pip install -U sentence-transformers

Then, you can use the model as follows:

💻 Usage Examples

Basic Usage

from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('ABrinkmann/sbert_xtremedistil-l6-h256-uncased-mean-cosine-h32')
embeddings = model.encode(sentences)
print(embeddings)

📚 Documentation

Evaluation Results

For an automated evaluation of this model, refer to the Sentence Embeddings Benchmark: https://seb.sbert.net

Training

The model was trained with the following parameters:

DataLoader: torch.utils.data.dataloader.DataLoader of length 251 with parameters:

{'batch_size': 32, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}

Loss: sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss

Parameters of the fit()-Method:

{
    "epochs": 1,
    "evaluation_steps": 1000,
    "evaluator": "sentence_transformers.evaluation.EmbeddingSimilarityEvaluator.EmbeddingSimilarityEvaluator",
    "max_grad_norm": 1,
    "optimizer_class": "<class 'transformers.optimization.AdamW'>",
    "optimizer_params": {
        "lr": 2e-05
    },
    "scheduler": "WarmupLinear",
    "steps_per_epoch": null,
    "warmup_steps": 26,
    "weight_decay": 0.01
}

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 16, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 256, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
  (2): Dense({'in_features': 256, 'out_features': 32, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
)

📄 License

No license information provided in the original document.

📝 Citing & Authors

No detailed information provided in the original document.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご