Longformer Base 4096 Sentence Transformers All Nli Stsb Quora Nq

## 🚀 SentenceTransformer based on allenai/longformer-base-4096

*This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on multiple datasets. It maps sentences & paragraphs to a 768 - dimensional dense vector space, useful for semantic textual similarity, search, mining, classification, clustering, etc.*

## 🚀 Quick Start
First, install the Sentence Transformers library:
```bash
pip install -U sentence-transformers

Then, you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Leo1212/longformer-base-4096-sentence-transformers-all-nli-stsb-quora-nq")
# Run inference
sentences = [
    'How do I attract a girl?',
    'How can I attract girls?',
    "Why isn't my iPhone 5 charging?",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

✨ Features

• Maps sentences and paragraphs to a 768 - dimensional dense vector space.
• Can be used for various tasks such as semantic textual similarity, semantic search, paraphrase mining, text classification, and clustering.

📦 Installation

To use this model, you need to install the sentence-transformers library:

pip install -U sentence-transformers

💻 Usage Examples

Basic Usage

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Leo1212/longformer-base-4096-sentence-transformers-all-nli-stsb-quora-nq")
# Run inference
sentences = [
    'How do I attract a girl?',
    'How can I attract girls?',
    "Why isn't my iPhone 5 charging?",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

📚 Documentation

Model Details

Model Description

Property	Details
Model Type	Sentence Transformer
Base model	allenai/longformer-base-4096
Maximum Sequence Length	4098 tokens
Output Dimensionality	768 tokens
Similarity Function	Cosine Similarity
Training Datasets	all-nli-pair, all-nli-pair-class, all-nli-pair-score, all-nli-triplet, stsb, quora, natural-questions
Language	en

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 4098, 'do_lower_case': False}) with Transformer model: LongformerModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

🔧 Technical Details

Training Details

Training Datasets

• all-nli-pair: Dataset at d482672, size 10,000 training samples. Loss: MultipleNegativesRankingLoss
• all-nli-pair-class: Dataset at d482672, size 10,000 training samples. Loss: SoftmaxLoss
• all-nli-pair-score: Dataset at d482672, size 10,000 training samples. Loss: CoSENTLoss
• all-nli-triplet: Dataset at d482672, size 10,000 training samples. Loss: MultipleNegativesRankingLoss
• stsb: Dataset at ab7a5ac, size 5,749 training samples. Loss: CoSENTLoss
• quora: Dataset at 451a485, size 10,000 training samples. Loss: MultipleNegativesRankingLoss
• natural-questions: Dataset at f9e894e, size 10,000 training samples. Loss: MultipleNegativesRankingLoss

Evaluation Datasets

• all-nli-triplet: Dataset at d482672, size 6,584 evaluation samples. Loss: MultipleNegativesRankingLoss
• stsb: Dataset at ab7a5ac, size 1,500 evaluation samples. Loss: CoSENTLoss
• quora: Dataset at 451a485, size 1,000 evaluation samples. Loss: MultipleNegativesRankingLoss
• natural-questions: Dataset at f9e894e, size 1,000 evaluation samples. Loss: MultipleNegativesRankingLoss