Word Order Jina

🚀 SentenceTransformer based on jinaai/jina-embeddings-v2-base-en

This SentenceTransformer model is finetuned from jinaai/jina-embeddings-v2-base-en on specific datasets. It maps sentences and paragraphs to a 768 - dimensional dense vector space, enabling applications like semantic textual similarity, semantic search, paraphrase mining, text classification, and clustering.

🚀 Quick Start

Prerequisites

First, you need to install the sentence-transformers library.

pip install -U sentence-transformers

Run Inference

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("bwang0911/word-order-jina")
# Run inference
sentences = [
    'Paint preserves wood',
    'Coating protects timber',
    'timber coating protects',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

✨ Features

• Semantic Understanding: Maps sentences and paragraphs to a 768 - dimensional dense vector space for semantic analysis.
• Fine - Tuned: Finetuned on word_orders and negation_dataset datasets.
• Multiple Applications: Can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, and clustering.

📦 Installation

To use this model, you need to install the sentence-transformers library:

pip install -U sentence-transformers

💻 Usage Examples

Basic Usage

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("bwang0911/word-order-jina")
# Run inference
sentences = [
    'Paint preserves wood',
    'Coating protects timber',
    'timber coating protects',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

📚 Documentation

Model Details

Model Description

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: JinaBertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Training Details

Training Datasets

word_orders

• Dataset: word_orders at 99609ac
• Size: 1,002 training samples
• Columns: <code>anchor</code>, <code>pos</code>, and <code>neg</code>
• Approximate statistics based on the first 1000 samples:

  	anchor	pos	neg
  type	string	string	string
  details	min: 5 tokens mean: 12.34 tokens max: 32 tokens	min: 5 tokens mean: 12.1 tokens max: 30 tokens	min: 5 tokens mean: 11.51 tokens max: 24 tokens

• Samples:

  anchor	pos	neg
  The river flows from the mountains to the sea	Water travels from mountain peaks to ocean	The river flows from the sea to the mountains
  Train departs London for Paris	Railway journey from London heading to Paris	Train departs Paris for London
  Cargo ship sails from Shanghai to Singapore	Maritime route Shanghai to Singapore	Cargo ship sails from Singapore to Shanghai

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20,
      "similarity_fct": "cos_sim"
  }
  

negation_dataset

• Dataset: negation_dataset at cd02256
• Size: 10,000 training samples
• Columns: <code>anchor</code>, <code>entailment</code>, and <code>negative</code>
• Approximate statistics based on the first 1000 samples:

  	anchor	entailment	negative
  type	string	string	string
  details	min: 6 tokens mean: 16.48 tokens max: 44 tokens	min: 4 tokens mean: 9.63 tokens max: 31 tokens	min: 5 tokens mean: 10.46 tokens max: 32 tokens

• Samples:

  anchor	entailment	negative
  Two young girls are playing outside in a non-urban environment.	Two girls are playing outside.	Two girls are not playing outside.
  A man with a red shirt is watching another man who is standing on top of a attached cart filled to the top.	A man is standing on top of a cart.	A man is not standing on top of a cart.
  A man in a blue shirt driving a Segway type vehicle.	A person is riding a motorized vehicle.	A person is not riding a motorized vehicle.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non - Default Hyperparameters

• per_device_train_batch_size: 128
• warmup_ratio: 0.1
• fp16: True
• batch_sampler: no_duplicates

All Hyperparameters