Amber Large

🚀 RetrievaEmbedding-01: AMBER

The AMBER (Adaptive Multitask Bilingual Embedding Representations) is a text embedding model developed by Retrieva, Inc. It's primarily tailored for Japanese text but also offers support for English. The model is trained on diverse Japanese and English datasets and has 315M parameters, falling into the large - size category.

🚀 Quick Start

Install the Required Libraries

First, install the necessary Python libraries using pip:

pip install sentence-transformers sentencepiece

Run Inference

You can load the model and perform inference. You can specify the prompt at inference time by adding an argument called prompt to model.encode. The prompts used in the Japanese benchmark are described in jmteb/tasks, and those used in the English benchmark are in mteb/models/retrieva_en.py.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("retrieva-jp/amber-large")
# Run inference
queries = [
    "自然言語処理とはなんですか？",
    "株式会社レトリバについて教えて",
]
documents = [
    "自然言語処理（しぜんげんごしょり、英語: Natural language processing、略称：NLP）は、人間が日常的に使っている自然言語をコンピュータに処理させる一連の技術であり、人工知能と言語学の一分野である。",
    "株式会社レトリバは、自然言語処理と機械学習を核としたAI技術で組織の課題解決を支援するテクノロジー企業である。",
]

queries_embeddings = model.encode(queries, prompt_name="Retrieval-query")
documents_embeddings = model.encode(documents, prompt_name="Retrieval-passage")

similarities = model.similarity(queries_embeddings, documents_embeddings)
print(similarities.shape)

✨ Features

• Multilingual Support: Primarily designed for Japanese, with optional support for English.
• Prompt - Based Inference: Allows specifying prompts at inference time for task - specific embeddings.
• Sentence Transformer: Based on the Sentence Transformer architecture, enabling effective sentence - level embeddings.

📦 Installation

To use the model, you need to install the required Python libraries. Run the following command:

pip install sentence-transformers sentencepiece

💻 Usage Examples

Basic Usage

The following code demonstrates how to load the model and perform basic inference:

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("retrieva-jp/amber-large")
# Run inference
queries = [
    "自然言語処理とはなんですか？",
    "株式会社レトリバについて教えて",
]
documents = [
    "自然言語処理（しぜんげんごしょり、英語: Natural language processing、略称：NLP）は、人間が日常的に使っている自然言語をコンピュータに処理させる一連の技術であり、人工知能と言語学の一分野である。",
    "株式会社レトリバは、自然言語処理と機械学習を核としたAI技術で組織の課題解決を支援するテクノロジー企業である。",
]

queries_embeddings = model.encode(queries, prompt_name="Retrieval-query")
documents_embeddings = model.encode(documents, prompt_name="Retrieval-passage")

similarities = model.similarity(queries_embeddings, documents_embeddings)
print(similarities.shape)

📚 Documentation

Model Details

• Developed by: Retrieva, Inc.
• Model type: Based on the ModernBERT Architecture.
• Language(s) (NLP): Primarily Japanese (optional support for English).
• License: Apache 2.0
• Finetuned from model: sbintuitions/modernbert-ja-310m
• Model Type: Sentence Transformer
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Training Details

Training Data

Multiple datasets were used for training. For Japanese datasets, selections were made from [llm - jp - eval](https://github.com/llm - jp/llm - jp - eval), [llm - japanese - dataset](https://github.com/masanorihirano/llm - japanese - dataset), and hpprc/emb. For English datasets, some of the datasets from Asai et al. (2023) were mainly used, along with partial data from [the sentence - transformers repository](https://huggingface.co/sentence - transformers) and kilt - tasks. Translation datasets between Japanese and English were also utilized to account for cross - lingual aspects. Synthetic data created by LLM was used for Japanese to ensure sufficient training data.

Evaluation

The model was evaluated on the following benchmarks:

• Japanese Benchmark: JMTEB
• Japanese Retrieval Tasks: JQaRA, JaCWIR, MLDR Japanese Subset
• English Benchmark: [MTEB(eng, v2)](https://github.com/embeddings - benchmark/mteb)

Japanese Benchmark: JMTEB

Japanese Retrieval Tasks: JQaRA, JaCWIR, MLDR Japanese Subset

English Benchmark: MTEB(eng, v2)

🔧 Technical Details

The AMBER model is based on the [sbintuitions/modernbert - ja - 310m](https://huggingface.co/sbintuitions/modernbert - ja - 310m) architecture, which is designed for Japanese text. During training, natural - language prompts were incorporated, enabling the model to generate embeddings tailored to specific tasks.

📄 License

This model is licensed under the Apache 2.0 license.

Citation

BibTeX:

@inproceedings{amber2025,
    title = {インストラクションと複数タスクを利用した日本語向け分散表現モデルの構築},
    author = {勝又智 and 木村大翼 and 西鳥羽二郎},
    booktitle = {言語処理学会第31回年次大会発表論文集},
    year = {2025},
}

More Information

For more details, refer to this link (in Japanese).

Model Card Authors

Satoru Katsumata, Daisuke Kimura, Jiro Nishitoba

Model Card Contact

Contact via pr[at]retrieva.jp