Amber Base

🚀 RetrievaEmbedding-01: AMBER

The AMBER (Adaptive Multitask Bilingual Embedding Representations) is a text embedding model developed by Retrieva, Inc. It's primarily crafted for Japanese text but also offers support for English. This model was trained on a diverse range of datasets related to both Japanese and English. With 132M parameters (base size), it's well - equipped for various text - related tasks.

🚀 Quick Start

Install Library

First, you need to install the necessary Python libraries using pip:

pip install sentence-transformers sentencepiece

Run Inference

After installation, you can load the model and perform inference. You can specify the prompt at inference time by adding an argument called prompt to model.encode. The prompts used in the Japanese benchmark are described in jmteb/tasks, and those used in the English benchmark are described in mteb/models/retrieva_en.py.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("retrieva-jp/amber-base")
# Run inference
queries = [
    "自然言語処理とはなんですか？",
    "株式会社レトリバについて教えて",
]
documents = [
    "自然言語処理（しぜんげんごしょり、英語: Natural language processing、略称：NLP）は、人間が日常的に使っている自然言語をコンピュータに処理させる一連の技術であり、人工知能と言語学の一分野である。",
    "株式会社レトリバは、自然言語処理と機械学習を核としたAI技術で組織の課題解決を支援するテクノロジー企業である。",
]

queries_embeddings = model.encode(queries, prompt_name="Retrieval-query")
documents_embeddings = model.encode(documents, prompt_name="Retrieval-passage")

similarities = model.similarity(queries_embeddings, documents_embeddings)
print(similarities.shape)

✨ Features

• Multilingual Support: Primarily designed for Japanese, it also offers optional support for English.
• Adaptive Embeddings: During training, natural - language prompts were included, enabling the model to generate task - specific embeddings.
• Versatile Usage: Can be used for various NLP tasks such as retrieval, classification, and more.

📦 Installation

The installation process mainly involves installing the required Python libraries. You can use the following command:

pip install sentence-transformers sentencepiece

💻 Usage Examples

Basic Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("retrieva-jp/amber-base")
queries = ["自然言語処理とはなんですか？"]
documents = ["自然言語処理（しぜんげんごしょり、英語: Natural language processing、略称：NLP）は、人間が日常的に使っている自然言語をコンピュータに処理させる一連の技術であり、人工知能と言語学の一分野である。"]

queries_embeddings = model.encode(queries, prompt_name="Retrieval-query")
documents_embeddings = model.encode(documents, prompt_name="Retrieval-passage")

similarities = model.similarity(queries_embeddings, documents_embeddings)
print(similarities)

📚 Documentation

Model Details

• Developed by: Retrieva, Inc.
• Model type: Based on the ModernBERT Architecture.
• Language(s) (NLP): Primarily Japanese (optional support for English).
• License: Apache 2.0
• Finetuned from model: sbintuitions/modernbert-ja-130m
• Model Type: Sentence Transformer
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 512 dimensions
• Similarity Function: Cosine Similarity

Training Details

Training Data

For Japanese datasets, data was selected from [llm - jp - eval](https://github.com/llm - jp/llm - jp - eval), [llm - japanese - dataset](https://github.com/masanorihirano/llm - japanese - dataset), and hpprc/emb. For English datasets, some of the datasets used in Asai et al. (2023) were mainly used. Additionally, English datasets from [the sentence - transformers repository](https://huggingface.co/sentence - transformers) and kilt - tasks were partially used. To account for cross - lingual aspects between Japanese and English, translation datasets between the two languages were also utilized. For Japanese, synthetic data created by LLM was used to ensure a sufficient amount of training data.

Evaluation

The model was evaluated on the following benchmarks:

• Japanese Benchmark: JMTEB
• Japanese Retrieval Tasks: JQaRA, JaCWIR, MLDR Japanese Subset
• English Benchmark: [MTEB(eng, v2)](https://github.com/embeddings - benchmark/mteb)

Japanese Benchmark: JMTEB

The Mean (TaskType) in the following leaderboard is equivalent to the Avg. in the original JMTEB leaderboard. The evaluation files are stored in the jmteb directory.

Japanese Retrieval Tasks: JQaRA, JaCWIR, MLDR Japanese Subset

The files for MLDR are stored in the mldr directory. The prompts used in JQaRA and JaCWIR are Retrieval - query and Retrieval - passage described in config_sentence_transformers.json.

English Benchmark: MTEB(eng, v2)

The evaluation files are stored in the mteb directory.

🔧 Technical Details

The AMBER model is based on the [sbintuitions/modernbert - ja - 130m](https://huggingface.co/sbintuitions/modernbert - ja - 130m) architecture. During training, natural - language prompts were incorporated, enabling the model to generate embeddings tailored to specific tasks. This approach allows the model to adapt well to different NLP tasks, whether in Japanese or English.

📄 License

The AMBER model is licensed under the Apache 2.0 license.

Citation

BibTeX:

@inproceedings{amber2025,
    title = {インストラクションと複数タスクを利用した日本語向け分散表現モデルの構築},
    author = {勝又智 and 木村大翼 and 西鳥羽二郎},
    booktitle = {言語処理学会第31回年次大会発表論文集},
    year = {2025},
}

More Information

For more information, please visit https://note.com/retrieva/n/n4ee9d304f44d (in Japanese).

Model Card Authors

Satoru Katsumata, Daisuke Kimura, Jiro Nishitoba

Model Card Contact

You can contact the model card maintainers at pr[at]retrieva.jp