Plamo Embedding 1b

🚀 PLaMo-Embedding-1B

PLaMo-Embedding-1B is a Japanese text embedding model developed by Preferred Networks, Inc. It can convert Japanese text input into numerical vectors and is applicable to various tasks such as information retrieval, text classification, and clustering.

🚀 Quick Start

PLaMo-Embedding-1B is a Japanese text embedding model developed by Preferred Networks, Inc. It can convert Japanese text input into numerical vectors and can be used for a wide range of applications, including information retrieval, text classification, and clustering. As of early April 2025, it achieved top - class scores on JMTEB, a benchmark for Japanese text embedding, especially outstanding in retrieval tasks.

✨ Features

• Convert Japanese text into numerical vectors.
• Applicable to multiple tasks like information retrieval, text classification, and clustering.
• Achieved top - class scores on the JMTEB benchmark.

📦 Installation

Requirements

sentencepiece
torch
transformers

💻 Usage Examples

Basic Usage

import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer

# You can download models from the Hugging Face Hub 🤗 as follows:
tokenizer = AutoTokenizer.from_pretrained("pfnet/plamo-embedding-1b", trust_remote_code=True)
model = AutoModel.from_pretrained("pfnet/plamo-embedding-1b", trust_remote_code=True)

device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)

query = "PLaMo-Embedding-1Bとは何ですか？"
documents = [
    "PLaMo-Embedding-1Bは、Preferred Networks, Inc. によって開発された日本語テキスト埋め込みモデルです。",
    "最近は随分と暖かくなりましたね。"
]

with torch.inference_mode():
    # For embedding query texts in information retrieval, please use the `encode_query` method.
    # You also need to pass the `tokenizer`.
    query_embedding = model.encode_query(query, tokenizer)
    # For other texts/sentences, please use the `encode_document` method.
    # Also, for applications other than information retrieval, please use the `encode_document` method.
    document_embeddings = model.encode_document(documents, tokenizer)

# The similarity between vectors obtained by inputting sentences into the model is high for similar sentences and low for dissimilar sentences.
# This feature can be utilized for applications such as information retrieval.
similarities = F.cosine_similarity(query_embedding, document_embeddings)
print(similarities)
# tensor([0.8812, 0.5533])

⚠️ Important Note

For encode_document and encode_query, texts exceeding the model's maximum context length of 4096 will be truncated. Be especially aware that for encode_query, a prefix is added internally, making the effective maximum context length slightly shorter.

📚 Documentation

Benchmarks

We conducted a performance evaluation using JMTEB, a benchmark for Japanese text embedding.

(*): Measured with a context length of 1024. Although the model supports a context length of up to 4096, we measured at 1024 because the context length included during training was up to 1024. However, it is known that evaluating at 4096 does not significantly affect the average score. (Ref: Tech Blog (Ja))

Model Details

📄 License

PLaMo-Embedding-1B is released under the Apache v2.0 license, and you can use it freely, including for commercial purposes.

How to cite

@online{PLaMoEmbedding1B,
    author    = {Preferred Networks, Inc},
    title     = {PLaMo-Embedding-1B},
    year      = {2025},
    url       = {https://huggingface.co/pfnet/plamo-embedding-1b},
    urldate   = {2025-04-17}
}