Glucose Base Ja

🚀 GLuCoSE (General Luke-based Contrastive Sentence Embedding)-base-Japanese

GLuCoSE (General LUke-based COntrastive Sentence Embedding, "glucose") is a Japanese text embedding model based on LUKE. It's trained on a mix of web data and various datasets related to natural language inference and search, aiming to create a general - purpose and user - friendly Japanese text embedding model. This model is suitable for both sentence vector similarity tasks and semantic search tasks.

🚀 Quick Start

You can use this model easily with sentence-transformers.

📦 Installation

First, install sentence-transformers with pip as follows:

pip install -U sentence-transformers

💻 Usage Examples

Basic Usage

from sentence_transformers import SentenceTransformer
sentences = [
    "PKSHA Technologyは機械学習/深層学習技術に関わるアルゴリズムソリューションを展開している。",
    "この深層学習モデルはPKSHA Technologyによって学習され、公開された。",
    "広目天は、仏教における四天王の一尊であり、サンスクリット語の「種々の眼をした者」を名前の由来とする。",
]

model = SentenceTransformer('pkshatech/GLuCoSE-base-ja')
embeddings = model.encode(sentences)
print(embeddings)

⚠️ Important Note

Since the loss function used during training is cosine similarity, we recommend using cosine similarity for downstream tasks.

💡 Usage Tip

This text embedding model can also be used in LangChain. Please refer to this page for more information.

📚 Documentation

Resources Used

The following resources were used to train this model.

Pre-trained model

• studio-ousia/luke-japanese-base-lite

Datasets

• mC4
• MQA
• JNLI
• JSNLI
• PAWS-X
• JSeM
• MoritzLaurer/multilingual-NLI-26lang-2mil7
  • MultiNLI
  • WANLI
  • FeverNLI
  • LingNLI
• JSICK
• Mr.Tidy
• JSTS (used for validation) [^1]

Benchmarks

Semantic Similarity Calculation (JSTS dev set)

Evaluation by Spearman's correlation coefficient and Pearson's correlation coefficient.

Zero-shot Search (AIO3 dev set)

Evaluation by top-k retrieval accuracy[^4] (the fraction of questions that have a correct answer in the top-k retrieved documents at least once.)

👨‍💻 Authors

Akihiko Fukuchi, Yuichiro Hoshino, Yotarow Watanabe

📄 License

This model is published under the Apache License, Version 2.0.

[^1]: When we trained this model, the test data of JGLUE was not released, so we used the dev set of JGLUE as a private evaluation data. Therefore, we selected the checkpoint on the train set of JGLUE insted of its dev set.
[^2]: https://qiita.com/akeyhero/items/ce371bfed64399027c23
[^3]: This is the model we have released before.
[^4]: For more details, please refer to https://arxiv.org/pdf/2004.04906.pdf.