Shlm Grc En

🚀 shlm - grc - en

本項目的模型可創建古希臘語和英語文本在共享向量空間中的句子嵌入。它通過特定方法和數據集進行訓練，能有效處理這兩種語言的句子嵌入任務。

🚀 快速開始

本模型可以使用sentence - transformers庫或HuggingFace Transformers庫調用，下面為你詳細介紹使用方法。

✨ 主要特性

• 該模型能為古希臘語和英語文本在共享向量空間中創建句子嵌入。
• 基礎模型採用了Heidelberg - Boston @ SIGTYP 2024 Shared Task: Enhancing Low - Resource Language Analysis With Character - Aware Hierarchical Transformers （arXiv）中描述的HLM架構的修改版本。
• 此模型使用Sentence Embedding Models for Ancient Greek Using Multilingual Knowledge Distillation （arXiv）中描述的多語言知識蒸餾方法和數據集進行訓練。
• 該模型是從BAAI/bge - base - en - v1.5蒸餾而來，用於嵌入英語和古希臘語文本。

📦 安裝指南

重要提示：本模型目前與sentence - transformers庫的最新版本不兼容。

你可以直接使用HuggingFace Transformers，或者使用sentence - transformers的以下分支： https://github.com/kevinkrahn/sentence - transformers

💻 使用示例

基礎用法（Sentence - Transformers）

from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('kevinkrahn/shlm-grc-en')
embeddings = model.encode(sentences)
print(embeddings)

基礎用法（HuggingFace Transformers）

在不使用sentence - transformers的情況下，你可以按以下方式使用該模型：首先，將輸入傳遞給transformer模型，然後對上下文詞嵌入應用正確的池化操作。

from transformers import AutoTokenizer, AutoModel
import torch


def cls_pooling(model_output):
    return model_output[0][:,0]


# Sentences we want sentence embeddings for
sentences = ['This is an English sentence', 'Ὁ Παρθενών ἐστιν ἱερὸν καλὸν τῆς Ἀθήνης.']

# Load model from HuggingFace Hub
model = AutoModel.from_pretrained('kevinkrahn/shlm-grc-en', trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained('kevinkrahn/shlm-grc-en', trust_remote_code=True)

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, cls pooling.
sentence_embeddings = cls_pooling(model_output)

print("Sentence embeddings:")
print(sentence_embeddings)

📚 詳細文檔

如果你使用此模型，請引用以下論文：

@inproceedings{riemenschneider-krahn-2024-heidelberg,
    title = "Heidelberg-Boston @ {SIGTYP} 2024 Shared Task: Enhancing Low-Resource Language Analysis With Character-Aware Hierarchical Transformers",
    author = "Riemenschneider, Frederick  and
      Krahn, Kevin",
    editor = "Hahn, Michael  and
      Sorokin, Alexey  and
      Kumar, Ritesh  and
      Shcherbakov, Andreas  and
      Otmakhova, Yulia  and
      Yang, Jinrui  and
      Serikov, Oleg  and
      Rani, Priya  and
      Ponti, Edoardo M.  and
      Murado{\u{g}}lu, Saliha  and
      Gao, Rena  and
      Cotterell, Ryan  and
      Vylomova, Ekaterina",
    booktitle = "Proceedings of the 6th Workshop on Research in Computational Linguistic Typology and Multilingual NLP",
    month = mar,
    year = "2024",
    address = "St. Julian's, Malta",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.sigtyp-1.16",
    pages = "131--141",
}

@inproceedings{krahn-etal-2023-sentence,
    title = "Sentence Embedding Models for {A}ncient {G}reek Using Multilingual Knowledge Distillation",
    author = "Krahn, Kevin  and
      Tate, Derrick  and
      Lamicela, Andrew C.",
    editor = "Anderson, Adam  and
      Gordin, Shai  and
      Li, Bin  and
      Liu, Yudong  and
      Passarotti, Marco C.",
    booktitle = "Proceedings of the Ancient Language Processing Workshop",
    month = sep,
    year = "2023",
    address = "Varna, Bulgaria",
    publisher = "INCOMA Ltd., Shoumen, Bulgaria",
    url = "https://aclanthology.org/2023.alp-1.2",
    pages = "13--22",
}

📄 許可證

本項目採用MIT許可證。