Shlm Grc En

🚀 shlm - grc - en

本项目的模型可创建古希腊语和英语文本在共享向量空间中的句子嵌入。它通过特定方法和数据集进行训练，能有效处理这两种语言的句子嵌入任务。

🚀 快速开始

本模型可以使用sentence - transformers库或HuggingFace Transformers库调用，下面为你详细介绍使用方法。

✨ 主要特性

• 该模型能为古希腊语和英语文本在共享向量空间中创建句子嵌入。
• 基础模型采用了Heidelberg - Boston @ SIGTYP 2024 Shared Task: Enhancing Low - Resource Language Analysis With Character - Aware Hierarchical Transformers （arXiv）中描述的HLM架构的修改版本。
• 此模型使用Sentence Embedding Models for Ancient Greek Using Multilingual Knowledge Distillation （arXiv）中描述的多语言知识蒸馏方法和数据集进行训练。
• 该模型是从BAAI/bge - base - en - v1.5蒸馏而来，用于嵌入英语和古希腊语文本。

📦 安装指南

重要提示：本模型目前与sentence - transformers库的最新版本不兼容。

你可以直接使用HuggingFace Transformers，或者使用sentence - transformers的以下分支： https://github.com/kevinkrahn/sentence - transformers

💻 使用示例

基础用法（Sentence - Transformers）

from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SentenceTransformer('kevinkrahn/shlm-grc-en')
embeddings = model.encode(sentences)
print(embeddings)

基础用法（HuggingFace Transformers）

在不使用sentence - transformers的情况下，你可以按以下方式使用该模型：首先，将输入传递给transformer模型，然后对上下文词嵌入应用正确的池化操作。

from transformers import AutoTokenizer, AutoModel
import torch


def cls_pooling(model_output):
    return model_output[0][:,0]


# Sentences we want sentence embeddings for
sentences = ['This is an English sentence', 'Ὁ Παρθενών ἐστιν ἱερὸν καλὸν τῆς Ἀθήνης.']

# Load model from HuggingFace Hub
model = AutoModel.from_pretrained('kevinkrahn/shlm-grc-en', trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained('kevinkrahn/shlm-grc-en', trust_remote_code=True)

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, cls pooling.
sentence_embeddings = cls_pooling(model_output)

print("Sentence embeddings:")
print(sentence_embeddings)

📚 详细文档

如果你使用此模型，请引用以下论文：

@inproceedings{riemenschneider-krahn-2024-heidelberg,
    title = "Heidelberg-Boston @ {SIGTYP} 2024 Shared Task: Enhancing Low-Resource Language Analysis With Character-Aware Hierarchical Transformers",
    author = "Riemenschneider, Frederick  and
      Krahn, Kevin",
    editor = "Hahn, Michael  and
      Sorokin, Alexey  and
      Kumar, Ritesh  and
      Shcherbakov, Andreas  and
      Otmakhova, Yulia  and
      Yang, Jinrui  and
      Serikov, Oleg  and
      Rani, Priya  and
      Ponti, Edoardo M.  and
      Murado{\u{g}}lu, Saliha  and
      Gao, Rena  and
      Cotterell, Ryan  and
      Vylomova, Ekaterina",
    booktitle = "Proceedings of the 6th Workshop on Research in Computational Linguistic Typology and Multilingual NLP",
    month = mar,
    year = "2024",
    address = "St. Julian's, Malta",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.sigtyp-1.16",
    pages = "131--141",
}

@inproceedings{krahn-etal-2023-sentence,
    title = "Sentence Embedding Models for {A}ncient {G}reek Using Multilingual Knowledge Distillation",
    author = "Krahn, Kevin  and
      Tate, Derrick  and
      Lamicela, Andrew C.",
    editor = "Anderson, Adam  and
      Gordin, Shai  and
      Li, Bin  and
      Liu, Yudong  and
      Passarotti, Marco C.",
    booktitle = "Proceedings of the Ancient Language Processing Workshop",
    month = sep,
    year = "2023",
    address = "Varna, Bulgaria",
    publisher = "INCOMA Ltd., Shoumen, Bulgaria",
    url = "https://aclanthology.org/2023.alp-1.2",
    pages = "13--22",
}

📄 许可证

本项目采用MIT许可证。