Nomic Xlm 2048

🚀 nomic-xlm-2048：带有RoPE的XLM-Roberta Base

nomic-xlm-2048 是一个经过微调的XLM-Roberta Base模型，它将学习到的位置嵌入替换为旋转位置编码（RoPE），并在 CC100 数据集上进行了10000步的训练。

nomic-xlm-2048 在GLUE和XTREME - R基准测试中，与其他多语言编码器相比具有竞争力。

🚀 快速开始

模型信息

性能对比

GLUE基准测试

XTREME - R基准测试

💻 使用示例

基础用法

from transformers import AutoModelForMaskedLM, AutoConfig, AutoTokenizer, pipeline

tokenizer = AutoTokenizer.from_pretrained('nomic-ai/nomic-xlm-2048') # `nomic-bert-2048` uses the standard BERT tokenizer

config = AutoConfig.from_pretrained('nomic-ai/nomic-xlm-2048', trust_remote_code=True) # the config needs to be passed in
model = AutoModelForMaskedLM.from_pretrained('nomic-ai/nomic-xlm-2048',config=config, trust_remote_code=True)

# To use this model directly for masked language modeling
classifier = pipeline('fill-mask', model=model, tokenizer=tokenizer,device="cpu")

print(classifier("I [MASK] to the store yesterday."))

高级用法

from transformers import AutoConfig, AutoModelForSequenceClassification
model_path = "nomic-ai/nomic-xlm-2048"
config = AutoConfig.from_pretrained(model_path, trust_remote_code=True)
# strict needs to be false here since we're initializing some new params
model = AutoModelForSequenceClassification.from_pretrained(model_path, config=config, trust_remote_code=True, strict=False)

📚 详细文档

加入Nomic社区

• Nomic官网：https://nomic.ai
• Discord社区：https://discord.gg/myY5YDR8z8
• Twitter：https://twitter.com/nomic_ai