Nomic Xlm 2048

🚀 nomic-xlm-2048：帶有RoPE的XLM-Roberta Base

nomic-xlm-2048 是一個經過微調的XLM-Roberta Base模型，它將學習到的位置嵌入替換為旋轉位置編碼（RoPE），並在 CC100 數據集上進行了10000步的訓練。

nomic-xlm-2048 在GLUE和XTREME - R基準測試中，與其他多語言編碼器相比具有競爭力。

🚀 快速開始

模型信息

性能對比

GLUE基準測試

XTREME - R基準測試

💻 使用示例

基礎用法

from transformers import AutoModelForMaskedLM, AutoConfig, AutoTokenizer, pipeline

tokenizer = AutoTokenizer.from_pretrained('nomic-ai/nomic-xlm-2048') # `nomic-bert-2048` uses the standard BERT tokenizer

config = AutoConfig.from_pretrained('nomic-ai/nomic-xlm-2048', trust_remote_code=True) # the config needs to be passed in
model = AutoModelForMaskedLM.from_pretrained('nomic-ai/nomic-xlm-2048',config=config, trust_remote_code=True)

# To use this model directly for masked language modeling
classifier = pipeline('fill-mask', model=model, tokenizer=tokenizer,device="cpu")

print(classifier("I [MASK] to the store yesterday."))

高級用法

from transformers import AutoConfig, AutoModelForSequenceClassification
model_path = "nomic-ai/nomic-xlm-2048"
config = AutoConfig.from_pretrained(model_path, trust_remote_code=True)
# strict needs to be false here since we're initializing some new params
model = AutoModelForSequenceClassification.from_pretrained(model_path, config=config, trust_remote_code=True, strict=False)

📚 詳細文檔

加入Nomic社區

• Nomic官網：https://nomic.ai
• Discord社區：https://discord.gg/myY5YDR8z8
• Twitter：https://twitter.com/nomic_ai