🚀 Snowflake的Arctic-embed-m-v2.0
Snowflake的Arctic-embed-m-v2.0是一款專為多語言工作負載設計的嵌入模型,在檢索性能和推理效率方面進行了優化。它能在不犧牲英語性能的前提下,實現高質量的多語言文本檢索,適用於需要大規模、可靠的企業級多語言搜索和檢索的應用場景。
🚀 快速開始
使用Sentence Transformers
from sentence_transformers import SentenceTransformer
model_name = 'Snowflake/snowflake-arctic-embed-m-v2.0'
model = SentenceTransformer(model_name, trust_remote_code=True)
queries = ['what is snowflake?', 'Where can I get the best tacos?']
documents = ['The Data Cloud!', 'Mexico City of Course!']
query_embeddings = model.encode(queries, prompt_name="query")
document_embeddings = model.encode(documents)
scores = model.similarity(query_embeddings, document_embeddings)
for query, query_scores in zip(queries, scores):
doc_score_pairs = list(zip(documents, query_scores))
doc_score_pairs = sorted(doc_score_pairs, key=lambda x: x[1], reverse=True)
print("Query:", query)
for document, score in doc_score_pairs:
print(score, document)
使用Huggingface Transformers
import torch
from transformers import AutoModel, AutoTokenizer
model_name = 'Snowflake/snowflake-arctic-embed-m-v2.0'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name, add_pooling_layer=False, trust_remote_code=True)
model.eval()
query_prefix = 'query: '
queries = ['what is snowflake?', 'Where can I get the best tacos?']
queries_with_prefix = ["{}{}".format(query_prefix, i) for i in queries]
query_tokens = tokenizer(queries_with_prefix, padding=True, truncation=True, return_tensors='pt', max_length=8192)
documents = ['The Data Cloud!', 'Mexico City of Course!']
document_tokens = tokenizer(documents, padding=True, truncation=True, return_tensors='pt', max_length=8192)
with torch.no_grad():
query_embeddings = model(**query_tokens)[0][:, 0]
document_embeddings = model(**document_tokens)[0][:, 0]
query_embeddings = torch.nn.functional.normalize(query_embeddings, p=2, dim=1)
document_embeddings = torch.nn.functional.normalize(document_embeddings, p=2, dim=1)
scores = torch.mm(query_embeddings, document_embeddings.transpose(0, 1))
for query, query_scores in zip(queries, scores):
doc_score_pairs = list(zip(documents, query_scores))
doc_score_pairs = sorted(doc_score_pairs, key=lambda x: x[1], reverse=True)
print("Query:", query)
for document, score in doc_score_pairs:
print(score, document)
使用Huggingface Transformers.js
npm i @huggingface/transformers
import { pipeline, dot } from '@huggingface/transformers';
const extractor = await pipeline('feature-extraction', 'Snowflake/snowflake-arctic-embed-m-v2.0');
const sentences = [
'query: what is snowflake?',
'The Data Cloud!',
'Mexico City of Course!',
]
const output = await extractor(sentences, { normalize: true, pooling: 'cls' });
const [source_embeddings, ...document_embeddings ] = output.tolist();
const similarities = document_embeddings.map(x => dot(source_embeddings, x));
console.log(similarities);
✨ 主要特性
- 多語言無妥協:在英語和非英語檢索任務中均表現出色,在MTEB Retrieval、CLEF和MIRACL等基準測試中超越了許多領先的開源和專有模型。
- 推理高效:擁有1.13億非嵌入參數,推理速度快,適用於任何規模的應用。
- 壓縮友好:通過Matryoshka Representation Learning (MRL)和量化感知嵌入訓練,即使嵌入向量低至128字節/向量,仍能實現高質量的檢索。請注意,與v1.5模型類似,此模型的MRL為256維,通過4位量化(例如使用
pq256x4fs
快速掃描FAISS索引或使用與1.5模型一起發佈的示例代碼)可實現高質量的128字節壓縮。
- 長上下文支持:基於GTE-multilingual-base構建,通過RoPE可支持長達8192的上下文窗口。
📦 質量基準
與大多數其他開源模型不同,Arctic-embed-m-v2.0在英語(通過MTEB Retrieval)和多語言(通過MIRACL和CLEF)任務中均表現出色。以下是各模型在不同數據集上的平均NDCG@10數據:
模型名稱 |
參數數量 |
非嵌入參數數量 |
維度 |
BEIR (15) |
MIRACL (4) |
CLEF (聚焦) |
CLEF (完整) |
snowflake-arctic-m-v2.0 |
3.05億 |
1.13億 |
768 |
55.4 |
55.2 |
51.7 |
53.9 |
snowflake-arctic-m |
1.09億 |
8600萬 |
768 |
54.9 |
24.9 |
34.4 |
29.1 |
me5 base |
5.6億 |
3.03億 |
1024 |
51.4 |
54.0 |
43.0 |
34.6 |
bge-m3 (BAAI) |
5.68億 |
3.03億 |
1024 |
48.8 |
56.8 |
40.8 |
41.3 |
gte (Alibaba) |
3.05億 |
1.13億 |
768 |
51.1 |
52.3 |
47.7 |
53.1 |
此外,Arctic-embed-m-v2.0的嵌入向量易於壓縮。通過MRL進行向量截斷可將向量大小縮小3倍,同時質量損失約3%。結合MRL向量和向量壓縮(Int4),可實現每個文檔128字節的檢索。
模型 |
維度 |
BEIR (15) |
相對性能 |
MIRACL (4) |
相對性能 |
CLEF (5) |
相對性能 |
CLEF (完整) |
相對性能 |
snowflake-arctic-m-v2.0 |
768 |
55.4 |
N/A |
55.2 |
N/A |
51.7 |
N/A |
53.9 |
N/A |
snowflake-arctic-m-v2.0 |
256 |
54.4 |
-1.81% |
54.0 |
-2.17% |
50.6 |
-2.13% |
52.3 |
-3.06% |
📄 許可證
Arctic採用Apache-2許可證。發佈的模型可免費用於商業用途。
📞 聯繫我們
如果您對本項目有任何疑問或建議,請隨時打開一個issue或提交pull request。您也可以通過電子郵件聯繫Daniel Campos(daniel.campos@snowflake.com)。