gte-small-zhオープンソース中国語モデル - 無料デプロイで様々な自然言語処理タスクを支援

ホーム

Gte Small Zh

thenlperによって開発

GTE Small 中国語モデルは中国語テキスト処理に特化した文変換モデルで、様々な自然言語処理タスクに適用可能です。

テキスト埋め込み

Safetensors

英語オープンソースライセンス:MIT #中文文類似度 #医療QA再ランキング #マルチタスクテキスト埋め込み

ダウンロード数 2,789

リリース時間 : 11/8/2023

モデル概要

このモデルは主に文類似度計算、テキスト分類、クラスタリング、検索などのタスクに使用され、特に中国語テキストに最適化されています。

モデル特徴

中国語最適化

中国語テキスト処理に特化して最適化されています

マルチタスク対応

文類似度計算、テキスト分類、クラスタリング、検索など様々なタスクをサポートします

効率的な性能

比較的小さなモデルサイズを維持しながら、合理的な性能を提供します

モデル能力

文類似度計算

テキスト分類

テキストクラスタリング

情報検索

意味マッチング

使用事例

QAシステム

医療QA検索

医療QAシステムでの質問マッチングと回答検索

CMedQAデータセットで優れた性能を発揮（MAP@1: 23.91%）

テキストマッチング

金融問題マッチング

金融分野での問題類似度マッチング

BQデータセットでコサイン類似度ピアソン値49.04%を達成

コンテンツ分類

製品レビュー分類

ECプラットフォームの製品レビューを分類

AmazonReviews分類タスクで精度38.69%を達成

🚀 gte-small-zh

このモデルは、文章の類似性や分類、検索、再ランキングなどのタスクに特化したモデルです。MTEBベンチマークの複数のデータセットで評価され、多様なタスクにおける性能を示しています。

📚 ドキュメント

モデル情報

属性	详情
モデルタイプ	gte-small-zh
タグ	mteb、sentence-similarity、sentence-transformers、Sentence Transformers

評価結果

STSタスク

MTEB AFQMC (validation)
- cos_sim_pearson: 35.80906032378281
- cos_sim_spearman: 36.688967176174415
- euclidean_pearson: 35.70701955438158
- euclidean_spearman: 36.6889470691436
- manhattan_pearson: 35.832741768286944
- manhattan_spearman: 36.831888591957195
MTEB ATEC (test)
- cos_sim_pearson: 44.667266488330384
- cos_sim_spearman: 45.77390794946174
- euclidean_pearson: 48.14272832901943
- euclidean_spearman: 45.77390569666109
- manhattan_pearson: 48.187667158563094
- manhattan_spearman: 45.80979161966117
MTEB LCQMC (test)
- cos_sim_pearson: 66.05467125345538
- cos_sim_spearman: 72.71921060562211
- euclidean_pearson: 71.28539457113986
- euclidean_spearman: 72.71920173126693
- manhattan_pearson: 71.23750818174456
- manhattan_spearman: 72.61025268693467

分類タスク

MTEB AmazonReviewsClassification (zh) (test)
- accuracy: 38.690000000000005
- f1: 36.868257131984016
MTEB IFlyTek (validation)
- accuracy: 41.146594844170835
- f1: 28.544218732704845
MTEB JDReview (test)
- accuracy: 82.83302063789868
- ap: 48.881798834997056
- f1: 77.28655923994657
MTEB MassiveIntentClassification (zh-CN) (test)
- accuracy: 59.27706792199058
- f1: 56.77545011902468
MTEB MassiveScenarioClassification (zh-CN) (test)
- accuracy: 66.47948890383321
- f1: 66.4502180376861

クラスタリングタスク

MTEB CLSClusteringP2P (test)
- v_measure: 39.538126779019755
MTEB CLSClusteringS2S (test)
- v_measure: 37.333105487031766

再ランキングタスク

MTEB CMedQAv1 (test)
- map: 86.08142426347963
- mrr: 88.04269841269841
MTEB CMedQAv2 (test)
- map: 87.25694119382474
- mrr: 89.36853174603175
MTEB MMarcoReranking (dev)
- map: 26.127712982639483
- mrr: 24.87420634920635

検索タスク

MTEB CmedqaRetrieval (dev)
- map_at_1: 23.913999999999998
- map_at_10: 35.913000000000004
- map_at_100: 37.836
- map_at_1000: 37.952000000000005
- map_at_3: 31.845000000000002
- map_at_5: 34.0
- mrr_at_1: 36.884
- mrr_at_10: 44.872
- mrr_at_100: 45.899
- mrr_at_1000: 45.945
- mrr_at_3: 42.331
- mrr_at_5: 43.674
- ndcg_at_1: 36.884
- ndcg_at_10: 42.459
- ndcg_at_100: 50.046
- ndcg_at_1000: 52.092000000000006
- ndcg_at_3: 37.225
- ndcg_at_5: 39.2
- precision_at_1: 36.884
- precision_at_10: 9.562
- precision_at_100: 1.572
- precision_at_1000: 0.183
- precision_at_3: 21.122
- precision_at_5: 15.274
- recall_at_1: 23.913999999999998
- recall_at_10: 52.891999999999996
- recall_at_100: 84.328
- recall_at_1000: 98.168
- recall_at_3: 37.095
- recall_at_5: 43.396
MTEB CovidRetrieval (dev)
- map_at_1: 48.419000000000004
- map_at_10: 57.367999999999995
- map_at_100: 58.081
- map_at_1000: 58.108000000000004
- map_at_3: 55.251
- map_at_5: 56.53399999999999
- mrr_at_1: 48.472
- mrr_at_10: 57.359
- mrr_at_100: 58.055
- mrr_at_1000: 58.082
- mrr_at_3: 55.303999999999995
- mrr_at_5: 56.542
- ndcg_at_1: 48.472
- ndcg_at_10: 61.651999999999994
- ndcg_at_100: 65.257
- ndcg_at_1000: 65.977
- ndcg_at_3: 57.401
- ndcg_at_5: 59.681
- precision_at_1: 48.472
- precision_at_10: 7.576
- precision_at_100: 0.932
- precision_at_1000: 0.099
- precision_at_3: 21.25
- precision_at_5: 13.888
- recall_at_1: 48.419000000000004
- recall_at_10: 74.97399999999999
- recall_at_100: 92.202
- recall_at_1000: 97.893
- recall_at_3: 63.541000000000004
- recall_at_5: 68.994
MTEB DuRetrieval (dev)
- map_at_1: 22.328
- map_at_10: 69.11
- map_at_100: 72.47
- map_at_1000: 72.54599999999999
- map_at_3: 46.938
- map_at_5: 59.56
- mrr_at_1: 81.35
- mrr_at_10: 87.066
- mrr_at_100: 87.212
- mrr_at_1000: 87.21799999999999
- mrr_at_3: 86.558
- mrr_at_5: 86.931
- ndcg_at_1: 81.35
- ndcg_at_10: 78.568
- ndcg_at_100: 82.86099999999999
- ndcg_at_1000: 83.628
- ndcg_at_3: 76.716
- ndcg_at_5: 75.664
- precision_at_1: 81.35
- precision_at_10: 38.545
- precision_at_100: 4.657
- precision_at_1000: 0.484
- precision_at_3: 69.18299999999999
- precision_at_5: 58.67
- recall_at_1: 22.328
- recall_at_10: 80.658
- recall_at_100: 94.093
- recall_at_1000: 98.137
- recall_at_3: 50.260000000000005
- recall_at_5: 66.045
MTEB EcomRetrieval (dev)
- map_at_1: 43.1
- map_at_10: 52.872
- map_at_100: 53.556000000000004
- map_at_1000: 53.583000000000006
- map_at_3: 50.14999999999999
- map_at_5: 51.925
- mrr_at_1: 43.1
- mrr_at_10: 52.872
- mrr_at_100: 53.556000000000004
- mrr_at_1000: 53.583000000000006
- mrr_at_3: 50.14999999999999
- mrr_at_5: 51.925
- ndcg_at_1: 43.1
- ndcg_at_10: 57.907
- ndcg_at_100: 61.517999999999994
- ndcg_at_1000: 62.175000000000004
- ndcg_at_3: 52.425
- ndcg_at_5: 55.631
- precision_at_1: 43.1
- precision_at_10: 7.380000000000001
- precision_at_100: 0.9129999999999999
- precision_at_1000: 0.096
- precision_at_3: 19.667
- precision_at_5: 13.36
- recall_at_1: 43.1
- recall_at_10: 73.8
- recall_at_100: 91.3
- recall_at_1000: 96.39999999999999
- recall_at_3: 59.0
- recall_at_5: 66.8
MTEB MMarcoRetrieval (dev)
- map_at_1: 62.517
- map_at_10: 71.251
- map_at_100: 71.647
- map_at_1000: 71.665
- map_at_3: 69.28
- map_at_5: 70.489
- mrr_at_1: 64.613
- mrr_at_10: 71.89
- mrr_at_100: 72.243
- mrr_at_1000: 72.259
- mrr_at_3: 70.138
- mrr_at_5: 71.232
- ndcg_at_1: 64.613
- ndcg_at_10: 75.005
- ndcg_at_100: 76.805
- ndcg_at_1000: 77.281
- ndcg_at_3: 71.234
- ndcg_at_5: 73.294
- precision_at_1: 64.613
- precision_at_10: 9.142
- precision_at_100: 1.004
- precision_at_1000: 0.104
- precision_at_3: 26.781
- precision_at_5: 17.149
- recall_at_1: 62.517
- recall_at_10: 85.997
- recall_at_100: 94.18299999999999
- recall_at_1000: 97.911
- recall_at_3: 75.993
- recall_at_5: 80.88300000000001
MTEB MedicalRetrieval (dev)
- map_at_1: 54.2
- map_at_10: 59.858
- map_at_100: 60.46
- map_at_1000: 60.507
- map_at_3: 58.41699999999999

ペア分類タスク

MTEB Cmnli (validation)
- cos_sim_accuracy: 68.91160553217077
- cos_sim_ap: 76.45769658379533
- cos_sim_f1: 72.07988702844463
- cos_sim_precision: 63.384779137839274
- cos_sim_recall: 83.53986439092822
- dot_accuracy: 68.91160553217077
- dot_ap: 76.47279917239219
- dot_f1: 72.07988702844463
- dot_precision: 63.384779137839274
- dot_recall: 83.53986439092822
- euclidean_accuracy: 68.91160553217077
- euclidean_ap: 76.45768544225383
- euclidean_f1: 72.07988702844463
- euclidean_precision: 63.384779137839274
- euclidean_recall: 83.53986439092822
- manhattan_accuracy: 69.21226698737222
- manhattan_ap: 76.6623683693766
- manhattan_f1: 72.14058164628506
- manhattan_precision: 64.35643564356435
- manhattan_recall: 82.06686930091185
- max_accuracy: 69.21226698737222
- max_ap: 76.6623683693766
- max_f1: 72.14058164628506