Ko Core News Lg
模型概述
spaCy的大型韓語處理模型,基於UD Korean Kaist和KLUE數據集訓練,支持完整的NLP處理流程,包括分詞、詞性標註、依存句法分析、命名實體識別等任務。
模型特點
完整的韓語處理流程
提供從分詞到命名實體識別的完整NLP處理能力
CPU優化
專門針對CPU環境進行優化,適合資源受限的場景
高質量詞向量
包含floret詞向量(200000, 300),支持更好的語義理解
多任務處理
同時支持詞性標註、依存分析、命名實體識別等多種NLP任務
模型能力
分詞
詞性標註
依存句法分析
命名實體識別
詞形還原
句子邊界檢測
使用案例
文本處理
韓語文本分析
對韓語新聞、社交媒體內容進行語法分析和結構解析
準確識別句子成分和語法關係
信息提取
從韓語文本中提取命名實體(人名、地名、組織名等)
F1值達到85.19%
語言學研究
韓語語法研究
分析韓語句法結構和詞法特徵
提供詳細的詞性標註和依存關係分析
🚀 ko_core_news_lg 韓語語言模型
ko_core_news_lg
是一款專為 CPU 優化的韓語處理管道,可用於詞性標註、命名實體識別、句法分析等多種自然語言處理任務,能有效提升韓語相關文本處理的效率和準確性。
📚 詳細文檔
模型詳情
詳情請見:https://spacy.io/models/ko#ko_core_news_lg
該韓語處理管道針對 CPU 進行了優化,包含以下組件:tok2vec
、tagger
、morphologizer
、parser
、lemmatizer
(可訓練詞形還原器)、senter
、ner
。
屬性 | 詳情 |
---|---|
模型類型 | ko_core_news_lg |
版本 | 3.7.0 |
spaCy 版本要求 | >=3.7.0,<3.8.0 |
默認管道 | tok2vec , tagger , morphologizer , parser , lemmatizer , attribute_ruler , ner |
組件 | tok2vec , tagger , morphologizer , parser , lemmatizer , senter , attribute_ruler , ner |
向量 | floret (200000, 300) |
訓練數據來源 | UD Korean Kaist v2.8 (Choi, Jinho; Han, Na - Rae; Hwang, Jena; Chun, Jayeol) [KLUE v1.1.0](https://github.com/KLUE - benchmark/KLUE) (Sungjoon Park, Jihyung Moon, Sungdong Kim, Won Ik Cho, Jiyoon Han, Jangwon Park, Chisung Song, Junseong Kim, Youngsook Song, Taehwan Oh, Joohong Lee, Juhyun Oh, Sungwon Ryu, Younghoon Jeong, Inkwon Lee, Sangwoo Seo, Dongjun Lee, Hyunwoo Kim, Myeonghwa Lee, Seongbo Jang, Seungwon Do, Sunkyoung Kim, Kyungtae Lim, Jongwon Lee, Kyumin Park, Jamin Shin, Seonghyun Kim, Lucy Park, Alice Oh, Jung - Woo Ha, Kyunghyun Cho) [Explosion Vectors (OSCAR 2109 + Wikipedia + OpenSubtitles + WMT News Crawl)](https://github.com/explosion/spacy - vectors - builder) (Explosion) |
許可證 | CC BY - SA 4.0 |
作者 | Explosion |
標籤方案
查看標籤方案(4 個組件共 2028 個標籤)
組件 | 標籤 |
---|---|
tagger |
_SP , ecs , etm , f , f+f+jcj , f+f+jcs , f+f+jct , f+f+jxt , f+jca , f+jca+jp+ecc , f+jca+jp+ep+ef , f+jca+jxc , f+jca+jxc+jcm , f+jca+jxt , f+jcj , f+jcm , f+jco , f+jcs , f+jct , f+jct+jcm , f+jp+ef , f+jp+ep+ef , f+jp+etm , f+jxc , f+jxt , f+ncn , f+ncn+jcm , f+ncn+jcs , f+ncn+jp+ecc , f+ncn+jxt , f+ncpa+jcm , f+npp+jcs , f+nq , f+xsn , f+xsn+jco , f+xsn+jxt , ii , jca , jca+jcm , jca+jxc , jca+jxt , jcc , jcj , jcm , jco , jcr , jcr+jxc , jcs , jct , jct+jcm , jct+jxt , jp+ecc , jp+ecs , jp+ef , jp+ef+jcr , jp+ef+jcr+jxc , jp+ep+ecs , jp+ep+ef , jp+ep+etm , jp+ep+etn , jp+etm , jp+etn , jp+etn+jco , jp+etn+jxc , jxc , jxc+jca , jxc+jco , jxc+jcs , jxt , mad , mad+jxc , mad+jxt , mag , mag+jca , mag+jcm , mag+jcs , mag+jp+ef+jcr , mag+jxc , mag+jxc+jxc , mag+jxt , mag+xsn , maj , maj+jxc , maj+jxt , mma , mmd , nbn , nbn+jca , nbn+jca+jcj , nbn+jca+jcm , nbn+jca+jp+ef , nbn+jca+jxc , nbn+jca+jxt , nbn+jcc , nbn+jcj , nbn+jcm , nbn+jco , nbn+jcr , nbn+jcs , nbn+jct , nbn+jct+jcm , nbn+jct+jxt , nbn+jp+ecc , nbn+jp+ecs , nbn+jp+ecs+jca , nbn+jp+ecs+jcm , nbn+jp+ecs+jco , nbn+jp+ecs+jxc , nbn+jp+ecs+jxt , nbn+jp+ecx , nbn+jp+ef , nbn+jp+ef+jca , nbn+jp+ef+jco , nbn+jp+ef+jcr , nbn+jp+ef+jcr+jxc , nbn+jp+ef+jcr+jxt , nbn+jp+ef+jcs , nbn+jp+ef+jxc , nbn+jp+ef+jxc+jco , nbn+jp+ef+jxf , nbn+jp+ef+jxt , nbn+jp+ep+ecc , nbn+jp+ep+ecs , nbn+jp+ep+ecs+jxc , nbn+jp+ep+ef , nbn+jp+ep+ef+jcr , nbn+jp+ep+etm , nbn+jp+ep+etn , nbn+jp+ep+etn+jco , nbn+jp+ep+etn+jcs , nbn+jp+etm , nbn+jp+etn , nbn+jp+etn+jca , nbn+jp+etn+jca+jxt , nbn+jp+etn+jco , nbn+jp+etn+jcs , nbn+jp+etn+jxc , nbn+jp+etn+jxt , nbn+jxc , nbn+jxc+jca , nbn+jxc+jca+jxc , nbn+jxc+jca+jxt , nbn+jxc+jcc , nbn+jxc+jcm , nbn+jxc+jco , nbn+jxc+jcs , nbn+jxc+jp+ef , nbn+jxc+jxc , nbn+jxc+jxt , nbn+jxt , nbn+nbn , nbn+nbn+jp+ef , nbn+xsm+ecs , nbn+xsm+ef , nbn+xsm+ep+ef , nbn+xsm+ep+ef+jcr , nbn+xsm+etm , nbn+xsn , nbn+xsn+jca , nbn+xsn+jca+jp+ef+jcr , nbn+xsn+jca+jxc , nbn+xsn+jca+jxt , nbn+xsn+jcm , nbn+xsn+jco , nbn+xsn+jcs , nbn+xsn+jct , nbn+xsn+jp+ecc , nbn+xsn+jp+ecs , nbn+xsn+jp+ef , nbn+xsn+jp+ef+jcr , nbn+xsn+jp+ep+ef , nbn+xsn+jxc , nbn+xsn+jxt , nbn+xsv+etm , nbu , nbu+jca , nbu+jca+jxc , nbu+jca+jxt , nbu+jcc , nbu+jcc+jxc , nbu+jcj , nbu+jcm , nbu+jco , nbu+jcs , nbu+jct , nbu+jct+jxc , nbu+jp+ecc , nbu+jp+ecs , nbu+jp+ef , nbu+jp+ef+jcr , nbu+jp+ef+jxc , nbu+jp+ep+ecc , nbu+jp+ep+ecs , nbu+jp+ep+ef , nbu+jp+ep+ef+jcr , nbu+jp+ep+etm , nbu+jp+ep+etn+jco , nbu+jp+etm , nbu+jxc , nbu+jxc+jca , nbu+jxc+jcs , nbu+jxc+jp+ef , nbu+jxc+jp+ep+ef , nbu+jxc+jxt , nbu+jxt , nbu+ncn , nbu+ncn+jca , nbu+ncn+jcm , nbu+xsn , nbu+xsn+jca , nbu+xsn+jca+jxc , nbu+xsn+jca+jxt , nbu+xsn+jcm , nbu+xsn+jco , nbu+xsn+jcs , nbu+xsn+jp+ecs , nbu+xsn+jp+ep+ef , nbu+xsn+jxc , nbu+xsn+jxc+jxt , nbu+xsn+jxt , nbu+xsv+ecc , nbu+xsv+etm , ncn , ncn+f+ncpa+jco , ncn+jca , ncn+jca+jca , ncn+jca+jcc , ncn+jca+jcj , ncn+jca+jcm , ncn+jca+jcs , ncn+jca+jct , ncn+jca+jp+ecc , ncn+jca+jp+ecs , ncn+jca+jp+ef , ncn+jca+jp+ep+ef , ncn+jca+jp+etm , ncn+jca+jp+etn+jxt , ncn+jca+jxc , ncn+jca+jxc+jcc , ncn+jca+jxc+jcm , ncn+jca+jxc+jxc , ncn+jca+jxc+jxt , ncn+jca+jxt , ncn+jcc , ncn+jcc+jxc , ncn+jcj , ncn+jcj+jxt , ncn+jcm , ncn+jco , ncn+jcr , ncn+jcr+jxc , ncn+jcs , ncn+jcs+jxt , ncn+jct , ncn+jct+jcm , ncn+jct+jxc , ncn+jct+jxt , ncn+jcv , ncn+jp+ecc , ncn+jp+ecc+jct , ncn+jp+ecc+jxc , ncn+jp+ecs , ncn+jp+ecs+jcm , ncn+jp+ecs+jco , ncn+jp+ecs+jxc , ncn+jp+ecs+jxt , ncn+jp+ecx , ncn+jp+ef , ncn+jp+ef+jca , ncn+jp+ef+jcm , ncn+jp+ef+jco , ncn+jp+ef+jcr , ncn+jp+ef+jcr+jxc , ncn+jp+ef+jcr+jxt , ncn+jp+ef+jp+etm , ncn+jp+ef+jxc , ncn+jp+ef+jxf , ncn+jp+ef+jxt , ncn+jp+ep+ecc , ncn+jp+ep+ecs , ncn+jp+ep+ecs+jxc , ncn+jp+ep+ecx , ncn+jp+ep+ef , ncn+jp+ep+ef+jcr , ncn+jp+ep+ef+jcr+jxc , ncn+jp+ep+ef+jxc , ncn+jp+ep+ef+jxf , ncn+jp+ep+ef+jxt , ncn+jp+ep+ep+etm , ncn+jp+ep+etm , ncn+jp+ep+etn , ncn+jp+ep+etn+jca , ncn+jp+ep+etn+jca+jxc , ncn+jp+ep+etn+jco , ncn+jp+ep+etn+jcs , ncn+jp+ep+etn+jxt , ncn+jp+etm , ncn+jp+etn , ncn+jp+etn+jca , ncn+jp+etn+jca+jxc , ncn+jp+etn+jca+jxt , ncn+jp+etn+jco , ncn+jp+etn+jcs , ncn+jp+etn+jct , ncn+jp+etn+jxc , ncn+jp+etn+jxt , ncn+jxc , ncn+jxc+jca , ncn+jxc+jca+jxc , ncn+jxc+jca+jxt , ncn+jxc+jcc , ncn+jxc+jcm , ncn+jxc+jco , ncn+jxc+jcs , ncn+jxc+jct+jxt , ncn+jxc+jp+ef , ncn+jxc+jp+ef+jcr , ncn+jxc+jp+ep+ecs , ncn+jxc+jp+ep+ef , ncn+jxc+jp+etm , ncn+jxc+jxc , ncn+jxc+jxt , ncn+jxt , ncn+jxt+jcm , ncn+jxt+jxc , ncn+nbn , ncn+nbn+jca , ncn+nbn+jcm , ncn+nbn+jcs , ncn+nbn+jp+ecc , ncn+nbn+jp+ep+ef , ncn+nbn+jxc , ncn+nbn+jxt , ncn+nbu , ncn+nbu+jca , ncn+nbu+jcm , ncn+nbu+jco , ncn+nbu+jp+ef , ncn+nbu+jxc , ncn+nbu+ncn , ncn+ncn , ncn+ncn+jca , ncn+ncn+jca+jcc , ncn+ncn+jca+jcm , ncn+ncn+jca+jxc , ncn+ncn+jca+jxc+jcm , ncn+ncn+jca+jxc+jxc , ncn+ncn+jca+jxt , ncn+ncn+jcc , ncn+ncn+jcj , ncn+ncn+jcm , ncn+ncn+jco , ncn+ncn+jcr , ncn+ncn+jcs , ncn+ncn+jct , ncn+ncn+jct+jcm , ncn+ncn+jct+jxc , ncn+ncn+jct+jxt , ncn+ncn+jp+ecc , ncn+ncn+jp+ecs , ncn+ncn+jp+ef , ncn+ncn+jp+ef+jcm , ncn+ncn+jp+ef+jcr , ncn+ncn+jp+ef+jcs , ncn+ncn+jp+ep+ecc , ncn+ncn+jp+ep+ecs , ncn+ncn+jp+ep+ef , ncn+ncn+jp+ep+ef+jcr , ncn+ncn+jp+ep+ep+etm , ncn+ncn+jp+ep+etm , ncn+ncn+jp+ep+etn , ncn+ncn+jp+etm , ncn+ncn+jp+etn , ncn+ncn+jp+etn+jca , ncn+ncn+jp+etn+jco , ncn+ncn+jp+etn+jxc , ncn+ncn+jxc , ncn+ncn+jxc+jca , ncn+ncn+jxc+jcc , ncn+ncn+jxc+jcm , ncn+ncn+jxc+jco , ncn+ncn+jxc+jcs , ncn+ncn+jxc+jxc , ncn+ncn+jxt , ncn+ncn+nbn , ncn+ncn+ncn , ncn+ncn+ncn+jca , ncn+ncn+ncn+jca+jcm , ncn+ncn+ncn+jca+jxt , ncn+ncn+ncn+jcj , ncn+ncn+ncn+jcm , ncn+ncn+ncn+jco , ncn+ncn+ncn+jcs , ncn+ncn+ncn+jct+jxt , ncn+ncn+ncn+jp+etn+jxc , ncn+ncn+ncn+jxt , ncn+ncn+ncn+ncn+jca , ncn+ncn+ncn+ncn+jca+jxt , ncn+ncn+ncn+ncn+jco , ncn+ncn+ncn+xsn+jp+etm , ncn+ncn+ncpa , ncn+ncn+ncpa+jca , ncn+ncn+ncpa+jcm , ncn+ncn+ncpa+jco , ncn+ncn+ncpa+jcs , ncn+ncn+ncpa+jxc , ncn+ncn+ncpa+jxt , ncn+ncn+ncpa+ncn , ncn+ncn+ncpa+ncn+jca , ncn+ncn+ncpa+ncn+jcj , ncn+ncn+ncpa+ncn+jcm , ncn+ncn+ncpa+ncn+jxt , ncn+ncn+xsn , ncn+ncn+xsn+jca , ncn+ncn+xsn+jca+jxt , ncn+ncn+xsn+jcj , ncn+ncn+xsn+jcm , ncn+ncn+xsn+jco , ncn+ncn+xsn+jcs , ncn+ncn+xsn+jct , ncn+ncn+xsn+jp+ecs , ncn+ncn+xsn+jp+ep+ef , ncn+ncn+xsn+jp+etm , ncn+ncn+xsn+jxc , ncn+ncn+xsn+jxc+jcs , ncn+ncn+xsn+jxt , ncn+ncn+xsv+ecc , ncn+ncn+xsv+etm , ncn+ncpa , ncn+ncpa+jca , ncn+ncpa+jca+jcm , ncn+ncpa+jca+jxc , ncn+ncpa+jca+jxt , ncn+ncpa+jcc , ncn+ncpa+jcj , ncn+ncpa+jcm , ncn+ncpa+jco , ncn+ncpa+jcr , ncn+ncpa+jcs , ncn+ncpa+jct , ncn+ncpa+jct+jcm , ncn+ncpa+jct+jxt , ncn+ncpa+jp+ecc , ncn+ncpa+jp+ecc+jxc , ncn+ncpa+jp+ecs , ncn+ncpa+jp+ecs+jxc , ncn+ncpa+jp+ef , ncn+ncpa+jp+ef+jcr , ncn+ncpa+jp+ef+jcr+jxc , ncn+ncpa+jp+ep+ef , ncn+ncpa+jp+ep+etm , ncn+ncpa+jp+ep+etn , ncn+ncpa+jp+etm , ncn+ncpa+jxc , ncn+ncpa+jxc+jca+jxc , ncn+ncpa+jxc+jco , ncn+ncpa+jxc+jcs , ncn+ncpa+jxt , ncn+ncpa+nbn+jcs , ncn+ncpa+ncn , ncn+ncpa+ncn+jca , ncn+ncpa+ncn+jca+jcm , ncn+ncpa+ncn+jca+jxc , ncn+ncpa+ncn+jca+jxt , ncn+ncpa+ncn+jcj , ncn+ncpa+ncn+jcm , ncn+ncpa+ncn+jco , ncn+ncpa+ncn+jcs , ncn+ncpa+ncn+jct , ncn+ncpa+ncn+jct+jcm , ncn+ncpa+ncn+jp+ef+jcr , ncn+ncpa+ncn+jp+ep+etm , ncn+ncpa+ncn+jxc , ncn+ncpa+ncn+jxt , ncn+ncpa+ncn+xsn+jcm , ncn+ncpa+ncn+xsn+jxt , ncn+ncpa+ncpa , ncn+ncpa+ncpa+jca , ncn+ncpa+ncpa+jcj , ncn+ncpa+ncpa+jcm , ncn+ncpa+ncpa+jco , ncn+ncpa+ncpa+jcs , ncn+ncpa+ncpa+jp+ep+ef , ncn+ncpa+ncpa+jxt , ncn+ncpa+ncpa+ncn , ncn+ncpa+xsn , ncn+ncpa+xsn+jcm , ncn+ncpa+xsn+jco , ncn+ncpa+xsn+jcs , ncn+ncpa+xsn+jp+ecc , ncn+ncpa+xsn+jp+etm , ncn+ncpa+xsn+jxt , ncn+ncpa+xsv+ecc , ncn+ncpa+xsv+ecs , ncn+ncpa+xsv+ecx , ncn+ncpa+xsv+ecx+px+etm , ncn+ncpa+xsv+ef , ncn+ncpa+xsv+ef+jcm , ncn+ncpa+xsv+ef+jcr , ncn+ncpa+xsv+etm , (截斷:完整列表見管道元數據) |
morphologizer |
POS=CCONJ , POS=ADV , POS=SCONJ , POS=DET , POS=NOUN , POS=VERB , POS=ADJ , POS=PUNCT , POS=SPACE , POS=AUX , POS=PRON , POS=PROPN , POS=NUM , POS=INTJ , POS=PART , POS=X , POS=ADP , POS=SYM |
parser |
ROOT , acl , advcl , advmod , amod , appos , aux , case , cc , ccomp , compound , conj , cop , csubj , dep , det , dislocated , fixed , flat , iobj , mark , nmod , nsubj , nummod , obj , obl , punct , xcomp |
ner |
DT , LC , OG , PS , QT , TI |
準確率
類型 | 得分 |
---|---|
TOKEN_ACC |
100.00 |
TOKEN_P |
100.00 |
TOKEN_R |
100.00 |
TOKEN_F |
100.00 |
TAG_ACC |
84.00 |
POS_ACC |
94.88 |
SENTS_P |
100.00 |
SENTS_R |
100.00 |
SENTS_F |
100.00 |
DEP_UAS |
84.17 |
DEP_LAS |
81.40 |
LEMMA_ACC |
90.09 |
ENTS_P |
86.69 |
ENTS_R |
83.73 |
ENTS_F |
85.19 |
📄 許可證
本項目採用 CC BY - SA 4.0
許可證。
Indonesian Roberta Base Posp Tagger
MIT
這是一個基於印尼語RoBERTa模型微調的詞性標註模型,在indonlu數據集上訓練,用於印尼語文本的詞性標註任務。
序列標註
Transformers 其他

I
w11wo
2.2M
7
Bert Base NER
MIT
基於BERT微調的命名實體識別模型,可識別四類實體:地點(LOC)、組織機構(ORG)、人名(PER)和雜項(MISC)
序列標註 英語
B
dslim
1.8M
592
Deid Roberta I2b2
MIT
該模型是基於RoBERTa微調的序列標註模型,用於識別和移除醫療記錄中的受保護健康信息(PHI/PII)。
序列標註
Transformers 支持多種語言

D
obi
1.1M
33
Ner English Fast
Flair自帶的英文快速4類命名實體識別模型,基於Flair嵌入和LSTM-CRF架構,在CoNLL-03數據集上達到92.92的F1分數。
序列標註
PyTorch 英語
N
flair
978.01k
24
French Camembert Postag Model
基於Camembert-base的法語詞性標註模型,使用free-french-treebank數據集訓練
序列標註
Transformers 法語

F
gilf
950.03k
9
Xlm Roberta Large Ner Spanish
基於XLM-Roberta-large架構微調的西班牙語命名實體識別模型,在CoNLL-2002數據集上表現優異。
序列標註
Transformers 西班牙語

X
MMG
767.35k
29
Nusabert Ner V1.3
MIT
基於NusaBert-v1.3在印尼語NER任務上微調的命名實體識別模型
序列標註
Transformers 其他

N
cahya
759.09k
3
Ner English Large
Flair框架內置的英文4類大型NER模型,基於文檔級XLM-R嵌入和FLERT技術,在CoNLL-03數據集上F1分數達94.36。
序列標註
PyTorch 英語
N
flair
749.04k
44
Punctuate All
MIT
基於xlm-roberta-base微調的多語言標點符號預測模型,支持12種歐洲語言的標點符號自動補全
序列標註
Transformers

P
kredor
728.70k
20
Xlm Roberta Ner Japanese
MIT
基於xlm-roberta-base微調的日語命名實體識別模型
序列標註
Transformers 支持多種語言

X
tsmatz
630.71k
25
精選推薦AI模型
Llama 3 Typhoon V1.5x 8b Instruct
專為泰語設計的80億參數指令模型,性能媲美GPT-3.5-turbo,優化了應用場景、檢索增強生成、受限生成和推理任務
大型語言模型
Transformers 支持多種語言

L
scb10x
3,269
16
Cadet Tiny
Openrail
Cadet-Tiny是一個基於SODA數據集訓練的超小型對話模型,專為邊緣設備推理設計,體積僅為Cosmo-3B模型的2%左右。
對話系統
Transformers 英語

C
ToddGoldfarb
2,691
6
Roberta Base Chinese Extractive Qa
基於RoBERTa架構的中文抽取式問答模型,適用於從給定文本中提取答案的任務。
問答系統 中文
R
uer
2,694
98