Ko Core News Sm
模型简介
这是一个韩语自然语言处理模型,提供分词、词性标注、依存句法分析、命名实体识别等基础NLP功能。模型针对CPU使用进行了优化,适合处理韩语文本数据。
模型特点
CPU优化
专门针对CPU使用进行了优化,适合在没有GPU的环境下运行
全面NLP功能
提供从基础分词到命名实体识别的完整NLP处理流程
高精度句子分割
句子分割F值达到99.93%,能准确识别句子边界
模型能力
分词
词性标注
依存句法分析
命名实体识别
词形还原
句子分割
使用案例
文本处理
韩语文本分析
对韩语新闻、社交媒体文本进行语法分析和结构解析
准确识别句子结构和词性关系
信息提取
从韩语文本中提取人名、组织名、地点等命名实体
NER F值达到71.11%
语言研究
韩语语法研究
分析韩语句法结构和词形变化
提供详细的词性标注和依存关系分析
🚀 ko_core_news_sm模型
ko_core_news_sm
是一款针对韩语进行优化的自然语言处理模型,专为CPU运行环境设计。它在命名实体识别(NER)、词性标注(POS)等多个任务上表现出色,能有效处理韩语的语言结构和语义信息。
✨ 主要特性
- 多任务支持:支持命名实体识别(NER)、词性标注(TAG、POS)、词形还原(LEMMA)、依存句法分析(UNLABELED_DEPENDENCIES、LABELED_DEPENDENCIES)和句子分割(SENTS)等多种任务。
- CPU优化:针对CPU进行了优化,能够在普通CPU环境下高效运行。
- 丰富的标注体系:提供了详细的标签体系,涵盖了多种语言成分和语法结构。
📦 安装指南
文档未提及安装步骤,跳过该章节。
💻 使用示例
文档未提供代码示例,跳过该章节。
📚 详细文档
模型详情
- 详情链接:https://spacy.io/models/ko#ko_core_news_sm
- 适用场景:韩语自然语言处理,如文本分类、信息提取等。
模型信息
属性 | 详情 |
---|---|
模型类型 | ko_core_news_sm |
版本 | 3.7.0 |
spaCy版本要求 | >=3.7.0,<3.8.0 |
默认管道 | tok2vec , tagger , morphologizer , parser , lemmatizer , attribute_ruler , ner |
组件 | tok2vec , tagger , morphologizer , parser , lemmatizer , senter , attribute_ruler , ner |
向量 | 0 个键,0 个唯一向量(0 维) |
数据源 | UD Korean Kaist v2.8 (Choi, Jinho; Han, Na - Rae; Hwang, Jena; Chun, Jayeol) KLUE v1.1.0 (Sungjoon Park, Jihyung Moon, Sungdong Kim, Won Ik Cho, Jiyoon Han, Jangwon Park, Chisung Song, Junseong Kim, Youngsook Song, Taehwan Oh, Joohong Lee, Juhyun Oh, Sungwon Ryu, Younghoon Jeong, Inkwon Lee, Sangwoo Seo, Dongjun Lee, Hyunwoo Kim, Myeonghwa Lee, Seongbo Jang, Seungwon Do, Sunkyoung Kim, Kyungtae Lim, Jongwon Lee, Kyumin Park, Jamin Shin, Seonghyun Kim, Lucy Park, Alice Oh, Jung - Woo Ha, Kyunghyun Cho) |
许可证 | CC BY - SA 4.0 |
作者 | Explosion |
标签体系
查看标签体系(4个组件共2028个标签)
组件 | 标签 |
---|---|
tagger |
_SP , ecs , etm , f , f+f+jcj , f+f+jcs , f+f+jct , f+f+jxt , f+jca , f+jca+jp+ecc , f+jca+jp+ep+ef , f+jca+jxc , f+jca+jxc+jcm , f+jca+jxt , f+jcj , f+jcm , f+jco , f+jcs , f+jct , f+jct+jcm , f+jp+ef , f+jp+ep+ef , f+jp+etm , f+jxc , f+jxt , f+ncn , f+ncn+jcm , f+ncn+jcs , f+ncn+jp+ecc , f+ncn+jxt , f+ncpa+jcm , f+npp+jcs , f+nq , f+xsn , f+xsn+jco , f+xsn+jxt , ii , jca , jca+jcm , jca+jxc , jca+jxt , jcc , jcj , jcm , jco , jcr , jcr+jxc , jcs , jct , jct+jcm , jct+jxt , jp+ecc , jp+ecs , jp+ef , jp+ef+jcr , jp+ef+jcr+jxc , jp+ep+ecs , jp+ep+ef , jp+ep+etm , jp+ep+etn , jp+etm , jp+etn , jp+etn+jco , jp+etn+jxc , jxc , jxc+jca , jxc+jco , jxc+jcs , jxt , mad , mad+jxc , mad+jxt , mag , mag+jca , mag+jcm , mag+jcs , mag+jp+ef+jcr , mag+jxc , mag+jxc+jxc , mag+jxt , mag+xsn , maj , maj+jxc , maj+jxt , mma , mmd , nbn , nbn+jca , nbn+jca+jcj , nbn+jca+jcm , nbn+jca+jp+ef , nbn+jca+jxc , nbn+jca+jxt , nbn+jcc , nbn+jcj , nbn+jcm , nbn+jco , nbn+jcr , nbn+jcs , nbn+jct , nbn+jct+jcm , nbn+jct+jxt , nbn+jp+ecc , nbn+jp+ecs , nbn+jp+ecs+jca , nbn+jp+ecs+jcm , nbn+jp+ecs+jco , nbn+jp+ecs+jxc , nbn+jp+ecs+jxt , nbn+jp+ecx , nbn+jp+ef , nbn+jp+ef+jca , nbn+jp+ef+jco , nbn+jp+ef+jcr , nbn+jp+ef+jcr+jxc , nbn+jp+ef+jcr+jxt , nbn+jp+ef+jcs , nbn+jp+ef+jxc , nbn+jp+ef+jxc+jco , nbn+jp+ef+jxf , nbn+jp+ef+jxt , nbn+jp+ep+ecc , nbn+jp+ep+ecs , nbn+jp+ep+ecs+jxc , nbn+jp+ep+ef , nbn+jp+ep+ef+jcr , nbn+jp+ep+etm , nbn+jp+ep+etn , nbn+jp+ep+etn+jco , nbn+jp+ep+etn+jcs , nbn+jp+etm , nbn+jp+etn , nbn+jp+etn+jca , nbn+jp+etn+jca+jxt , nbn+jp+etn+jco , nbn+jp+etn+jcs , nbn+jp+etn+jxc , nbn+jp+etn+jxt , nbn+jxc , nbn+jxc+jca , nbn+jxc+jca+jxc , nbn+jxc+jca+jxt , nbn+jxc+jcc , nbn+jxc+jcm , nbn+jxc+jco , nbn+jxc+jcs , nbn+jxc+jp+ef , nbn+jxc+jxc , nbn+jxc+jxt , nbn+jxt , nbn+nbn , nbn+nbn+jp+ef , nbn+xsm+ecs , nbn+xsm+ef , nbn+xsm+ep+ef , nbn+xsm+ep+ef+jcr , nbn+xsm+etm , nbn+xsn , nbn+xsn+jca , nbn+xsn+jca+jp+ef+jcr , nbn+xsn+jca+jxc , nbn+xsn+jca+jxt , nbn+xsn+jcm , nbn+xsn+jco , nbn+xsn+jcs , nbn+xsn+jct , nbn+xsn+jp+ecc , nbn+xsn+jp+ecs , nbn+xsn+jp+ef , nbn+xsn+jp+ef+jcr , nbn+xsn+jp+ep+ef , nbn+xsn+jxc , nbn+xsn+jxt , nbn+xsv+etm , nbu , nbu+jca , nbu+jca+jxc , nbu+jca+jxt , nbu+jcc , nbu+jcc+jxc , nbu+jcj , nbu+jcm , nbu+jco , nbu+jcs , nbu+jct , nbu+jct+jxc , nbu+jp+ecc , nbu+jp+ecs , nbu+jp+ef , nbu+jp+ef+jcr , nbu+jp+ef+jxc , nbu+jp+ep+ecc , nbu+jp+ep+ecs , nbu+jp+ep+ef , nbu+jp+ep+ef+jcr , nbu+jp+ep+etm , nbu+jp+ep+etn+jco , nbu+jp+etm , nbu+jxc , nbu+jxc+jca , nbu+jxc+jcs , nbu+jxc+jp+ef , nbu+jxc+jp+ep+ef , nbu+jxc+jxt , nbu+jxt , nbu+ncn , nbu+ncn+jca , nbu+ncn+jcm , nbu+xsn , nbu+xsn+jca , nbu+xsn+jca+jxc , nbu+xsn+jca+jxt , nbu+xsn+jcm , nbu+xsn+jco , nbu+xsn+jcs , nbu+xsn+jp+ecs , nbu+xsn+jp+ep+ef , nbu+xsn+jxc , nbu+xsn+jxc+jxt , nbu+xsn+jxt , nbu+xsv+ecc , nbu+xsv+etm , ncn , ncn+f+ncpa+jco , ncn+jca , ncn+jca+jca , ncn+jca+jcc , ncn+jca+jcj , ncn+jca+jcm , ncn+jca+jcs , ncn+jca+jct , ncn+jca+jp+ecc , ncn+jca+jp+ecs , ncn+jca+jp+ef , ncn+jca+jp+ep+ef , ncn+jca+jp+etm , ncn+jca+jp+etn+jxt , ncn+jca+jxc , ncn+jca+jxc+jcc , ncn+jca+jxc+jcm , ncn+jca+jxc+jxc , ncn+jca+jxc+jxt , ncn+jca+jxt , ncn+jcc , ncn+jcc+jxc , ncn+jcj , ncn+jcj+jxt , ncn+jcm , ncn+jco , ncn+jcr , ncn+jcr+jxc , ncn+jcs , ncn+jcs+jxt , ncn+jct , ncn+jct+jcm , ncn+jct+jxc , ncn+jct+jxt , ncn+jcv , ncn+jp+ecc , ncn+jp+ecc+jct , ncn+jp+ecc+jxc , ncn+jp+ecs , ncn+jp+ecs+jcm , ncn+jp+ecs+jco , ncn+jp+ecs+jxc , ncn+jp+ecs+jxt , ncn+jp+ecx , ncn+jp+ef , ncn+jp+ef+jca , ncn+jp+ef+jcm , ncn+jp+ef+jco , ncn+jp+ef+jcr , ncn+jp+ef+jcr+jxc , ncn+jp+ef+jcr+jxt , ncn+jp+ef+jp+etm , ncn+jp+ef+jxc , ncn+jp+ef+jxf , ncn+jp+ef+jxt , ncn+jp+ep+ecc , ncn+jp+ep+ecs , ncn+jp+ep+ecs+jxc , ncn+jp+ep+ecx , ncn+jp+ep+ef , ncn+jp+ep+ef+jcr , ncn+jp+ep+ef+jcr+jxc , ncn+jp+ep+ef+jxc , ncn+jp+ep+ef+jxf , ncn+jp+ep+ef+jxt , ncn+jp+ep+ep+etm , ncn+jp+ep+etm , ncn+jp+ep+etn , ncn+jp+ep+etn+jca , ncn+jp+ep+etn+jca+jxc , ncn+jp+ep+etn+jco , ncn+jp+ep+etn+jcs , ncn+jp+ep+etn+jxt , ncn+jp+etm , ncn+jp+etn , ncn+jp+etn+jca , ncn+jp+etn+jca+jxc , ncn+jp+etn+jca+jxt , ncn+jp+etn+jco , ncn+jp+etn+jcs , ncn+jp+etn+jct , ncn+jp+etn+jxc , ncn+jp+etn+jxt , ncn+jxc , ncn+jxc+jca , ncn+jxc+jca+jxc , ncn+jxc+jca+jxt , ncn+jxc+jcc , ncn+jxc+jcm , ncn+jxc+jco , ncn+jxc+jcs , ncn+jxc+jct+jxt , ncn+jxc+jp+ef , ncn+jxc+jp+ef+jcr , ncn+jxc+jp+ep+ecs , ncn+jxc+jp+ep+ef , ncn+jxc+jp+etm , ncn+jxc+jxc , ncn+jxc+jxt , ncn+jxt , ncn+jxt+jcm , ncn+jxt+jxc , ncn+nbn , ncn+nbn+jca , ncn+nbn+jcm , ncn+nbn+jcs , ncn+nbn+jp+ecc , ncn+nbn+jp+ep+ef , ncn+nbn+jxc , ncn+nbn+jxt , ncn+nbu , ncn+nbu+jca , ncn+nbu+jcm , ncn+nbu+jco , ncn+nbu+jp+ef , ncn+nbu+jxc , ncn+nbu+ncn , ncn+ncn , ncn+ncn+jca , ncn+ncn+jca+jcc , ncn+ncn+jca+jcm , ncn+ncn+jca+jxc , ncn+ncn+jca+jxc+jcm , ncn+ncn+jca+jxc+jxc , ncn+ncn+jca+jxt , ncn+ncn+jcc , ncn+ncn+jcj , ncn+ncn+jcm , ncn+ncn+jco , ncn+ncn+jcr , ncn+ncn+jcs , ncn+ncn+jct , ncn+ncn+jct+jcm , ncn+ncn+jct+jxc , ncn+ncn+jct+jxt , ncn+ncn+jp+ecc , ncn+ncn+jp+ecs , ncn+ncn+jp+ef , ncn+ncn+jp+ef+jcm , ncn+ncn+jp+ef+jcr , ncn+ncn+jp+ef+jcs , ncn+ncn+jp+ep+ecc , ncn+ncn+jp+ep+ecs , ncn+ncn+jp+ep+ef , ncn+ncn+jp+ep+ef+jcr , ncn+ncn+jp+ep+ep+etm , ncn+ncn+jp+ep+etm , ncn+ncn+jp+ep+etn , ncn+ncn+jp+etm , ncn+ncn+jp+etn , ncn+ncn+jp+etn+jca , ncn+ncn+jp+etn+jco , ncn+ncn+jp+etn+jxc , ncn+ncn+jxc , ncn+ncn+jxc+jca , ncn+ncn+jxc+jcc , ncn+ncn+jxc+jcm , ncn+ncn+jxc+jco , ncn+ncn+jxc+jcs , ncn+ncn+jxc+jxc , ncn+ncn+jxt , ncn+ncn+nbn , ncn+ncn+ncn , ncn+ncn+ncn+jca , ncn+ncn+ncn+jca+jcm , ncn+ncn+ncn+jca+jxt , ncn+ncn+ncn+jcj , ncn+ncn+ncn+jcm , ncn+ncn+ncn+jco , ncn+ncn+ncn+jcs , ncn+ncn+ncn+jct+jxt , ncn+ncn+ncn+jp+etn+jxc , ncn+ncn+ncn+jxt , ncn+ncn+ncn+ncn+jca , ncn+ncn+ncn+ncn+jca+jxt , ncn+ncn+ncn+ncn+jco , ncn+ncn+ncn+xsn+jp+etm , ncn+ncn+ncpa , ncn+ncn+ncpa+jca , ncn+ncn+ncpa+jcm , ncn+ncn+ncpa+jco , ncn+ncn+ncpa+jcs , ncn+ncn+ncpa+jxc , ncn+ncn+ncpa+jxt , ncn+ncn+ncpa+ncn , ncn+ncn+ncpa+ncn+jca , ncn+ncn+ncpa+ncn+jcj , ncn+ncn+ncpa+ncn+jcm , ncn+ncn+ncpa+ncn+jxt , ncn+ncn+xsn , ncn+ncn+xsn+jca , ncn+ncn+xsn+jca+jxt , ncn+ncn+xsn+jcj , ncn+ncn+xsn+jcm , ncn+ncn+xsn+jco , ncn+ncn+xsn+jcs , ncn+ncn+xsn+jct , ncn+ncn+xsn+jp+ecs , ncn+ncn+xsn+jp+ep+ef , ncn+ncn+xsn+jp+etm , ncn+ncn+xsn+jxc , ncn+ncn+xsn+jxc+jcs , ncn+ncn+xsn+jxt , ncn+ncn+xsv+ecc , ncn+ncn+xsv+etm , ncn+ncpa , ncn+ncpa+jca , ncn+ncpa+jca+jcm , ncn+ncpa+jca+jxc , ncn+ncpa+jca+jxt , ncn+ncpa+jcc , ncn+ncpa+jcj , ncn+ncpa+jcm , ncn+ncpa+jco , ncn+ncpa+jcr , ncn+ncpa+jcs , ncn+ncpa+jct , ncn+ncpa+jct+jcm , ncn+ncpa+jct+jxt , ncn+ncpa+jp+ecc , ncn+ncpa+jp+ecc+jxc , ncn+ncpa+jp+ecs , ncn+ncpa+jp+ecs+jxc , ncn+ncpa+jp+ef , ncn+ncpa+jp+ef+jcr , ncn+ncpa+jp+ef+jcr+jxc , ncn+ncpa+jp+ep+ef , ncn+ncpa+jp+ep+etm , ncn+ncpa+jp+ep+etn , ncn+ncpa+jp+etm , ncn+ncpa+jxc , ncn+ncpa+jxc+jca+jxc , ncn+ncpa+jxc+jco , ncn+ncpa+jxc+jcs , ncn+ncpa+jxt , ncn+ncpa+nbn+jcs , ncn+ncpa+ncn , ncn+ncpa+ncn+jca , ncn+ncpa+ncn+jca+jcm , ncn+ncpa+ncn+jca+jxc , ncn+ncpa+ncn+jca+jxt , ncn+ncpa+ncn+jcj , ncn+ncpa+ncn+jcm , ncn+ncpa+ncn+jco , ncn+ncpa+ncn+jcs , ncn+ncpa+ncn+jct , ncn+ncpa+ncn+jct+jcm , ncn+ncpa+ncn+jp+ef+jcr , ncn+ncpa+ncn+jp+ep+etm , ncn+ncpa+ncn+jxc , ncn+ncpa+ncn+jxt , ncn+ncpa+ncn+xsn+jcm , ncn+ncpa+ncn+xsn+jxt , ncn+ncpa+ncpa , ncn+ncpa+ncpa+jca , ncn+ncpa+ncpa+jcj , ncn+ncpa+ncpa+jcm , ncn+ncpa+ncpa+jco , ncn+ncpa+ncpa+jcs , ncn+ncpa+ncpa+jp+ep+ef , ncn+ncpa+ncpa+jxt , ncn+ncpa+ncpa+ncn , ncn+ncpa+xsn , ncn+ncpa+xsn+jcm , ncn+ncpa+xsn+jco , ncn+ncpa+xsn+jcs , ncn+ncpa+xsn+jp+ecc , ncn+ncpa+xsn+jp+etm , ncn+ncpa+xsn+jxt , ncn+ncpa+xsv+ecc , ncn+ncpa+xsv+ecs , ncn+ncpa+xsv+ecx , ncn+ncpa+xsv+ecx+px+etm , ncn+ncpa+xsv+ef , ncn+ncpa+xsv+ef+jcm , ncn+ncpa+xsv+ef+jcr , ncn+ncpa+xsv+etm , (truncated: full list in pipeline meta) |
morphologizer |
POS=CCONJ , POS=ADV , POS=SCONJ , POS=DET , POS=NOUN , POS=VERB , POS=ADJ , POS=PUNCT , POS=SPACE , POS=AUX , POS=PRON , POS=PROPN , POS=NUM , POS=INTJ , POS=PART , POS=X , POS=ADP , POS=SYM |
parser |
ROOT , acl , advcl , advmod , amod , appos , aux , case , cc , ccomp , compound , conj , cop , csubj , dep , det , dislocated , fixed , flat , iobj , mark , nmod , nsubj , nummod , obj , obl , punct , xcomp |
ner |
DT , LC , OG , PS , QT , TI |
准确率
类型 | 得分 |
---|---|
TOKEN_ACC |
100.00 |
TOKEN_P |
100.00 |
TOKEN_R |
100.00 |
TOKEN_F |
100.00 |
TAG_ACC |
73.06 |
POS_ACC |
85.82 |
SENTS_P |
99.90 |
SENTS_R |
99.95 |
SENTS_F |
99.93 |
DEP_UAS |
73.61 |
DEP_LAS |
65.59 |
LEMMA_ACC |
83.57 |
ENTS_P |
77.04 |
ENTS_R |
66.03 |
ENTS_F |
71.11 |
🔧 技术细节
文档未提供技术实现细节,跳过该章节。
📄 许可证
本模型使用的许可证为 CC BY - SA 4.0
。
Indonesian Roberta Base Posp Tagger
MIT
这是一个基于印尼语RoBERTa模型微调的词性标注模型,在indonlu数据集上训练,用于印尼语文本的词性标注任务。
序列标注
Transformers 其他

I
w11wo
2.2M
7
Bert Base NER
MIT
基于BERT微调的命名实体识别模型,可识别四类实体:地点(LOC)、组织机构(ORG)、人名(PER)和杂项(MISC)
序列标注 英语
B
dslim
1.8M
592
Deid Roberta I2b2
MIT
该模型是基于RoBERTa微调的序列标注模型,用于识别和移除医疗记录中的受保护健康信息(PHI/PII)。
序列标注
Transformers 支持多种语言

D
obi
1.1M
33
Ner English Fast
Flair自带的英文快速4类命名实体识别模型,基于Flair嵌入和LSTM-CRF架构,在CoNLL-03数据集上达到92.92的F1分数。
序列标注
PyTorch 英语
N
flair
978.01k
24
French Camembert Postag Model
基于Camembert-base的法语词性标注模型,使用free-french-treebank数据集训练
序列标注
Transformers 法语

F
gilf
950.03k
9
Xlm Roberta Large Ner Spanish
基于XLM-Roberta-large架构微调的西班牙语命名实体识别模型,在CoNLL-2002数据集上表现优异。
序列标注
Transformers 西班牙语

X
MMG
767.35k
29
Nusabert Ner V1.3
MIT
基于NusaBert-v1.3在印尼语NER任务上微调的命名实体识别模型
序列标注
Transformers 其他

N
cahya
759.09k
3
Ner English Large
Flair框架内置的英文4类大型NER模型,基于文档级XLM-R嵌入和FLERT技术,在CoNLL-03数据集上F1分数达94.36。
序列标注
PyTorch 英语
N
flair
749.04k
44
Punctuate All
MIT
基于xlm-roberta-base微调的多语言标点符号预测模型,支持12种欧洲语言的标点符号自动补全
序列标注
Transformers

P
kredor
728.70k
20
Xlm Roberta Ner Japanese
MIT
基于xlm-roberta-base微调的日语命名实体识别模型
序列标注
Transformers 支持多种语言

X
tsmatz
630.71k
25
精选推荐AI模型
Llama 3 Typhoon V1.5x 8b Instruct
专为泰语设计的80亿参数指令模型,性能媲美GPT-3.5-turbo,优化了应用场景、检索增强生成、受限生成和推理任务
大型语言模型
Transformers 支持多种语言

L
scb10x
3,269
16
Cadet Tiny
Openrail
Cadet-Tiny是一个基于SODA数据集训练的超小型对话模型,专为边缘设备推理设计,体积仅为Cosmo-3B模型的2%左右。
对话系统
Transformers 英语

C
ToddGoldfarb
2,691
6
Roberta Base Chinese Extractive Qa
基于RoBERTa架构的中文抽取式问答模型,适用于从给定文本中提取答案的任务。
问答系统 中文
R
uer
2,694
98