The open-source Chinese text processing model `zh_core_web_sm` - Freely deploy to implement functions such as word segmentation and part-of-speech tagging.

Zh Core Web Sm

Developed by spacy

Chinese processing pipeline optimized for CPU, including tokenization, part-of-speech tagging, dependency parsing, named entity recognition, etc.

Sequence Labeling ChineseOpen Source License:MIT #Chinese NLP Processing #OntoNotes Annotation #Dependency Parsing

Downloads 144

Release Time : 3/2/2022

Model Overview

A small Chinese language model provided by spaCy, suitable for basic natural language processing tasks such as part-of-speech tagging, dependency parsing, named entity recognition, etc.

Model Features

CPU Optimization

Specifically optimized for CPU environments, suitable for resource-constrained scenarios

Multi-task Processing

A single model supports multiple NLP tasks, including tokenization, part-of-speech tagging, dependency parsing, etc.

Lightweight

Designed as a small model, suitable for quick deployment and real-time processing

Model Capabilities

Chinese Tokenization

Part-of-Speech Tagging

Dependency Parsing

Named Entity Recognition

Sentence Segmentation

Use Cases

Text Analysis

News Content Analysis

Extract entities and analyze sentence structures from news texts

Identifies entities such as people, places, and organizations with an accuracy of approximately 72%

Social Media Monitoring

Analyze the grammatical structure and entities in social media texts

Can be used for sentiment analysis and topic tracking

Language Learning

Chinese Grammar Analysis

Help learners understand Chinese sentence structures and parts of speech

Part-of-speech tagging accuracy is approximately 89.33%

🚀 zh_core_web_sm

A Chinese pipeline optimized for CPU, which includes components like tok2vec, tagger, parser, senter, ner, and attribute_ruler.

🚀 Quick Start

The detailed information about this model can be found at: Details

✨ Features

Optimized for CPU processing.
Contains multiple components for different NLP tasks.

📚 Documentation

Model Information

Property	Details
Model Name	`zh_core_web_sm`
Version	`3.7.0`
spaCy Compatibility	`>=3.7.0,<3.8.0`
Default Pipeline	`tok2vec`, `tagger`, `parser`, `attribute_ruler`, `ner`
Components	`tok2vec`, `tagger`, `parser`, `senter`, `attribute_ruler`, `ner`
Vectors	0 keys, 0 unique vectors (0 dimensions)
Sources	OntoNotes 5 (Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, Mohammed El-Bachouti, Robert Belvin, Ann Houston) CoreNLP Universal Dependencies Converter (Stanford NLP Group)
License	`MIT`
Author	Explosion

Label Scheme

View label scheme (100 labels for 3 components)

Component	Labels
`tagger`	`AD`, `AS`, `BA`, `CC`, `CD`, `CS`, `DEC`, `DEG`, `DER`, `DEV`, `DT`, `ETC`, `FW`, `IJ`, `INF`, `JJ`, `LB`, `LC`, `M`, `MSP`, `NN`, `NR`, `NT`, `OD`, `ON`, `P`, `PN`, `PU`, `SB`, `SP`, `URL`, `VA`, `VC`, `VE`, `VV`, `X`, `_SP`
`parser`	`ROOT`, `acl`, `advcl:loc`, `advmod`, `advmod:dvp`, `advmod:loc`, `advmod:rcomp`, `amod`, `amod:ordmod`, `appos`, `aux:asp`, `aux:ba`, `aux:modal`, `aux:prtmod`, `auxpass`, `case`, `cc`, `ccomp`, `compound:nn`, `compound:vc`, `conj`, `cop`, `dep`, `det`, `discourse`, `dobj`, `etc`, `mark`, `mark:clf`, `name`, `neg`, `nmod`, `nmod:assmod`, `nmod:poss`, `nmod:prep`, `nmod:range`, `nmod:tmod`, `nmod:topic`, `nsubj`, `nsubj:xsubj`, `nsubjpass`, `nummod`, `parataxis:prnmod`, `punct`, `xcomp`
`ner`	`CARDINAL`, `DATE`, `EVENT`, `FAC`, `GPE`, `LANGUAGE`, `LAW`, `LOC`, `MONEY`, `NORP`, `ORDINAL`, `ORG`, `PERCENT`, `PERSON`, `PRODUCT`, `QUANTITY`, `TIME`, `WORK_OF_ART`

Accuracy

Type	Score
`TOKEN_ACC`	95.85
`TOKEN_P`	94.58
`TOKEN_R`	91.36
`TOKEN_F`	92.94
`TAG_ACC`	89.33
`SENTS_P`	77.85
`SENTS_R`	72.62
`SENTS_F`	75.14
`DEP_UAS`	69.60
`DEP_LAS`	64.08
`ENTS_P`	72.03
`ENTS_R`	64.93
`ENTS_F`	68.30

Model Index

Task	Metrics
NER (Token Classification)
- NER Precision	0.7203462148
- NER Recall	0.6493406593
- NER F Score	0.6830029475
TAG (Token Classification)
- TAG (XPOS) Accuracy	0.8933253054
UNLABELED_DEPENDENCIES (Token Classification)
- Unlabeled Attachment Score (UAS)	0.6960047338
LABELED_DEPENDENCIES (Token Classification)
- Labeled Attachment Score (LAS)	0.640776699
SENTS (Token Classification)
- Sentences F-Score	0.7514211886

📄 License

This project is licensed under the MIT license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご