Roberta Small

🚀 KLUE RoBERTa small

A pre-trained RoBERTa model designed for the Korean language. It offers efficient performance for various NLP tasks related to Korean text.

Metadata

Widget Example

- text: 대한민국의 수도는 [MASK] 입니다.

🚀 Quick Start

This is a pre-trained RoBERTa model for the Korean language. For more details, please refer to Github and Paper.

💻 Usage Examples

Basic Usage

# NOTE: Use `BertTokenizer` instead of RobertaTokenizer. (`AutoTokenizer` will load `BertTokenizer`)
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("klue/roberta-small")
tokenizer = AutoTokenizer.from_pretrained("klue/roberta-small")

📚 Documentation

BibTeX entry and citation info

@misc{park2021klue,
      title={KLUE: Korean Language Understanding Evaluation},
      author={Sungjoon Park and Jihyung Moon and Sungdong Kim and Won Ik Cho and Jiyoon Han and Jangwon Park and Chisung Song and Junseong Kim and Yongsook Song and Taehwan Oh and Joohong Lee and Juhyun Oh and Sungwon Lyu and Younghoon Jeong and Inkwon Lee and Sangwoo Seo and Dongjun Lee and Hyunwoo Kim and Myeonghwa Lee and Seongbo Jang and Seungwon Do and Sunkyoung Kim and Kyungtae Lim and Jongwon Lee and Kyumin Park and Jamin Shin and Seonghyun Kim and Lucy Park and Alice Oh and Jungwoo Ha and Kyunghyun Cho},
      year={2021},
      eprint={2105.09680},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}