Ke T5 Base Ko

🚀 ke-t5-base-ko 模型卡片

ke-t5-base-ko 是一个用于文本到文本生成任务的模型，由韩国电子技术研究院人工智能研究中心开发。它基于 T5 架构，在大规模语料上进行预训练，可用于多种自然语言处理任务。

🚀 快速开始

使用以下代码开始使用该模型：

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("KETI-AIR/ke-t5-base-ko")

model = AutoModelForSeq2SeqLM.from_pretrained("KETI-AIR/ke-t5-base-ko")

✨ 主要特性

• 文本到文本生成：该模型可用于文本到文本生成任务。
• 多语言支持：支持韩语和英语。

📦 安装指南

暂未提供安装步骤相关内容。

💻 使用示例

基础用法

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("KETI-AIR/ke-t5-base-ko")
model = AutoModelForSeq2SeqLM.from_pretrained("KETI-AIR/ke-t5-base-ko")

input_text = "아버지가 방에 들어가신다.</s>"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids
outputs = model.generate(input_ids)
output_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(output_text)

📚 详细文档

模型详情

模型描述

• 开发者：韩国电子技术研究院人工智能研究中心
• 模型类型：文本到文本生成
• 相关模型：父模型为 T5
• 更多信息资源：
  • GitHub 仓库
  • KE-T5 Github 仓库
  • 论文
  • 关联论文
  • 博客文章

用途

直接用途

该模型可用于文本到文本生成任务。

超出范围的使用

该模型不应用于故意为人们创造敌对或疏远的环境。

偏差、风险和局限性

大量研究已经探讨了语言模型的偏差和公平性问题。该模型生成的预测可能包括受保护类别、身份特征以及敏感、社会和职业群体的令人不安和有害的刻板印象。

建议

用户（直接用户和下游用户）应了解该模型的风险、偏差和局限性。

训练详情

训练数据

该模型在 Colossal Clean Crawled Corpus (C4) 上进行预训练，该语料库是在与 T5 相同的 研究论文 背景下开发和发布的。模型在无监督（1.）和有监督任务（2.）的多任务混合上进行预训练。

评估

相关测试数据、因素、指标及结果的详细信息暂未提供。

模型检查

相关信息暂未提供。

环境影响

可以使用 Lacoste 等人（2019） 提出的 机器学习影响计算器 来估算碳排放。但硬件类型、使用时长、云服务提供商、计算区域和碳排放等具体信息暂未提供。

技术规格（可选）

相关模型架构和目标、计算基础设施（硬件和软件）的详细信息暂未提供。

引用

BibTeX

@inproceedings{kim-etal-2021-model-cross,
    title = "A Model of Cross-Lingual Knowledge-Grounded Response Generation for Open-Domain Dialogue Systems",
    author = "Kim, San  and
      Jang, Jin Yea  and
      Jung, Minyoung  and
      Shin, Saim",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021",
    month = nov,
    year = "2021",
    address = "Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.findings-emnlp.33",
    doi = "10.18653/v1/2021.findings-emnlp.33",
    pages = "352--365",
    abstract = "Research on open-domain dialogue systems that allow free topics is challenging in the field of natural language processing (NLP). The performance of the dialogue system has been improved recently by the method utilizing dialogue-related knowledge; however, non-English dialogue systems suffer from reproducing the performance of English dialogue systems because securing knowledge in the same language with the dialogue system is relatively difficult. Through experiments with a Korean dialogue system, this paper proves that the performance of a non-English dialogue system can be improved by utilizing English knowledge, highlighting the system uses cross-lingual knowledge. For the experiments, we 1) constructed a Korean version of the Wizard of Wikipedia dataset, 2) built Korean-English T5 (KE-T5), a language model pre-trained with Korean and English corpus, and 3) developed a knowledge-grounded Korean dialogue model based on KE-T5. We observed the performance improvement in the open-domain Korean dialogue model even only English knowledge was given. The experimental results showed that the knowledge inherent in cross-lingual language models can be helpful for generating responses in open dialogue systems.",
}

@article{2020t5,
  author  = {Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu},
  title   = {Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer},
  journal = {Journal of Machine Learning Research},
  year    = {2020},
  volume  = {21},
  number  = {140},
  pages   = {1-67},
  url     = {http://jmlr.org/papers/v21/20-074.html}
}

APA

- Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., ... & Liu, P. J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140), 1-67.

📄 许可证

该模型使用 Apache-2.0 许可证。