EEVE Korean Instruct 2.8B V1.0

🚀 EEVE-Korean-Instruct-2.8B-v1.0

EEVE-Korean-Instruct-2.8B-v1.0 是基於大語言模型領域開發的模型，它在特定基礎模型上進行微調，適用於韓語相關的語言處理任務，為韓語語言交互等場景提供支持。

🚀 快速開始

加入我們的 Discord 社區！

如果您對大語言模型領域充滿熱情，希望交流知識和見解，我們誠摯地邀請您加入我們的 Discord 服務器。值得注意的是，該服務器主要使用韓語。大語言模型的發展日新月異，如果不積極分享，我們的知識很快就會過時。讓我們攜手合作，共同產生更大的影響！點擊此處加入：Discord 鏈接。

✨ 主要特性

團隊信息

模型介紹

該模型是 yanolja/EEVE-Korean-2.8B-v1.0 的微調版本，而 yanolja/EEVE-Korean-2.8B-v1.0 是 microsoft/phi-2 的韓語詞彙擴展版本。具體來說，我們通過 Axolotl 使用了直接偏好優化（DPO）方法。

更多詳細信息，請參考我們的技術報告：Efficient and Effective Vocabulary Expansion Towards Multilingual Large Language Models。

提示模板

A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
Human: {prompt}
Assistant:

📦 安裝指南

文檔未提供安裝步驟，暫不展示。

💻 使用示例

基礎用法

from transformers import AutoTokenizer
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("yanolja/EEVE-Korean-Instruct-2.8B-v1.0", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("yanolja/EEVE-Korean-Instruct-2.8B-v1.0", trust_remote_code=True)

prompt_template = "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.\nHuman: {prompt}\nAssistant:\n"
text = '한국의 수도는 어디인가요? 아래 선택지 중 골라주세요.\n\n(A) 경성\n(B) 부산\n(C) 평양\n(D) 서울\n(E) 전주'
model_inputs = tokenizer(prompt_template.format(prompt=text), return_tensors='pt')

outputs = model.generate(**model_inputs, max_new_tokens=256)
output_text = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
print(output_text)

示例輸出

A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
Human: 한국의 수도는 어디인가요? 아래 선택지 중 골라주세요.

(A) 경성
(B) 부산
(C) 평양
(D) 서울
(E) 전주
Assistant:
한국의 수도는 (D) 서울입니다. 서울은 수도권과 수도권 내의 주요 도시들을 포함하는 광역 행정구역으로, 대한민국의 수도입니다. 서울은 수도권 인구의 약 70%를 차지하며, 대한민국의 경제, 정치, 문화의 중심지입니다.

📚 詳細文檔

訓練數據

• Open-Orca/SlimOrca-Dedup 的韓語翻譯版本
• argilla/ultrafeedback-binarized-preferences-cleaned 的韓語翻譯版本
• 未使用其他數據集

引用信息

@misc{kim2024efficient,
      title={Efficient and Effective Vocabulary Expansion Towards Multilingual Large Language Models}, 
      author={Seungduk Kim and Seungtaek Choi and Myeongho Jeong},
      year={2024},
      eprint={2402.14714},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

@misc{cui2023ultrafeedback,
      title={UltraFeedback: Boosting Language Models with High-quality Feedback}, 
      author={Ganqu Cui and Lifan Yuan and Ning Ding and Guanming Yao and Wei Zhu and Yuan Ni and Guotong Xie and Zhiyuan Liu and Maosong Sun},
      year={2023},
      eprint={2310.01377},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

@misc{SlimOrcaDedup,
  title = {SlimOrca Dedup: A Deduplicated Subset of SlimOrca},
  author = {Wing Lian and Guan Wang and Bleys Goodson and Eugene Pentland and Austin Cook and Chanvichet Vong and "Teknium" and Nathan Hoos},
  year = {2023},
  publisher = {HuggingFace},
  url = {https://huggingface.co/datasets/Open-Orca/SlimOrca-Dedup/}
}

@misc{mukherjee2023orca,
      title={Orca: Progressive Learning from Complex Explanation Traces of GPT-4}, 
      author={Subhabrata Mukherjee and Arindam Mitra and Ganesh Jawahar and Sahaj Agarwal and Hamid Palangi and Ahmed Awadallah},
      year={2023},
      eprint={2306.02707},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Open LLM Leaderboard Evaluation Results

詳細結果可在 此處 查看。

📄 許可證

本項目採用 Apache-2.0 許可證。