Yi 34B 200K

🚀 Yi: Next Generation of Open-Source LLMs

The Yi series models are the next generation of open - source large language models, offering powerful language understanding and reasoning capabilities. Trained on a vast multilingual corpus, they stand out among global LLMs, suitable for various personal, academic, and commercial use cases.

🚀 Quick Start

💡 Tip: If you want to get started with the Yi model and explore different methods for inference, check out the Yi Cookbook.

Choose your path

Select one of the following paths to begin your journey with Yi!

🎯 Deploy Yi locally

If you prefer to deploy Yi models locally,

• 🙋‍♀️ and you have sufficient resources (for example, NVIDIA A800 80GB), you can follow the steps below.

Quick start - pip

Quick start - docker

Quick start - llama.cpp

Quick start - conda - lock

Web demo

✨ Features

Introduction

• 🤖 The Yi series models are the next generation of open - source large language models trained from scratch by 01.AI.
• 🙌 Targeted as a bilingual language model and trained on 3T multilingual corpus, the Yi series models become one of the strongest LLM worldwide, showing promise in language understanding, commonsense reasoning, reading comprehension, and more. For example,
  • Yi - 34B - Chat model landed in second place (following GPT - 4 Turbo), outperforming other LLMs (such as GPT - 4, Mixtral, Claude) on the AlpacaEval Leaderboard (based on data available up to January 2024).
  • Yi - 34B model ranked first among all existing open - source models (such as Falcon - 180B, Llama - 70B, Claude) in both English and Chinese on various benchmarks, including Hugging Face Open LLM Leaderboard (pre - trained) and C - Eval (based on data available up to November 2023).
  • 🙏 (Credits to Llama) Thanks to the Transformer and Llama open - source communities, as they reduce the efforts required to build from scratch and enable the utilization of the same tools within the AI ecosystem.

News

Models

Yi models come in multiple sizes and cater to different use cases. You can also fine - tune Yi models to meet your specific requirements. If you want to deploy Yi models, make sure you meet the software and hardware requirements.

Chat models

<sub><sup>- 4 - bit series models are quantized by AWQ.
- 8 - bit series models are quantized by GPTQ
- All quantized models have a low barrier to use since they can be deployed on consumer - grade GPUs (e.g., 3090, 4090).</sup></sub>

Base models

<sub><sup>- 200k is roughly equivalent to 400,000 Chinese characters.
- If you want to use the previous version of the Yi - 34B - 200K (released on Nov 5, 2023), run git checkout 069cd341d60f4ce4b07ec394e82b79e94f656cf to download the weight.</sup></sub>

Model info

• For chat and base models

• For chat models

  For chat model limitations, see the explanations below. ⬇️

  
  The released chat model has undergone exclusive training using Supervised Fine - Tuning (SFT). Compared to other standard chat models, our model produces more diverse responses, making it suitable for various downstream tasks, such as creative scenarios. Furthermore, this diversity is expected to enhance the likelihood of generating higher quality responses, which will be advantageous for subsequent Reinforcement Learning (RL) training.

  
  However, this higher diversity might amplify certain existing issues, including:

  • Hallucination: This refers to the model generating factually incorrect or nonsensical information. With the model's responses being more varied, there's a higher chance of hallucination that are not based on accurate data or logical reasoning.
  • Non - determinism in re - generation: When attempting to regenerate or sample responses, inconsistencies in the outcomes may occur. The increased diversity can lead to varying results even under similar input conditions.
  • Cumulative Error: This occurs when errors in the model's responses compound over time. As the model generates more diverse responses, the likelihood of small inaccuracies building up into larger errors increases, especially in complex tasks like extended reasoning, mathematical problem - solving, etc.
  • To achieve more coherent and consistent responses, it is advisable to adjust generation configuration parameters such as temperature, top_p, or top_k. These adjustments can help in the balance between creativity and coherence in the model's outputs.

📚 Documentation

Fine - tuning

Quantization

Deployment

FAQ

Learning hub

🔧 Technical Details

Ecosystem

Upstream

Downstream

Serving

Quantization

Fine - tuning

API

Benchmarks

Base model performance

Chat model performance

Tech report

Citation

📄 License

The Yi series models are released under the Apache - 2.0 license.