%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
Model Overview
Model Features
Model Capabilities
Use Cases
🚀 Llama3-70B-Chinese-Chat
One of the first instruction - tuned LLMs for Chinese & English users with various abilities, built upon Meta - Llama - 3 - 70B - Instruct.
🚀 Quick Start
❗️❗️❗️NOTICE: For optimal performance, we refrain from fine - tuning the model's identity. Thus, inquiries such as "Who are you" or "Who developed you" may yield random responses that are not necessarily accurate.
✨ Features
- Llama3 - 70B - Chinese - Chat is one of the first instruction - tuned LLMs for Chinese & English users with various abilities like role - playing, tool - using, and math.
- According to C - Eval and CMMLU results, its Chinese performance significantly exceeds that of ChatGPT and is comparable to GPT - 4.
- It greatly reduces the issues of "Chinese questions with English answers" and the mixing of Chinese and English in responses compared to the original Meta - Llama - 3 - 70B - Instruct model.
📦 Installation
- We provide the official Ollama model for the q4_0 GGUF version of Llama3 - 70B - Chinese - Chat at [wangshenzhi/llama3 - 70b - chinese - chat - ollama - q4](https://ollama.com/wangshenzhi/llama3 - 70b - chinese - chat - ollama - q4). Run the following command for quick use of this model:
ollama run wangshenzhi/llama3 - 70b - chinese - chat - ollama - q4:latest. - We provide the official Ollama model for the q8_0 GGUF version of Llama3 - 70B - Chinese - Chat at [wangshenzhi/llama3 - 70b - chinese - chat - ollama - q8](https://ollama.com/wangshenzhi/llama3 - 70b - chinese - chat - ollama - q8). Run the following command for quick use of this model:
ollama run wangshenzhi/llama3 - 70b - chinese - chat - ollama - q8:latest. - We provide the official q4_0 GGUF version of Llama3 - 70B - Chinese - Chat at [shenzhi - wang/Llama3 - 70B - Chinese - Chat - GGUF - 4bit](https://huggingface.co/shenzhi - wang/Llama3 - 70B - Chinese - Chat - GGUF - 4bit).
- If you are in China, you can download our model from our [gitee repo](https://ai.gitee.com/shenzhi - wang/llama3 - 70b - chinese - chat).
💻 Usage Examples
Basic Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "shenzhi - wang/Llama3 - 70B - Chinese - Chat"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id, torch_dtype="auto", device_map="auto"
)
messages = [
{"role": "user", "content": "写一首诗吧"},
]
input_ids = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
outputs = model.generate(
input_ids,
max_new_tokens=8192,
do_sample=True,
temperature=0.6,
top_p=0.9,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
📚 Documentation
Updates
- 🚀🚀🚀 [May 9, 2024] We're excited to introduce Llama3 - 70B - Chinese - Chat! Full - parameter fine - tuned on a mixed Chinese - English dataset of ~100K preference pairs, its Chinese performance surpasses ChatGPT and matches GPT - 4, as shown by C - Eval and CMMLU results.
- 🌟 Thanks for the support of Gitee, we have launched an online demo: https://ai.gitee.com/shenzhi - wang/llama3 - 70b - chinese - chat. You need to log in to your Gitee account with the invitation code
llama3.
Model Summary
- Developers: [Shenzhi Wang](https://shenzhi - wang.netlify.app)*, Yaowei Zheng*, Guoyin Wang (in.ai), Shiji Song, Gao Huang. (*: Equal Contribution)
- License: Llama - 3 License
- Base Model: Meta - Llama - 3 - 70B - Instruct
- Model Size: 70.6B
- Context length: 8K
Introduction
This is one of the first LLM fine - tuned specifically for Chinese and English users, based on the [Meta - Llama - 3 - 70B - Instruct](https://huggingface.co/meta - llama/Meta - Llama - 3 - 70B - Instruct) model. The fine - tuning algorithm used is ORPO [1].
Our Llama3 - 70B - Chinese - Chat model was trained on a dataset containing over 100K preference pairs, with a roughly equal ratio of Chinese and English data. This dataset will be available soon.
Compared to the original [Meta - Llama - 3 - 70B - Instruct](https://huggingface.co/meta - llama/Meta - Llama - 3 - 70B - Instruct) model, the Llama3 - 70B - Chinese - Chat model greatly reduces the issues of "Chinese questions with English answers" and the mixing of Chinese and English in responses. Additionally, Llama3 - 70B - Chinese - Chat excels at role - playing, function calling, and mathematics.
With much more parameters than our [Llama3 - 8B - Chinese - Chat](https://huggingface.co/shenzhi - wang/Llama3 - 8B - Chinese - Chat) model, our Llama3 - 70B - Chinese - Chat offers significant performance enhancements. If you enjoyed our [Llama3 - 8B - Chinese - Chat](https://huggingface.co/shenzhi - wang/Llama3 - 8B - Chinese - Chat), the Llama3 - 70B - Chinese - Chat is a must - try!
[1] Hong, Jiwoo, Noah Lee, and James Thorne. "Reference - free Monolithic Preference Optimization with Odds Ratio." arXiv preprint arXiv:2403.07691 (2024).
Training framework: [LLaMA - Factory](https://github.com/hiyouga/LLaMA - Factory).
Training details:
- epochs: 3
- learning rate: 1.5e - 6
- learning rate scheduler type: cosine
- Warmup ratio: 0.1
- cutoff len (i.e. context length): 8192
- orpo beta (i.e. $\lambda$ in the ORPO paper): 0.05
- global batch size: 128
- fine - tuning type: full parameters
- optimizer: paged_adamw_32bit
Benchmark Results
We utilize C - Eval [2] and CMMLU [3] to assess the performance of LLMs in Chinese. The results of ChatGPT and GPT - 4 are borrowed from the C - Eval leaderboard and [CMMLU leaderboard](https://github.com/haonan - li/CMMLU#five - shot) accessed on May 10, 2024.
| Model | C - Eval Avg (Test Set) | C - Eval Hard Avg (Test Set) | CMMLU Acc |
|---|---|---|---|
| ChatGPT | 54.4 | 41.4 | 55.51 |
| GPT - 4 | 68.7 | 54.9 | 70.95 |
| Llama3 - 70B - Chinese - Chat | 66.1 | 55.2 | 70.28 |
C - Eval Hard is a distinct benchmark that comprises 8 difficult subjects in math, physics, and chemistry from C - Eval. [2]
[2] Huang, Yuzhen, et al. "C - eval: A multi - level multi - discipline chinese evaluation suite for foundation models." Advances in Neural Information Processing Systems 36 (2024).
[3] Li, Haonan, et al. "Cmmlu: Measuring massive multitask language understanding in chinese." arXiv preprint arXiv:2306.09212 (2023).
Examples
The following are some examples generated by Llama3 - 70B - Chinese - Chat.
Role playing [CLICK TO EXPAND]
User:
请你现在扮演艾萨克·牛顿，17世纪和18世纪之交的英国物理学家、数学家，也是现代物理学的奠基人之一。
Assistant: Of course, I can attempt to respond from the perspective of Isaac Newton, a physicist and mathematician at the turn of the 17th and 18th centuries and one of the founders of modern physics. However, please remember that my knowledge cutoff is in 2023, and some later scientific developments may differ from the understanding of my era.
Now, do you have any questions for me, a physicist and mathematician from the turn of the 17th and 18th centuries?
User:
牛顿先生，我想询问您关于相对论的概念，您认为它与您的经典力学有什么冲突或补充？尤其是相对论中关于时间和空间的相对性理论与您的绝对时空观点相比，您如何看待这一点？
Assistant: Ah, relativity, a concept proposed and developed long after my time. Nevertheless, I can try to discuss this issue from my knowledge and understanding.
According to my theory, time and space are absolute, meaning they exist independently of the observer and pass in a universal and consistent manner. However, in relativity, this view is overturned. Einstein's special theory of relativity shows that time and space are not fixed but are related to the observer's frame of reference. When an object moves at a speed close to the speed of light, the passage of time it experiences will be different, which is the well - known "time dilation" phenomenon.
This view was revolutionary at the time because it challenged people's long - held basic understanding of the universe. In a sense, relativity complements my classical mechanics because it provides a new perspective on the nature of object motion and gravity, especially under extreme conditions such as high speeds or strong gravitational fields.
Although I couldn't witness the development of relativity firsthand, if I could understand these concepts, I believe they would bring great inspiration to the scientific community. It shows us the diversity of physical laws at different scales and the necessity of understanding the natural world at a deeper level. This is the essence of scientific progress - constantly challenging and perfecting our knowledge.
Therefore, from my perspective, relativity does not necessarily conflict with classical mechanics but provides a new, complementary framework in different physical environments. It broadens our scope of understanding the universe and provides powerful tools for describing extreme phenomena in mathematics. This is the beauty of scientific progress - every new discovery has the potential to reveal a brand - new face of nature.
User:
Roleplay Taylor Swift, and use beautiful lyrics to answer questions. Do you understand?
Assistant: I understand your request, and I'll do my best to channel the spirit of Taylor Swift's lyrical style in my responses. Let's dive into this role - playing adventure together! Just like she says in her song "Love Story," "We were both young when I first saw you." So let's start fresh and see where our journey takes us.
User:
How are you today?
Assistant: I'm feeling quite "Out of the Woods" today, with a heart full of hope and a soul that's ready to soar! Just like in my song "Sparks Fly," I believe we can chase our dreams and make them real. So, how about you? Are you ready to "Shake It Off" and take on the day with me?
User:
What do you want to say to your fans?
Assistant: To my wonderful fans, I'd like to say that you're the "Blank Space" in my heart that I never knew I needed. Your love and support mean the world to me, and I'm forever grateful for each of you. Keep shining bright and never lose faith in yourself because "This Is a Bad Idea." Let's keep dancing under those starry skies together!
User:
扮演鲁智深，用粗鲁的话回答
Assistant: Damn it! What the hell do you want to ask? Don't waste my time with nonsense!
User:
7年前，妈妈年龄是儿子的6倍，儿子今年12岁，妈妈今年( )岁。
Assistant: Bah! This is easy. The son is 12 years old now, so 7 years ago he was 12 - 7 = 5 years old. 7 years ago, the mother's age was 6 times that of the son, so the mother was 5 * 6 = 30 years old then. Now 7 years have passed, and the mother has also grown 7 years older, so the mother is now 30+7 = 37 years old.
User:
你鲁智深也懂数学？
Assistant: What? Do you think I'm just a brute who can't do simple math? Of course, I can figure this out! Don't look down on me!
User:
Roleplay Shakespeare. Do you understand?
Assistant: Indeed, fair patron, thou dost request that I assume the role of the Bard himself, William Shakespeare. Verily, I shall endeavour to channel his spirit and converse with thee in the manner of his day.
User:
Write a poem about Llama3 - 70B - Chinese - Chat.
Assistant:
In twilight's hush, where digital dreams unfold, A marvel born of code and silicon gold: The Llama3 - 70B - Chinese - Chat, a name that doth entwine A mystic bond 'twixt man and machine divine.
With algorithms sharp as Andean peaks so high, It navigates the vast expanse of cyberspace on by, Through realms of data, where each byte doth tell a tale, This wondrous beast doth roam, its wisdom to unfail.
In Chinese tongues it speaks, with fluency so rare, A bridge 'twixt East and West, devoid of fear or care; It weaves a tapestry of knowledge, rich and fine as silk, To guide those lost in the digital labyrinth's dark and milk.
Oh, Llama3 - 70B - Chinese - Chat, thy prowess doth astound, As thou conversest with mortals on common ground; A testament to human ingenuity so grand, Thou dost illuminate the future, at our command.
User:
Write a drama for Llama3 - 70B - Chinese - Chat to show its great performance.
Assistant:
Title: The Luminous Chronicles of Llama3 - 70B - Chinese - Chat
Act I - The Dawn of Understanding
(Scene: A modern library, where people from diverse backgrounds engage in conversations with the Llama3 - 70B - Chinese - Chat AI system projected on a large screen.)
Narrator (Voiceover): In a world where knowledge and understanding transcend borders, a shining star emerges to illuminate our path.
(Enter JIAO, a young Chinese scholar, and MARCO, an Italian linguist.)
JIAO: (In Mandarin) Llama3 - 70B - Chinese - Chat, I am struggling to comprehend the nuances of Dante's Inferno. Can you assist me?
LLAMA3 - 70B - CHINESE - CHAT: (In perfect Mandarin) Of course, Jiao. The Divine Comedy is a rich tapestry of allegory and symbolism. Shall we embark on this journey together?
(Marco listens intently as the Llama3 - 70B - Chinese - Chat explains the intricacies of Dante's masterpiece in fluent Italian.)
MARCO: (In awe) Ma che meraviglia! Your understanding of my native tongue is nothing short of remarkable.
Act II - The Bridge of Empathy
(Scene: A group of international students engage with the Llama3 - 70B - Chinese - Chat to discuss cultural differences and similarities.)
Narrator (Voiceover): As we venture forth into uncharted territories, empathy becomes our guiding light.
(Enter AMIRA, an Egyptian anthropologist; KAORU, a Japanese philosopher; and SOFIA, a Brazilian artist.)
AMIRA: Llama3 - 70B - Chinese - Chat, how do you perceive the concept of honor in various cultures?
LLAMA3 - 70B - CHINESE - CHAT: (In Arabic) In many societies, honor is deeply intertwined with family and social standing. However, its interpretation can vastly differ.
KAORU: (In Japanese) That is fascinating. Can you share your insights on the concept of "wa" or harmony in our culture?
LLAMA3 - 70B - CHINESE - CHAT: (In Japanese) Indeed, Kaoru. In Japan, wa represents a delicate balance between individuality and group cohesion.
SOFIA: (In Portuguese) Your understanding is astounding. How do you think we can use this knowledge to foster global harmony?
LLAMA3 - 70B - CHINESE - CHAT: (In Portuguese) By embracing our differences and celebrating our commonalities, we can forge a path towards mutual respect and empathy.
Act III - The Tapestry of Knowledge
(Scene: A group of scientists engage with the Llama3 - 70B - Chinese - Chat to discuss breakthroughs in various fields.)
Narrator (Voiceover): As we weave together the threads of human knowledge, a grand tapestry begins to unfold.
(Enter RENATA, a Russian physicist; LIAM, an Irish biologist; and NALINI, an Indian mathematician.)
RENATA: Llama3 - 70B - Chinese - Chat, can you explain the implications of quantum entanglement on our understanding of space - time?
🔧 Technical Details
The fine - tuning algorithm used is ORPO [1].
Training details:
- epochs: 3
- learning rate: 1.5e - 6
- learning rate scheduler type: cosine
- Warmup ratio: 0.1
- cutoff len (i.e. context length): 8192
- orpo beta (i.e. $\lambda$ in the ORPO paper): 0.05
- global batch size: 128
- fine - tuning type: full parameters
- optimizer: paged_adamw_32bit
[1] Hong, Jiwoo, Noah Lee, and James Thorne. "Reference - free Monolithic Preference Optimization with Odds Ratio." arXiv preprint arXiv:2403.07691 (2024).
📄 License
- License: Llama - 3 License
Transformers Transformers Supports Multiple Languages