Mixtral 8x7B Instruct V0.1

🚀 Mixtral-8x7B大語言模型

Mixtral-8x7B大語言模型（LLM）是一個預訓練的生成式稀疏專家混合模型。在我們測試的大多數基準測試中，Mixtral-8x7B的表現優於Llama 2 70B。

🚀 快速開始

模型信息

⚠️ 重要提示

如果你想了解更多關於我們如何處理你的個人數據的信息，請閱讀我們的隱私政策。

✨ 主要特性

Mixtral-8x7B是一個強大的大語言模型，在多個基準測試中表現出色。它可以通過不同的方式進行推理，包括使用mistral_inference、Hugging Face的transformers庫等。同時，它還支持不同的精度設置，以滿足不同的內存和性能需求。

📦 安裝指南

文檔未提及具體安裝步驟，暫不提供。

💻 使用示例

基礎用法

使用mistral-common進行分詞

from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from mistral_common.protocol.instruct.messages import UserMessage
from mistral_common.protocol.instruct.request import ChatCompletionRequest
 
mistral_models_path = "MISTRAL_MODELS_PATH"
 
tokenizer = MistralTokenizer.v1()
 
completion_request = ChatCompletionRequest(messages=[UserMessage(content="Explain Machine Learning to me in a nutshell.")])
 
tokens = tokenizer.encode_chat_completion(completion_request).tokens

使用mistral_inference進行推理

from mistral_inference.transformer import Transformer
from mistral_inference.generate import generate
 
model = Transformer.from_folder(mistral_models_path)
out_tokens, _ = generate([tokens], model, max_tokens=64, temperature=0.0, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)

result = tokenizer.decode(out_tokens[0])

print(result)

使用Hugging Face的transformers進行推理

from transformers import AutoModelForCausalLM
 
model = AutoModelForCausalLM.from_pretrained("mistralai/Mixtral-8x7B-Instruct-v0.1")
model.to("cuda")
 
generated_ids = model.generate(tokens, max_new_tokens=1000, do_sample=True)

# decode with mistral tokenizer
result = tokenizer.decode(generated_ids[0].tolist())
print(result)

高級用法

運行模型

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mistralai/Mixtral-8x7B-Instruct-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

messages = [
    {"role": "user", "content": "What is your favourite condiment?"},
    {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
    {"role": "user", "content": "Do you have mayonnaise recipes?"}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")

outputs = model.generate(inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

半精度推理

注意：float16精度僅適用於GPU設備。

+ import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mistralai/Mixtral-8x7B-Instruct-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)

+ model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")

messages = [
    {"role": "user", "content": "What is your favourite condiment?"},
    {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
    {"role": "user", "content": "Do you have mayonnaise recipes?"}
]

input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")

outputs = model.generate(input_ids, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

使用bitsandbytes進行低精度（8位和4位）推理

+ import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mistralai/Mixtral-8x7B-Instruct-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)

+ model = AutoModelForCausalLM.from_pretrained(model_id, load_in_4bit=True, device_map="auto")

text = "Hello my name is"
messages = [
    {"role": "user", "content": "What is your favourite condiment?"},
    {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
    {"role": "user", "content": "Do you have mayonnaise recipes?"}
]

input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")

outputs = model.generate(input_ids, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

使用Flash Attention 2加載模型

+ import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mistralai/Mixtral-8x7B-Instruct-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)

+ model = AutoModelForCausalLM.from_pretrained(model_id, use_flash_attention_2=True, device_map="auto")

messages = [
    {"role": "user", "content": "What is your favourite condiment?"},
    {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
    {"role": "user", "content": "Do you have mayonnaise recipes?"}
]

input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")

outputs = model.generate(input_ids, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

📚 詳細文檔

指令格式

此格式必須嚴格遵守，否則模型將生成不理想的輸出。 用於為指令模型構建提示的模板定義如下：

<s> [INST] 指令 [/INST] 模型回答</s> [INST] 後續指令 [/INST]

請注意，<s>和</s>是字符串開頭（BOS）和字符串結尾（EOS）的特殊標記，而[INST]和[/INST]是常規字符串。

作為參考，以下是在微調期間用於對指令進行分詞的偽代碼：

def tokenize(text):
    return tok.encode(text, add_special_tokens=False)

[BOS_ID] + 
tokenize("[INST]") + tokenize(用戶消息1) + tokenize("[/INST]") +
tokenize(模型回答1) + [EOS_ID] +
…
tokenize("[INST]") + tokenize(用戶消息N) + tokenize("[/INST]") +
tokenize(模型回答N) + [EOS_ID]

在上述偽代碼中，請注意tokenize方法不應自動添加BOS或EOS標記，但應添加前綴空格。

在Transformers庫中，可以使用聊天模板來確保應用正確的格式。

模型侷限性

Mixtral-8x7B指令模型是一個快速演示，表明基礎模型可以很容易地進行微調以實現出色的性能。它沒有任何審核機制。我們期待與社區合作，探討如何使模型更好地遵守規則，以便在需要審核輸出的環境中進行部署。

🔧 技術細節

文檔未提及具體技術細節，暫不提供。

📄 許可證

本項目採用Apache-2.0許可證。

團隊信息

Mistral AI團隊成員包括：Albert Jiang、Alexandre Sablayrolles、Arthur Mensch、Blanche Savary、Chris Bamford、Devendra Singh Chaplot、Diego de las Casas、Emma Bou Hanna、Florian Bressand、Gianna Lengyel、Guillaume Bour、Guillaume Lample、Lélio Renard Lavaud、Louis Ternon、Lucile Saulnier、Marie-Anne Lachaux、Pierre Stock、Teven Le Scao、Théophile Gervet、Thibaut Lavril、Thomas Wang、Timothée Lacroix、William El Sayed。

💡 使用建議

非常歡迎提交PR來修正transformers分詞器，使其與mistral-common參考實現的結果完全一致！