Mistral 7B Instruct V0.2

🚀 Mistral-7B-Instruct-v0.2模型卡片

Mistral-7B-Instruct-v0.2大語言模型（LLM）是Mistral-7B-v0.2的指令微調版本，能有效處理文本生成任務。

🚀 快速開始

✨ 主要特性

• Mistral-7B-v0.2與Mistral-7B-v0.1相比，有以下改進：
  • 上下文窗口從8k提升到32k。
  • Rope-theta設置為1e6。
  • 不使用滑動窗口注意力機制。

📦 安裝指南

若遇到以下錯誤：

Traceback (most recent call last):
File "", line 1, in
File "/transformers/models/auto/auto_factory.py", line 482, in from_pretrained
config, kwargs = AutoConfig.from_pretrained(
File "/transformers/models/auto/configuration_auto.py", line 1022, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
File "/transformers/models/auto/configuration_auto.py", line 723, in getitem
raise KeyError(key)
KeyError: 'mistral'

從源代碼安裝transformers可解決此問題：

pip install git+https://github.com/huggingface/transformers

在transformers-v4.33.4之後，此步驟應該不再需要。

💻 使用示例

基礎用法

以下是使用mistral_common進行編碼和解碼的示例：

from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from mistral_common.protocol.instruct.messages import UserMessage
from mistral_common.protocol.instruct.request import ChatCompletionRequest

mistral_models_path = "MISTRAL_MODELS_PATH"

tokenizer = MistralTokenizer.v1()

completion_request = ChatCompletionRequest(messages=[UserMessage(content="Explain Machine Learning to me in a nutshell.")])

tokens = tokenizer.encode_chat_completion(completion_request).tokens

高級用法

以下是使用mistral_inference進行推理的示例：

from mistral_inference.transformer import Transformer
from mistral_inference.generate import generate

model = Transformer.from_folder(mistral_models_path)
out_tokens, _ = generate([tokens], model, max_tokens=64, temperature=0.0, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)

result = tokenizer.decode(out_tokens[0])

print(result)

與Hugging Face transformers結合使用

from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
model.to("cuda")

generated_ids = model.generate(tokens, max_new_tokens=1000, do_sample=True)

# decode with mistral tokenizer
result = tokenizer.decode(generated_ids[0].tolist())
print(result)

🔧 技術細節

指令格式

為了利用指令微調，你的提示應使用[INST]和[/INST]標記包圍。第一條指令應從句子起始ID開始，後續指令則不需要。助手生成的內容將以句子結束ID結束。

例如：

text = "<s>[INST] What is your favourite condiment? [/INST]"
"Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!</s> "
"[INST] Do you have mayonnaise recipes? [/INST]"

這種格式可通過apply_chat_template()方法作為聊天模板使用：

from transformers import AutoModelForCausalLM, AutoTokenizer

device = "cuda" # the device to load the model onto

model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")

messages = [
    {"role": "user", "content": "What is your favourite condiment?"},
    {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
    {"role": "user", "content": "Do you have mayonnaise recipes?"}
]

encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")

model_inputs = encodeds.to(device)
model.to(device)

generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
decoded = tokenizer.batch_decode(generated_ids)
print(decoded[0])

📄 許可證

本模型採用Apache-2.0許可證。

注意事項

⚠️ 重要提示 若你想了解我們如何處理你的個人數據，請閱讀我們的隱私政策。

💡 使用建議 非常歡迎提交PR來修正transformers分詞器，使其結果與mistral_common參考實現完全一致！

侷限性

Mistral 7B Instruct模型是一個快速演示，表明基礎模型可以輕鬆進行微調以實現出色的性能。該模型沒有任何審核機制。我們期待與社區合作，使模型能夠更好地遵循規則，以便在需要審核輸出的環境中部署。

開發團隊

Albert Jiang, Alexandre Sablayrolles, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Louis Ternon, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, William El Sayed.