Lake 1 Advanced

🚀 Mistral-7B-Instruct-v0.3模型卡片

Mistral-7B-Instruct-v0.3大语言模型（LLM）是Mistral-7B-v0.3的指令微调版本。

与Mistral-7B-v0.2相比，Mistral-7B-v0.3有以下变化：

• 词汇表扩展到32768
• 支持v3分词器
• 支持函数调用

🚀 快速开始

安装

建议将mistralai/Mistral-7B-Instruct-v0.3与mistral-inference一起使用。有关HF transformers的代码片段，请继续向下滚动。

pip install mistral_inference

下载

from huggingface_hub import snapshot_download
from pathlib import Path

mistral_models_path = Path.home().joinpath('mistral_models', '7B-Instruct-v0.3')
mistral_models_path.mkdir(parents=True, exist_ok=True)

snapshot_download(repo_id="mistralai/Mistral-7B-Instruct-v0.3", allow_patterns=["params.json", "consolidated.safetensors", "tokenizer.model.v3"], local_dir=mistral_models_path)

聊天

安装mistral_inference后，你的环境中应该会有一个mistral-chat CLI命令。你可以使用以下命令与模型进行聊天：

mistral-chat $HOME/mistral_models/7B-Instruct-v0.3 --instruct --max_tokens 256

指令跟随

from mistral_inference.transformer import Transformer
from mistral_inference.generate import generate

from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from mistral_common.protocol.instruct.messages import UserMessage
from mistral_common.protocol.instruct.request import ChatCompletionRequest


tokenizer = MistralTokenizer.from_file(f"{mistral_models_path}/tokenizer.model.v3")
model = Transformer.from_folder(mistral_models_path)

completion_request = ChatCompletionRequest(messages=[UserMessage(content="Explain Machine Learning to me in a nutshell.")])

tokens = tokenizer.encode_chat_completion(completion_request).tokens

out_tokens, _ = generate([tokens], model, max_tokens=64, temperature=0.0, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)
result = tokenizer.instruct_tokenizer.tokenizer.decode(out_tokens[0])

print(result)

函数调用

from mistral_common.protocol.instruct.tool_calls import Function, Tool
from mistral_inference.transformer import Transformer
from mistral_inference.generate import generate

from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from mistral_common.protocol.instruct.messages import UserMessage
from mistral_common.protocol.instruct.request import ChatCompletionRequest


tokenizer = MistralTokenizer.from_file(f"{mistral_models_path}/tokenizer.model.v3")
model = Transformer.from_folder(mistral_models_path)

completion_request = ChatCompletionRequest(
    tools=[
        Tool(
            function=Function(
                name="get_current_weather",
                description="Get the current weather",
                parameters={
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA",
                        },
                        "format": {
                            "type": "string",
                            "enum": ["celsius", "fahrenheit"],
                            "description": "The temperature unit to use. Infer this from the users location.",
                        },
                    },
                    "required": ["location", "format"],
                },
            )
        )
    ],
    messages=[
        UserMessage(content="What's the weather like today in Paris?"),
        ],
)

tokens = tokenizer.encode_chat_completion(completion_request).tokens

out_tokens, _ = generate([tokens], model, max_tokens=64, temperature=0.0, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)
result = tokenizer.instruct_tokenizer.tokenizer.decode(out_tokens[0])

print(result)

使用transformers生成文本

如果你想使用Hugging Face的transformers库来生成文本，可以这样做：

from transformers import pipeline

messages = [
    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
    {"role": "user", "content": "Who are you?"},
]
chatbot = pipeline("text-generation", model="mistralai/Mistral-7B-Instruct-v0.3")
chatbot(messages)

使用transformers进行函数调用

要使用此示例，你需要transformers版本4.42.0或更高版本。有关更多信息，请参阅transformers文档中的函数调用指南。

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "mistralai/Mistral-7B-Instruct-v0.3"
tokenizer = AutoTokenizer.from_pretrained(model_id)

def get_current_weather(location: str, format: str):
    """
    Get the current weather

    Args:
        location: The city and state, e.g. San Francisco, CA
        format: The temperature unit to use. Infer this from the users location. (choices: ["celsius", "fahrenheit"])
    """
    pass

conversation = [{"role": "user", "content": "What's the weather like in Paris?"}]
tools = [get_current_weather]


# format and tokenize the tool use prompt 
inputs = tokenizer.apply_chat_template(
            conversation,
            tools=tools,
            add_generation_prompt=True,
            return_dict=True,
            return_tensors="pt",
)

model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")

inputs.to(model.device)
outputs = model.generate(**inputs, max_new_tokens=1000)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

请注意，由于篇幅原因，此示例未展示调用工具并将工具调用和工具结果添加到聊天历史记录的完整循环，以便模型在下次生成时可以使用它们。有关完整的工具调用示例，请参阅函数调用指南，并注意Mistral 确实 使用工具调用ID，因此这些ID必须包含在你的工具调用和工具结果中。它们应该正好是9个字母数字字符。

🔧 技术细节

Mistral 7B指令模型是一个快速演示，表明基础模型可以很容易地进行微调以实现出色的性能。它没有任何审核机制。我们期待与社区合作，探讨如何使模型更好地遵守规则，以便在需要审核输出的环境中进行部署。

📄 许可证

本项目采用MIT许可证。

团队成员

Albert Jiang、Alexandre Sablayrolles、Alexis Tacnet、Antoine Roux、Arthur Mensch、Audrey Herblin - Stoop、Baptiste Bout、Baudouin de Monicault、Blanche Savary、Bam4d、Caroline Feldman、Devendra Singh Chaplot、Diego de las Casas、Eleonore Arcelin、Emma Bou Hanna、Etienne Metzger、Gianna Lengyel、Guillaume Bour、Guillaume Lample、Harizo Rajaona、Jean - Malo Delignon、Jia Li、Justus Murke、Louis Martin、Louis Ternon、Lucile Saulnier、Lélio Renard Lavaud、Margaret Jennings、Marie Pellat、Marie Torelli、Marie - Anne Lachaux、Nicolas Schuhl、Patrick von Platen、Pierre Stock、Sandeep Subramanian、Sophia Yang、Szymon Antoniak、Teven Le Scao、Thibaut Lavril、Timothée Lacroix、Théophile Gervet、Thomas Wang、Valera Nemychnikova、William El Sayed、William Marshall

⚠️ 重要提示

如果你想了解更多关于我们如何处理你的个人数据的信息，请阅读我们的隐私政策。