Llama 4 Scout 17B 16E Instruct GGUF

🚀 Llama-4-Scout-17B-16E-Instruct-GGUF

本项目提供了Llama-4-Scout-17B-16E-Instruct模型的量化版本，支持多语言，可通过LlamaEdge运行。

🚀 快速开始

运行环境

目前仅支持Llama-4的文本模式！

原始模型

unsloth/Llama-4-Scout-17B-16E-Instruct

使用LlamaEdge运行

版本要求

LlamaEdge版本需为 v0.16.16 及以上。

提示模板

• 提示类型：llama-4-chat
• 提示字符串

<|begin_of_text|><|header_start|>system<|header_end|>

{system_prompt}<|eot|><|header_start|>user<|header_end|>

{user_message_1}<|eot|><|header_start|>assistant<|header_end|>

{assistant_message_1}<|eot|><|header_start|>user<|header_end|>

{user_message_2}<|eot|>
<|header_start|>assistant<|header_end|>

• 示例：工具使用

  • 带有用户问题和工具信息的提示

    展开查看示例

    ```text <|begin_of_text|><|header_start|>system<|header_end|>

    You are a helpful assistant.<|eot|> <|header_start|>user<|header_end|>

    Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.

    Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables.

    [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": { "type": "string", "description": "The temperature unit to use. Infer this from the users location.", "enum": [ "celsius", "fahrenheit" ] } }, "required": [ "location", "unit" ] } } }, { "type": "function", "function": { "name": "predict_weather", "description": "Predict the weather in 24 hours", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g. San Francisco, CA" }, "unit": { "type": "string", "description": "The temperature unit to use. Infer this from the users location.", "enum": [ "celsius", "fahrenheit" ] } }, "required": [ "location", "unit" ] } } }, { "type": "function", "function": { "name": "sum", "description": "Calculate the sum of two numbers", "parameters": { "type": "object", "properties": { "a": { "type": "integer", "description": "the left hand side number" }, "b": { "type": "integer", "description": "the right hand side number" } }, "required": [ "a", "b" ] } } } ]

    Question: How is the weather of Beijing, China in celsius?<|eot|><|header_start|>assistant<|header_end|>

    </details>
    

  • 带有工具结果的提示

    展开查看示例

    ```text <|begin_of_text|><|header_start|>system<|header_end|>

    You are a helpful assistant.<|eot|> <|header_start|>user<|header_end|>

    How is the weather of Beijing, China in celsius?<|eot|> <|header_start|>assistant<|header_end|>

    {"name":"get_current_weather","arguments":"{"location":"Beijing, China","unit":"celsius"}"}<|eot|> <|header_start|>ipython<|header_end|>

    {"temperature":"30","unit":"celsius"}<|eot|><|header_start|>assistant<|header_end|>

    </details>
    
    

上下文大小

10M

作为LlamaEdge服务运行

wasmedge --dir .:. --nn-preload default:GGML:AUTO:Llama-4-Scout-17B-16E-Instruct-Q5_K_M.gguf \
  llama-api-server.wasm \
  --prompt-template llama-4-chat \
  --ctx-size 1000000 \
  --model-name Llama-4-Scout

作为LlamaEdge命令应用运行

wasmedge --dir .:. --nn-preload default:GGML:AUTO:Llama-4-Scout-17B-16E-Instruct-Q5_K_M.gguf \
  llama-chat.wasm \
  --prompt-template llama-4-chat \
  --ctx-size 1000000

📦 量化的GGUF模型

使用llama.cpp b5074进行量化。

📄 许可证

许可证类型：other 许可证名称：llama4

📚 详细文档