Mistral Small 3.2 24B Instruct 2506 GGUF
Mistral-Small-3.2-24B-Instruct-2506 是一個圖像文本到文本的模型,在模型量化方面表現出色,指令遵循、減少重複錯誤和函數調用等方面有顯著提升。
下載量 8,640
發布時間 : 6/20/2025
模型概述
該模型是一個多語言支持的圖像文本到文本模型,具有出色的量化性能和指令遵循能力,適用於多種任務。
模型特點
指令遵循
Small-3.2 更擅長遵循精確指令。
減少重複錯誤
Small-3.2 減少了無限生成或重複答案的情況。
函數調用
Small-3.2 的函數調用模板更加健壯。
量化性能
在模型量化方面達到了 SOTA 性能。
模型能力
圖像文本轉換
多語言支持
指令遵循
函數調用
減少重複錯誤
使用案例
視覺推理
圖像分析
分析圖像內容並生成文本描述或建議。
提供詳細的圖像分析和建議。
文本處理
文本重寫
將輸入文本重寫為更簡潔或更清晰的版本。
生成更簡潔的文本。
函數調用
動態函數調用
根據用戶需求動態調用函數完成任務。
成功調用函數並返回結果。
🚀 Mistral-Small-3.2-24B-Instruct-2506
Mistral-Small-3.2-24B-Instruct-2506 是一個圖像文本到文本的模型,在模型量化方面具有出色表現,且在指令遵循、減少重複錯誤和函數調用等方面有顯著提升。
🚀 快速開始
運行環境
支持多種語言,包括英語、法語、德語、西班牙語等。
許可證
採用 Apache-2.0 許可證。
運行代碼
在 llama.cpp 中運行
./llama.cpp/llama-cli -hf unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF:UD-Q4_K_XL --jinja --temp 0.15 --top-k -1 --top-p 1.00 -ngl 99
在 Ollama 中運行
ollama run hf.co/unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF:UD-Q4_K_XL
注意事項
⚠️ 重要提示
本模型包含 GGUF 聊天模板修復!工具調用也能正常工作!如果你使用
llama.cpp
,請使用--jinja
來啟用系統提示。
💡 使用建議
建議使用相對較低的溫度,例如
temperature=0.15
。同時,確保為模型添加系統提示,以滿足你的特定需求。如果將模型用作通用助手,建議使用 SYSTEM_PROMPT.txt 文件中提供的提示。
Unsloth Dynamic 2.0 在模型量化方面達到了 SOTA 性能。
✨ 主要特性
- 與 Mistral-Small-3.1-24B-Instruct-2503 具有相同的核心特性。
- 指令遵循:Small-3.2 更擅長遵循精確指令。
- 重複錯誤:Small-3.2 減少了無限生成或重複答案的情況。
- 函數調用:Small-3.2 的函數調用模板更加健壯。
📦 安裝指南
vLLM(推薦)
確保安裝 vLLM >= 0.9.1
:
pip install vllm --upgrade
安裝完成後,應會自動安裝 mistral_common >= 1.6.2
。你可以使用以下命令進行檢查:
python -c "import mistral_common; print(mistral_common.__version__)"
你還可以使用 docker 鏡像 或在 docker hub 上運行。
💻 使用示例
基礎用法
啟動服務器
vllm serve mistralai/Mistral-Small-3.2-24B-Instruct-2506 --tokenizer_mode mistral --config_format mistral --load_format mistral --tool-call-parser mistral --enable-auto-tool-choice --limit_mm_per_prompt 'image=10' --tensor-parallel-size 2
注意:在 GPU 上運行 Mistral-Small-3.2-24B-Instruct-2506 需要約 55 GB 的 GPU RAM(bf16 或 fp16)。
高級用法
視覺推理
from datetime import datetime, timedelta
from openai import OpenAI
from huggingface_hub import hf_hub_download
# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"
TEMP = 0.15
MAX_TOK = 131072
client = OpenAI(
api_key=openai_api_key,
base_url=openai_api_base,
)
models = client.models.list()
model = models.data[0].id
def load_system_prompt(repo_id: str, filename: str) -> str:
file_path = hf_hub_download(repo_id=repo_id, filename=filename)
with open(file_path, "r") as file:
system_prompt = file.read()
today = datetime.today().strftime("%Y-%m-%d")
yesterday = (datetime.today() - timedelta(days=1)).strftime("%Y-%m-%d")
model_name = repo_id.split("/")[-1]
return system_prompt.format(name=model_name, today=today, yesterday=yesterday)
model_id = "mistralai/Mistral-Small-3.2-24B-Instruct-2506"
SYSTEM_PROMPT = load_system_prompt(model_id, "SYSTEM_PROMPT.txt")
image_url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438"
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{
"role": "user",
"content": [
{
"type": "text",
"text": "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.",
},
{"type": "image_url", "image_url": {"url": image_url}},
],
},
]
response = client.chat.completions.create(
model=model,
messages=messages,
temperature=TEMP,
max_tokens=MAX_TOK,
)
print(response.choices[0].message.content)
# In this situation, you are playing a Pokémon game where your Pikachu (Level 42) is facing a wild Pidgey (Level 17). Here are the possible actions you can take and an analysis of each:
# 1. **FIGHT**:
# - **Pros**: Pikachu is significantly higher level than the wild Pidgey, which suggests that it should be able to defeat Pidgey easily. This could be a good opportunity to gain experience points and possibly items or money.
# - **Cons**: There is always a small risk of Pikachu fainting, especially if Pidgey has a powerful move or a status effect that could hinder Pikachu. However, given the large level difference, this risk is minimal.
# 2. **BAG**:
# - **Pros**: You might have items in your bag that could help in this battle, such as Potions, Poké Balls, or Berries. Using an item could help you capture the Pidgey or heal your Pikachu if needed.
# - **Cons**: Using items might not be necessary given the level difference. It could be more efficient to just fight and defeat the Pidgey quickly.
# 3. **POKÉMON**:
# - **Pros**: You might have another Pokémon in your party that is better suited for this battle or that you want to gain experience. Switching Pokémon could also be a strategic move if you want to train a lower-level Pokémon.
# - **Cons**: Switching Pokémon might not be necessary since Pikachu is at a significant advantage. It could also waste time and potentially give Pidgey a turn to attack.
# 4. **RUN**:
# - **Pros**: Running away could save time and conserve your Pokémon's health and resources. If you are in a hurry or do not need the experience or items, running away is a safe option.
# - **Cons**: Running away means you miss out on the experience points and potential items or money that you could gain from defeating the Pidgey. It also means you do not get the chance to capture the Pidgey if you wanted to.
# ### Recommendation:
# Given the significant level advantage, the best action is likely to **FIGHT**. This will allow you to quickly defeat the Pidgey, gain experience points, and potentially earn items or money. If you are concerned about Pikachu's health, you could use an item from your **BAG** to heal it before or during the battle. Running away or switching Pokémon does not seem necessary in this situation.
函數調用
from openai import OpenAI
from huggingface_hub import hf_hub_download
# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"
TEMP = 0.15
MAX_TOK = 131072
client = OpenAI(
api_key=openai_api_key,
base_url=openai_api_base,
)
models = client.models.list()
model = models.data[0].id
def load_system_prompt(repo_id: str, filename: str) -> str:
file_path = hf_hub_download(repo_id=repo_id, filename=filename)
with open(file_path, "r") as file:
system_prompt = file.read()
return system_prompt
model_id = "mistralai/Mistral-Small-3.2-24B-Instruct-2506"
SYSTEM_PROMPT = load_system_prompt(model_id, "SYSTEM_PROMPT.txt")
image_url = "https://huggingface.co/datasets/patrickvonplaten/random_img/resolve/main/europe.png"
tools = [
{
"type": "function",
"function": {
"name": "get_current_population",
"description": "Get the up-to-date population of a given country.",
"parameters": {
"type": "object",
"properties": {
"country": {
"type": "string",
"description": "The country to find the population of.",
},
"unit": {
"type": "string",
"description": "The unit for the population.",
"enum": ["millions", "thousands"],
},
},
"required": ["country", "unit"],
},
},
},
{
"type": "function",
"function": {
"name": "rewrite",
"description": "Rewrite a given text for improved clarity",
"parameters": {
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The input text to rewrite",
}
},
},
},
},
]
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{
"role": "user",
"content": "Could you please make the below article more concise?\n\nOpenAI is an artificial intelligence research laboratory consisting of the non-profit OpenAI Incorporated and its for-profit subsidiary corporation OpenAI Limited Partnership.",
},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "bbc5b7ede",
"type": "function",
"function": {
"name": "rewrite",
"arguments": '{"text": "OpenAI is an artificial intelligence research laboratory consisting of the non-profit OpenAI Incorporated and its for-profit subsidiary corporation OpenAI Limited Partnership."}',
},
}
],
},
{
"role": "tool",
"content": '{"action":"rewrite","outcome":"OpenAI is a FOR-profit company."}',
"tool_call_id": "bbc5b7ede",
"name": "rewrite",
},
{
"role": "assistant",
"content": "---\n\nOpenAI is a FOR-profit company.",
},
{
"role": "user",
"content": [
{
"type": "text",
"text": "Can you tell me what is the biggest country depicted on the map?",
},
{
"type": "image_url",
"image_url": {
"url": image_url,
},
},
],
}
]
response = client.chat.completions.create(
model=model,
messages=messages,
temperature=TEMP,
max_tokens=MAX_TOK,
tools=tools,
tool_choice="auto",
)
assistant_message = response.choices[0].message.content
print(assistant_message)
# The biggest country depicted on the map is Russia.
messages.extend([
{"role": "assistant", "content": assistant_message},
{"role": "user", "content": "What is the population of that country in millions?"},
])
response = client.chat.completions.create(
model=model,
messages=messages,
temperature=TEMP,
max_tokens=MAX_TOK,
tools=tools,
tool_choice="auto",
)
print(response.choices[0].message.tool_calls)
# [ChatCompletionMessageToolCall(id='3e92V6Vfo', function=Function(arguments='{"country": "Russia", "unit": "millions"}', name='get_current_population'), type='function')]
📚 詳細文檔
基準測試結果
將 Mistral-Small-3.2-24B 與 Mistral-Small-3.1-24B-Instruct-2503 進行了比較。如需與其他類似規模的模型進行更多比較,請查看 Mistral-Small-3.1 的基準測試。
文本測試
模型 | Wildbench v2 | Arena Hard v2 | IF(內部;準確率) |
---|---|---|---|
Small 3.1 24B Instruct | 55.6% | 19.56% | 82.75% |
Small 3.2 24B Instruct | 65.33% | 43.1% | 84.78% |
無限生成測試
Small 3.2 在具有挑戰性、長且重複的提示下,將無限生成情況減少了 2 倍。
模型 | 無限生成(內部;越低越好) |
---|---|
Small 3.1 24B Instruct | 2.11% |
Small 3.2 24B Instruct | 1.29% |
STEM 測試
模型 | MMLU | MMLU Pro(5-shot CoT) | MATH | GPQA Main(5-shot CoT) | GPQA Diamond(5-shot CoT) | MBPP Plus - Pass@5 | HumanEval Plus - Pass@5 | SimpleQA(總準確率) |
---|---|---|---|---|---|---|---|---|
Small 3.1 24B Instruct | 80.62% | 66.76% | 69.30% | 44.42% | 45.96% | 74.63% | 88.99% | 10.43% |
Small 3.2 24B Instruct | 80.50% | 69.06% | 69.42% | 44.22% | 46.13% | 78.33% | 92.90% | 12.10% |
視覺測試
模型 | MMMU | Mathvista | ChartQA | DocVQA | AI2D |
---|---|---|---|---|---|
Small 3.1 24B Instruct | 64.00% | 68.91% | 86.24% | 94.08% | 93.72% |
Small 3.2 24B Instruct | 62.50% | 67.09% | 87.4% | 94.86% | 92.91% |
🔧 技術細節
基礎模型
- mistralai/Mistral-Small-3.2-24B-Instruct-2506
支持語言
- 英語、法語、德語、西班牙語、葡萄牙語、意大利語、日語、韓語、俄語、中文、阿拉伯語、波斯語、印尼語、馬來語、尼泊爾語、波蘭語、羅馬尼亞語、塞爾維亞語、瑞典語、土耳其語、烏克蘭語、越南語、印地語、孟加拉語
推理設置
推理功能未啟用。
管道標籤
image-text-to-text
📄 許可證
本項目採用 Apache-2.0 許可證。
Clip Vit Large Patch14
CLIP是由OpenAI開發的視覺-語言模型,通過對比學習將圖像和文本映射到共享的嵌入空間,支持零樣本圖像分類
圖像生成文本
C
openai
44.7M
1,710
Clip Vit Base Patch32
CLIP是由OpenAI開發的多模態模型,能夠理解圖像和文本之間的關係,支持零樣本圖像分類任務。
圖像生成文本
C
openai
14.0M
666
Siglip So400m Patch14 384
Apache-2.0
SigLIP是基於WebLi數據集預訓練的視覺語言模型,採用改進的sigmoid損失函數,優化了圖像-文本匹配任務。
圖像生成文本
Transformers

S
google
6.1M
526
Clip Vit Base Patch16
CLIP是由OpenAI開發的多模態模型,通過對比學習將圖像和文本映射到共享的嵌入空間,實現零樣本圖像分類能力。
圖像生成文本
C
openai
4.6M
119
Blip Image Captioning Base
Bsd-3-clause
BLIP是一個先進的視覺-語言預訓練模型,擅長圖像描述生成任務,支持條件式和非條件式文本生成。
圖像生成文本
Transformers

B
Salesforce
2.8M
688
Blip Image Captioning Large
Bsd-3-clause
BLIP是一個統一的視覺-語言預訓練框架,擅長圖像描述生成任務,支持條件式和無條件式圖像描述生成。
圖像生成文本
Transformers

B
Salesforce
2.5M
1,312
Openvla 7b
MIT
OpenVLA 7B是一個基於Open X-Embodiment數據集訓練的開源視覺-語言-動作模型,能夠根據語言指令和攝像頭圖像生成機器人動作。
圖像生成文本
Transformers 英語

O
openvla
1.7M
108
Llava V1.5 7b
LLaVA 是一款開源多模態聊天機器人,基於 LLaMA/Vicuna 微調,支持圖文交互。
圖像生成文本
Transformers

L
liuhaotian
1.4M
448
Vit Gpt2 Image Captioning
Apache-2.0
這是一個基於ViT和GPT2架構的圖像描述生成模型,能夠為輸入圖像生成自然語言描述。
圖像生成文本
Transformers

V
nlpconnect
939.88k
887
Blip2 Opt 2.7b
MIT
BLIP-2是一個視覺語言模型,結合了圖像編碼器和大型語言模型,用於圖像到文本的生成任務。
圖像生成文本
Transformers 英語

B
Salesforce
867.78k
359
精選推薦AI模型
Llama 3 Typhoon V1.5x 8b Instruct
專為泰語設計的80億參數指令模型,性能媲美GPT-3.5-turbo,優化了應用場景、檢索增強生成、受限生成和推理任務
大型語言模型
Transformers 支持多種語言

L
scb10x
3,269
16
Cadet Tiny
Openrail
Cadet-Tiny是一個基於SODA數據集訓練的超小型對話模型,專為邊緣設備推理設計,體積僅為Cosmo-3B模型的2%左右。
對話系統
Transformers 英語

C
ToddGoldfarb
2,691
6
Roberta Base Chinese Extractive Qa
基於RoBERTa架構的中文抽取式問答模型,適用於從給定文本中提取答案的任務。
問答系統 中文
R
uer
2,694
98