Devstral Small 2505

🚀 Devstral-Small-2505

Devstral is an agentic LLM designed for software engineering tasks. It's the result of a collaboration between Mistral AI and All Hands AI. Devstral shines in using tools to explore codebases, edit multiple files, and power software engineering agents. It achieves remarkable performance on SWE-bench, positioning it as the #1 open source model on this benchmark.

It's fine - tuned from Mistral-Small-3.1, boasting a long context window of up to 128k tokens. As a coding agent, Devstral is text - only, and the vision encoder was removed before fine - tuning from Mistral-Small-3.1.

For enterprises needing specialized capabilities (such as increased context or domain - specific knowledge), we'll release commercial models beyond what Mistral AI contributes to the community.

Learn more about Devstral in our blog post.

✨ Features

• Agentic coding: Devstral is tailored for agentic coding tasks, making it an excellent choice for software engineering agents.
• Lightweight: With just 24 billion parameters, Devstral is light enough to run on a single RTX 4090 or a Mac with 32GB RAM, suitable for local deployment and on - device use.
• Apache 2.0 License: An open license that allows usage and modification for both commercial and non - commercial purposes.
• Context Window: It has a 128k context window.
• Tokenizer: Utilizes a Tekken tokenizer with a 131k vocabulary size.

📚 Documentation

📊 Benchmark Results

SWE - Bench

Devstral scores 46.8% on SWE - Bench Verified, outperforming the prior open - source SoTA by 6%.

When evaluated under the same test scaffold (OpenHands, provided by All Hands AI), Devstral outperforms much larger models like Deepseek - V3 - 0324 and Qwen3 232B - A22B.

💻 Usage Examples

We recommend using Devstral with the [OpenHands](https://github.com/All - Hands - AI/OpenHands/tree/main) scaffold. You can use it either through our API or by running it locally.

🌐 API

Follow these [instructions](https://docs.mistral.ai/getting - started/quickstart/#account - setup) to create a Mistral account and get an API key.

Then run these commands to start the OpenHands docker container.

export MISTRAL_API_KEY=<MY_KEY>

docker pull docker.all - hands.dev/all - hands - ai/runtime:0.39 - nikolaik

mkdir -p ~/.openhands - state && echo '{"language":"en","agent":"CodeActAgent","max_iterations":null,"security_analyzer":null,"confirmation_mode":false,"llm_model":"mistral/devstral - small - 2505","llm_api_key":"'$MISTRAL_API_KEY'","remote_runtime_resource_factor":null,"github_token":null,"enable_default_condenser":true}' > ~/.openhands - state/settings.json

docker run -it --rm --pull=always \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all - hands.dev/all - hands - ai/runtime:0.39 - nikolaik \
    -e LOG_ALL_EVENTS=true \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v ~/.openhands - state:/.openhands - state \
    -p 3000:3000 \
    --add - host host.docker.internal:host - gateway \
    --name openhands - app \
    docker.all - hands.dev/all - hands - ai/openhands:0.39

💻 Local inference

The model can be deployed with the following libraries:

• [vllm (recommended)](https://github.com/vllm - project/vllm): See [here](#vllm - recommended)
• [mistral - inference](https://github.com/mistralai/mistral - inference): See [here](#mistral - inference)
• transformers: See here
• LMStudio: See here
• [llama.cpp](https://github.com/ggml - org/llama.cpp): See here
• ollama: See here

🚀 OpenHands (recommended)

Launch a server to deploy Devstral - Small - 2505

Make sure you've launched an OpenAI - compatible server like vLLM or Ollama as described above. Then, you can use OpenHands to interact with Devstral - Small - 2505.

In the tutorial, we spin up a vLLM server with the command:

vllm serve mistralai/Devstral - Small - 2505 --tokenizer_mode mistral --config_format mistral --load_format mistral --tool - call - parser mistral --enable - auto - tool - choice --tensor - parallel - size 2

The server address should be in the format: http://<your - server - url>:8000/v1

Launch OpenHands

You can follow the OpenHands installation [here](https://docs.all - hands.dev/modules/usage/installation).

The easiest way to launch OpenHands is to use the Docker image:

docker pull docker.all - hands.dev/all - hands - ai/runtime:0.38 - nikolaik

docker run -it --rm --pull=always \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all - hands.dev/all - hands - ai/runtime:0.38 - nikolaik \
    -e LOG_ALL_EVENTS=true \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v ~/.openhands - state:/.openhands - state \
    -p 3000:3000 \
    --add - host host.docker.internal:host - gateway \
    --name openhands - app \
    docker.all - hands.dev/all - hands - ai/openhands:0.38

Then, you can access the OpenHands UI at http://localhost:3000.

Connect to the server

When accessing the OpenHands UI, you'll be prompted to connect to a server. You can use the advanced mode to connect to the server you launched earlier.

Fill in the following fields:

• Custom Model: openai/mistralai/Devstral - Small - 2505
• Base URL: http://<your - server - url>:8000/v1
• API Key: token (or any other token you used to launch the server if applicable)

Use OpenHands powered by Devstral

Now you're ready to use Devstral Small inside OpenHands by starting a new conversation. Let's build a To - Do list app.

Build a To - Do list app with the following requirements:
- Built using FastAPI and React.
- Make it a one - page app that:
   - Allows to add a task.
   - Allows to delete a task.
   - Allows to mark a task as done.
   - Displays the list of tasks.
- Store the tasks in a SQLite database.

🚀 vLLM (recommended)

We recommend using this model with the [vLLM library](https://github.com/vllm - project/vllm) to implement production - ready inference pipelines.

Installation Make sure you install [vLLM >= 0.8.5](https://github.com/vllm - project/vllm/releases/tag/v0.8.5):

pip install vllm --upgrade

Doing so should automatically install [mistral_common >= 1.5.5](https://github.com/mistralai/mistral - common/releases/tag/v1.5.5).

To check:

python -c "import mistral_common; print(mistral_common.__version__)"

You can also use a ready - to - go [docker image](https://github.com/vllm - project/vllm/blob/main/Dockerfile) or the one on the [docker hub](https://hub.docker.com/layers/vllm/vllm - openai/latest/images/sha256 - de9032a92ffea7b5c007dad80b38fd44aac11eddc31c435f8e52f3b7404bbf39).

Server

We recommend using Devstral in a server/client setting.

1. Spin up a server:

vllm serve mistralai/Devstral - Small - 2505 --tokenizer_mode mistral --config_format mistral --load_format mistral --tool - call - parser mistral --enable - auto - tool - choice --tensor - parallel - size 2

2. To ping the client, you can use a simple Python snippet.

import requests
import json
from huggingface_hub import hf_hub_download


url = "http://<your - server - url>:8000/v1/chat/completions"
headers = {"Content - Type": "application/json", "Authorization": "Bearer token"}

model = "mistralai/Devstral - Small - 2505"

def load_system_prompt(repo_id: str, filename: str) -> str:
    file_path = hf_hub_download(repo_id = repo_id, filename = filename)
    with open(file_path, "r") as file:
        system_prompt = file.read()
    return system_prompt

SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt")

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "<your - command>",
            },
        ],
    },
]

data = {"model": model, "messages": messages, "temperature": 0.15}

response = requests.post(url, headers = headers, data = json.dumps(data))
print(response.json()["choices"][0]["message"]["content"])

🚀 Mistral - inference

We recommend using mistral - inference to quickly try out / "vibe - check" Devstral.

Install

Make sure to have mistral_inference >= 1.6.0 installed.

pip install mistral_inference --upgrade

Download

from huggingface_hub import snapshot_download
from pathlib import Path

mistral_models_path = Path.home().joinpath('mistral_models', 'Devstral')
mistral_models_path.mkdir(parents = True, exist_ok = True)

snapshot_download(repo_id = "mistralai/Devstral - Small - 2505", allow_patterns = ["params.json", "consolidated.safetensors", "tekken.json"], local_dir = mistral_models_path)

Python

You can run the model using the following command:

mistral - chat $HOME/mistral_models/Devstral --instruct --max_tokens 300

You can then prompt it with anything you'd like.

🚀 Transformers

To make the best use of our model with transformers, make sure to have [installed](https://github.com/mistralai/mistral - common) mistral - common >= 1.5.5 to use our tokenizer.

pip install mistral - common --upgrade

Then load our tokenizer along with the model and generate:

import torch

from mistral_common.protocol.instruct.messages import (
    SystemMessage, UserMessage
)
from mistral_common.protocol.instruct.request import ChatCompletionRequest
from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from huggingface_hub import hf_hub_download
from transformers import AutoModelForCausalLM

def load_system_prompt(repo_id: str, filename: str) -> str:
    file_path = hf_hub_download(repo_id = repo_id, filename = filename)
    with open(file_path, "r") as file:
        system_prompt = file.read()
    return system_prompt

model_id = "mistralai/Devstral - Small - 2505"
tekken_file = hf_hub_download(repo_id = model_id, filename = "tekken.json")
SYSTEM_PROMPT = load_system_prompt(model_id, "SYSTEM_PROMPT.txt")

tokenizer = MistralTokenizer.from_file(tekken_file)

model = AutoModelForCausalLM.from_pretrained(model_id)

tokenized = tokenizer.encode_chat_completion(
    ChatCompletionRequest(
        messages = [
            SystemMessage(content = SYSTEM_PROMPT),
            UserMessage(content = "<your - command>"),
        ],
    )
)

output = model.generate(
    input_ids = torch.tensor([tokenized.tokens]),
    max_new_tokens = 1000,
)[0]

decoded_output = tokenizer.decode(output[len(tokenized.tokens):])
print(decoded_output)

🚀 LMStudio

Download the weights from huggingface:

pip install -U "huggingface_hub[cli]"
huggingface-cli download \
"mistralai/Devstral - Small - 2505_gguf" \
--include "devstralQ4_K_M.gguf" \
--local-dir "mistralai/Devstral - Small - 2505_gguf/"

You can serve the model locally with LMStudio.

• Download LM Studio and install it
• Install lms cli ~/.lmstudio/bin/lms bootstrap
• In a bash terminal, run lms import devstralQ4_K_M.gguf in the directory where you've downloaded the model check

📄 License

This project is licensed under the Apache 2.0 license.

⚠️ Important Note

If you want to learn more about how we process your personal data, please read our Privacy Policy.