Deepseek Coder V2 Instruct 0724

🚀 DeepSeek-Coder-V2：突破代码智能领域闭源模型的壁垒

DeepSeek-Coder-V2是一个开源的混合专家（MoE）代码语言模型，在特定代码任务中表现可与GPT4-Turbo相媲美。它在DeepSeek-V2的中间检查点基础上，额外使用6万亿个标记进行进一步预训练，显著提升了DeepSeek-V2的编码和数学推理能力，同时在通用语言任务中保持了相当的性能。与DeepSeek-Coder-33B相比，DeepSeek-Coder-V2在各种代码相关任务、推理和通用能力方面都有显著进步。此外，它支持的编程语言从86种扩展到338种，上下文长度从16K扩展到128K。

🚀 快速开始

模型下载

我们基于DeepSeekMoE框架，向公众发布了具有16B和236B参数的DeepSeek-Coder-V2，其激活参数仅为2.4B和21B，包括基础模型和指令模型。

在线体验

你可以在DeepSeek的官方网站上与DeepSeek-Coder-V2进行对话：coder.deepseek.com

API平台

我们还在DeepSeek平台上提供了与OpenAI兼容的API：platform.deepseek.com，你可以按需付费使用，价格极具竞争力。

✨ 主要特性

• 性能卓越：在标准基准评估中，DeepSeek-Coder-V2在编码和数学基准测试中表现优于GPT4-Turbo、Claude 3 Opus和Gemini 1.5 Pro等闭源模型。
• 语言支持广泛：支持的编程语言从86种扩展到338种，具体支持的语言列表可查看此处。
• 上下文长度扩展：上下文长度从16K扩展到128K，能够处理更长的输入。
• 新功能丰富：支持函数调用、JSON输出和FIM补全。

📦 安装指南

若要在本地使用DeepSeek-Coder-V2-Lite模型进行推理，使用BF16格式时需要80GB * 8的GPU。

💻 使用示例

基于Huggingface的Transformers进行推理

你可以直接使用Huggingface的Transformers进行模型推理。

代码补全

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-Coder-V2-Lite-Base", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-Coder-V2-Lite-Base", trust_remote_code=True, torch_dtype=torch.bfloat16).cuda()
input_text = "#write a quick sort algorithm"
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_length=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

代码插入

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-Coder-V2-Lite-Base", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-Coder-V2-Lite-Base", trust_remote_code=True, torch_dtype=torch.bfloat16).cuda()
input_text = """<｜fim▁begin｜>def quick_sort(arr):
    if len(arr) <= 1:
        return arr
    pivot = arr[0]
    left = []
    right = []
<｜fim▁hole｜>
        if arr[i] < pivot:
            left.append(arr[i])
        else:
            right.append(arr[i])
    return quick_sort(left) + [pivot] + quick_sort(right)<｜fim▁end｜>"""
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_length=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True)[len(input_text):])

对话补全

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct", trust_remote_code=True, torch_dtype=torch.bfloat16).cuda()
messages=[
    { 'role': 'user', 'content': "write a quick sort algorithm in python."}
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
# tokenizer.eos_token_id is the id of <｜end▁of▁sentence｜>  token
outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, top_k=50, top_p=0.95, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True))

完整的对话模板可以在Huggingface模型仓库的tokenizer_config.json中找到。

对话模板示例如下：

<｜begin▁of▁sentence｜>User: {user_message_1}

Assistant: {assistant_message_1}<｜end▁of▁sentence｜>User: {user_message_2}

Assistant:

你还可以添加可选的系统消息：

<｜begin▁of▁sentence｜>{system_message}

User: {user_message_1}

Assistant: {assistant_message_1}<｜end▁of▁sentence｜>User: {user_message_2}

Assistant:

使用vLLM进行推理（推荐）

若要使用vLLM进行模型推理，请将此拉取请求合并到你的vLLM代码库中：https://github.com/vllm-project/vllm/pull/4650。

from transformers import AutoTokenizer
from vllm import LLM, SamplingParams

max_model_len, tp_size = 8192, 1
model_name = "deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
llm = LLM(model=model_name, tensor_parallel_size=tp_size, max_model_len=max_model_len, trust_remote_code=True, enforce_eager=True)
sampling_params = SamplingParams(temperature=0.3, max_tokens=256, stop_token_ids=[tokenizer.eos_token_id])

messages_list = [
    [{"role": "user", "content": "Who are you?"}],
    [{"role": "user", "content": "write a quick sort algorithm in python."}],
    [{"role": "user", "content": "Write a piece of quicksort code in C++."}],
]

prompt_token_ids = [tokenizer.apply_chat_template(messages, add_generation_prompt=True) for messages in messages_list]

outputs = llm.generate(prompt_token_ids=prompt_token_ids, sampling_params=sampling_params)

generated_text = [output.outputs[0].text for output in outputs]
print(generated_text)

新功能使用示例

函数调用

函数调用允许模型调用外部工具以增强其能力。

# Assume that `model` and `tokenizer` are loaded
model.generation_config = GenerationConfig(do_sample=False, max_new_tokens=128, eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.eos_token_id)

tool_system_prompt = """You are a helpful Assistant.

## Tools

### Function

You have the following functions available:

- `get_current_weather`:
```json
{
    "name": "get_current_weather",
    "description": "Get the current weather in a given location",
    "parameters": {
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "The city and state, e.g. San Francisco, CA"
            },
            "unit": {
                "type": "string",
                "enum": [
                    "celsius",
                    "fahrenheit"
                ]
            }
        },
        "required": [
            "location"
        ]
    }
}
```"""

tool_call_messages = [{"role": "system", "content": tool_system_prompt}, {"role": "user", "content": "What's the weather like in Tokyo and Paris?"}]
tool_call_inputs = tokenizer.apply_chat_template(tool_call_messages, add_generation_prompt=True, return_tensors="pt")
tool_call_outputs = model.generate(tool_call_inputs.to(model.device))
# Generated text: '<｜tool▁calls▁begin｜><｜tool▁call▁begin｜>function<｜tool▁sep｜>get_current_weather\n```json\n{"location": "Tokyo"}\n```<｜tool▁call▁end｜>\n<｜tool▁call▁begin｜>function<｜tool▁sep｜>get_current_weather\n```json\n{"location": "Paris"}\n```<｜tool▁call▁end｜><｜tool▁calls▁end｜><｜end▁of▁sentence｜>'

# Mock response of calling `get_current_weather`
tool_messages = [{"role": "tool", "content": '{"location": "Tokyo", "temperature": "10", "unit": null}'}, {"role": "tool", "content": '{"location": "Paris", "temperature": "22", "unit": null}'}]
tool_inputs = tokenizer.apply_chat_template(tool_messages, add_generation_prompt=False, return_tensors="pt")[:, 1:]
tool_inputs = torch.cat([tool_call_outputs, tool_inputs.to(model.device)], dim=1)
tool_outputs = model.generate(tool_inputs)
# Generated text: The current weather in Tokyo is 10 degrees, and in Paris, it is 22 degrees.<｜end▁of▁sentence｜>

JSON输出

你可以使用JSON输出模式确保模型生成有效的JSON对象。要激活此模式，需要在系统提示中添加特殊指令。

# Assume that `model` and `tokenizer` are loaded
model.generation_config = GenerationConfig(do_sample=False, max_new_tokens=128, eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.eos_token_id)

user_system_prompt = 'The user will provide some exam text. Please parse the "question" and "answer" and output them in JSON format.'
json_system_prompt = f"""{user_system_prompt}

## Response Format

Reply with JSON object ONLY."""

json_messages = [{"role": "system", "content": json_system_prompt}, {"role": "user", "content": "Which is the highest mountain in the world? Mount Everest."}]
json_inputs = tokenizer.apply_chat_template(json_messages, add_generation_prompt=True, return_tensors="pt")
json_outpus = model.generate(json_inputs.to(model.device))
# Generated text: '```json\n{\n  "question": "Which is the highest mountain in the world?",\n  "answer": "Mount Everest."\n}\n```<｜end▁of▁sentence｜>'

FIM补全

在FIM（中间填充）补全中，你可以提供前缀和可选的后缀，模型将补全中间的内容。

# Assume that `model` and `tokenizer` are loaded
model.generation_config = GenerationConfig(do_sample=False, max_new_tokens=128, eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.eos_token_id)

prefix = """def quick_sort(arr):
    if len(arr) <= 1:
        return arr
    pivot = arr[0]
    left = []
    right = []
"""

suffix = """
        if arr[i] < pivot:
            left.append(arr[i])
        else:
            right.append(arr[i])
    return quick_sort(left) + [pivot] + quick_sort(right)"""

fim_prompt = f"<｜fim▁begin｜>{prefix}<｜fim▁hole｜>{suffix}<｜fim▁end｜>"
fim_inputs = tokenizer(fim_prompt, add_special_tokens=True, return_tensors="pt").input_ids
fim_outputs = model.generate(fim_inputs.to(model.device))
# Generated text: "    for i in range(1, len(arr)):<｜end▁of▁sentence｜>"

📚 详细文档

有关模型的详细信息，请参考论文链接。

🔧 技术细节

DeepSeek-Coder-V2是一个基于混合专家（MoE）架构的代码语言模型，它在DeepSeek-V2的中间检查点基础上进行了进一步预训练，使用了额外的6万亿个标记。通过这种方式，它显著提升了编码和数学推理能力，同时在通用语言任务中保持了相当的性能。

📄 许可证

本代码仓库遵循MIT许可证。DeepSeek-Coder-V2基础/指令模型的使用需遵循模型许可证。DeepSeek-Coder-V2系列（包括基础和指令模型）支持商业使用。

联系我们

如果你有任何问题，请提出问题或通过service@deepseek.com联系我们。