Deepseek V3 0324 BF16

🚀 DeepSeek-V3-0324

This is the BF16 model of DeekSeek V3-0324, useful for quantization and inference on GPUs that do not support FP8 (Nvidia Ampere).

🚀 Quick Start

This is the BF16 model of DeekSeek V3-0324. The BF16 weights are obtained by dequantizing the FP8 quantized weights from DeepSeek AI: https://huggingface.co/deepseek-ai/DeepSeek-V3-0324. GPTQModel is the recommended quantization toolkit for DeepSeek V3-0324 inference on vLLM and SGLang.

✨ Features

DeepSeek-V3-0324 demonstrates notable improvements over its predecessor, DeepSeek-V3, in several key aspects.

Reasoning Capabilities

• Significant improvements in benchmark performance:
  • MMLU-Pro: 75.9 → 81.2 (+5.3)
  • GPQA: 59.1 → 68.4 (+9.3)
  • AIME: 39.6 → 59.4 (+19.8)
  • LiveCodeBench: 39.2 → 49.2 (+10.0)

Front-End Web Development

• Improved the executability of the code
• More aesthetically pleasing web pages and game front-ends

Chinese Writing Proficiency

• Enhanced style and content quality:
  • Aligned with the R1 writing style
  • Better quality in medium-to-long-form writing
• Feature Enhancements
  • Improved multi-turn interactive rewriting
  • Optimized translation quality and letter writing

Chinese Search Capabilities

• Enhanced report analysis requests with more detailed outputs

Function Calling Improvements

• Increased accuracy in Function Calling, fixing issues from previous V3 versions

💡 Usage Tip

System Prompt

In the official DeepSeek web/app, we use the same system prompt with a specific date.

The assistant is DeepSeek Chat, created by DeepSeek.
Today is {current date}.

For example,

The assistant is DeepSeek Chat, created by DeepSeek.
Today is March 24th, Monday.

Temperature

In our web and application environments, the temperature parameter $T_{model}$ is set to 0.3. Because many users use the default temperature 1.0 in API call, we have implemented an API temperature $T_{api}$ mapping mechanism that adjusts the input API temperature value of 1.0 to the most suitable model temperature setting of 0.3.

$$ T_{model} = T_{api} \times 0.3 \quad (0 \leq T_{api} \leq 1) $$

$$ T_{model} = T_{api} - 0.7 \quad (1 < T_{api} \leq 2) $$

Thus, if you call V3 via API, temperature 1.0 equals to the model temperature 0.3.

Prompts for File Uploading and Web Search

For file uploading, please follow the template to create prompts, where {file_name}, {file_content} and {question} are arguments.

file_template = \
"""[file name]: {file_name}
[file content begin]
{file_content}
[file content end]
{question}"""

For Web Search, {search_results}, {cur_date}, and {question} are arguments.

For Chinese query, we use the prompt:

search_answer_zh_template = \
'''# The following contents are the search results related to the user's message:
{search_results}
In the search results I provide to you, each result is formatted as [webpage X begin]...[webpage X end], where X represents the numerical index of each article. Please cite the context at the end of the relevant sentence when appropriate. Use the citation format [citation:X] in the corresponding part of your answer. If a sentence is derived from multiple contexts, list all relevant citation numbers, such as [citation:3][citation:5]. Be sure not to cluster all citations at the end; instead, include them in the corresponding parts of the answer.
When responding, please keep the following points in mind:
- Today is {cur_date}.
- Not all content in the search results is closely related to the user's question. You need to evaluate and filter the search results based on the question.
- For listing-type questions (e.g., listing all flight information), try to limit the answer to 10 key points and inform the user that they can refer to the search sources for complete information. Prioritize providing the most complete and relevant items in the list. Avoid mentioning content not provided in the search results unless necessary.
- For creative tasks (e.g., writing an essay), ensure that references are cited within the body of the text, such as [citation:3][citation:5], rather than only at the end of the text. You need to interpret and summarize the user's requirements, choose an appropriate format, fully utilize the search results, extract key information, and generate an answer that is insightful, creative, and professional. Extend the length of your response as much as possible, addressing each point in detail and from multiple perspectives, ensuring the content is rich and thorough.
- If the response is lengthy, structure it well and summarize it in paragraphs. If a point-by-point format is needed, try to limit it to 5 points and merge related content.
- For objective Q&A, if the answer is very brief, you may add one or two related sentences to enrich the content.
- Choose an appropriate and visually appealing format for your response based on the user's requirements and the content of the answer, ensuring strong readability.
- Your answer should synthesize information from multiple relevant webpages and avoid repeatedly citing the same webpage.
- Unless the user requests otherwise, your response should be in the same language as the user's question.

# The user's message is:
{question}'''

For English query, we use the prompt:

search_answer_en_template = \
'''# The following contents are the search results related to the user's message:
{search_results}
In the search results I provide to you, each result is formatted as [webpage X begin]...[webpage X end], where X represents the numerical index of each article. Please cite the context at the end of the relevant sentence when appropriate. Use the citation format [citation:X] in the corresponding part of your answer. If a sentence is derived from multiple contexts, list all relevant citation numbers, such as [citation:3][citation:5]. Be sure not to cluster all citations at the end; instead, include them in the corresponding parts of the answer.
When responding, please keep the following points in mind:
- Today is {cur_date}.
- Not all content in the search results is closely related to the user's question. You need to evaluate and filter the search results based on the question.
- For listing-type questions (e.g., listing all flight information), try to limit the answer to 10 key points and inform the user that they can refer to the search sources for complete information. Prioritize providing the most complete and relevant items in the list. Avoid mentioning content not provided in the search results unless necessary.
- For creative tasks (e.g., writing an essay), ensure that references are cited within the body of the text, such as [citation:3][citation:5], rather than only at the end of the text. You need to interpret and summarize the user's requirements, choose an appropriate format, fully utilize the search results, extract key information, and generate an answer that is insightful, creative, and professional. Extend the length of your response as much as possible, addressing each point in detail and from multiple perspectives, ensuring the content is rich and thorough.
- If the response is lengthy, structure it well and summarize it in paragraphs. If a point-by-point format is needed, try to limit it to 5 points and merge related content.
- For objective Q&A, if the answer is very brief, you may add one or two related sentences to enrich the content.
- Choose an appropriate and visually appealing format for your response based on the user's requirements and the content of the answer, ensuring strong readability.
- Your answer should synthesize information from multiple relevant webpages and avoid repeatedly citing the same webpage.
- Unless the user requests otherwise, your response should be in the same language as the user's question.

# The user's message is:
{question}'''

📚 Documentation

The model structure of DeepSeek-V3-0324 is exactly the same as DeepSeek-V3. Please visit DeepSeek-V3 repo for more information about running this model locally.

This model supports features such as function calling, JSON output, and FIM completion. For instructions on how to construct prompts to use these features, please refer to DeepSeek-V2.5 repo.

⚠️ Important Note

Hugging Face's Transformers has not been directly supported yet.

📄 License

This repository and the model weights are licensed under the MIT License.

📖 Citation

@misc{deepseekai2024deepseekv3technicalreport,
      title={DeepSeek-V3 Technical Report}, 
      author={DeepSeek-AI},
      year={2024},
      eprint={2412.19437},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2412.19437}, 
}

📞 Contact

If you have any questions, please raise an issue or contact us at service@deepseek.com.