Phi 3 Mini 4k Instruct GGUF

🚀 Phi 3 Mini 4K Instruct GGUF

This repository contains GGUF format model files for Microsoft's Phi 3 Mini 4K Instruct, updated with the latest model changes as of July 21, 2024.

The Phi-3-Mini-4K-Instruct is a 3.8B parameters, lightweight, state-of-the-art open model. It is trained with the Phi-3 datasets, which includes both synthetic data and the filtered publicly available websites data, focusing on high-quality and reasoning dense properties. Learn more on Microsoft’s Model page.

🚀 Quick Start

What is GGUF?

GGUF is a file format for representing AI models. It is the third version of the format, introduced by the llama.cpp team on August 21st 2023. It replaces GGML, which is no longer supported by llama.cpp. This model is converted with llama.cpp build 3432 (revision 45f2c19), using autogguf.

Prompt template

<|system|>
{{system_prompt}}<|end|>
<|user|>
{{prompt}}<|end|>
<|assistant|>

Download & run with cnvrs on iPhone, iPad, and Mac!

cnvrs is the best app for private, local AI on your device:

• Create & save Characters with custom system prompts & temperature settings.
• Download and experiment with any GGUF model you can find on HuggingFace.
• Customize it with custom Theme colors.
• Powered by Metal ⚡️ & Llama.cpp, with haptics during response streaming.
• Try it out yourself today, on Testflight!
• Follow cnvrs on twitter to stay up to date.

📚 Documentation

Original Model Evaluation

Comparison of July update vs original April release

Original April release

As is now standard, we use few-shot prompts to evaluate the models, at temperature 0. The prompts and number of shots are part of a Microsoft internal tool to evaluate language models, and in particular, we did no optimization to the pipeline for Phi-3. More specifically, we do not change prompts, pick different few-shot examples, change prompt format, or do any other form of optimization for the model. The number of k–shot examples is listed per-benchmark.

📄 License

This project is licensed under the MIT License. View the license.

📋 Information Table