Kunoichi-DPO-v2-7B-GGUF-Imatrix Open Source Large Language Model - Excellent Performance and Support for Diverse QA Applications

Kunoichi DPO V2 7B GGUF Imatrix

Developed by Lewdiculous

A 7B-parameter large language model based on the Mistral architecture, trained with DPO (Direct Preference Optimization), demonstrating excellent performance in multiple benchmarks

Large Language Model #High-precision text generation #Low-resource inference optimization #Leading in multi-domain benchmarks

Downloads 3,705

Release Time : 2/27/2024

Model Overview

A 7B-parameter large language model trained with Direct Preference Optimization (DPO), excelling in tasks such as dialogue generation and logical reasoning, supporting text generation tasks

Model Features

Direct Preference Optimization (DPO)

Utilizes DPO training method, enabling the model to better understand human preferences and generate more desired text

High-performance quantization

Provides GGUF-Imatrix quantized versions, maintaining model performance post-quantization through importance matrix technology

Leading in multiple benchmarks

Outperforms similar 7B models in benchmarks such as MT Bench and EQ Bench, approaching the performance of some larger models

Model Capabilities

Text generation

Dialogue systems

Logical reasoning

Knowledge Q&A

Use Cases

Dialogue systems

Intelligent assistant

Used for building high-performance dialogue assistants

Achieved a 17.19% win rate in AlpacaEval2 tests, surpassing Claude 2 and GPT-3.5 Turbo

Knowledge Q&A

Open-domain Q&A

Answers various knowledge-based questions

Scored 64.94 in MMLU tests, exceeding similar 7B models

🚀 GGUF-Imatrix Quantizations for SanjiWatsuki/Kunoichi-DPO-v2-7B

This project provides GGUF-Imatrix quantizations for the SanjiWatsuki/Kunoichi-DPO-v2-7B model. It aims to improve the quality of quantized models using the Importance Matrix (Imatrix) technique.

⚠️ Important Note

My upload speeds have been cooked and unstable lately. Realistically I'd need to move to get a better provider.

💡 Usage Tip

If you want and you are able to... You can support my various endeavors here (Ko-fi). I apologize for disrupting your experience.

✨ Features

Importance Matrix (Imatrix): A technique used to improve the quality of quantized models by preserving the most important information during quantization.
Better Model Performance: Using an Imatrix can lead to better model performance, especially when the calibration data is diverse.
Multiple Quantization Options: Allows users to choose different quantization options based on their needs.

📚 Documentation

What does "Imatrix" mean?

It stands for Importance Matrix, a technique used to improve the quality of quantized models.

The Imatrix is calculated based on calibration data, and it helps determine the importance of different model activations during the quantization process. The idea is to preserve the most important information during quantization, which can help reduce the loss of model performance.

One of the benefits of using an Imatrix is that it can lead to better model performance, especially when the calibration data is diverse.

More information: [1] [2]

If you want any specific quantization to be added, feel free to ask.

All credits belong to the creator.

Base⇢ GGUF(F16)⇢ Imatrix-Data(F16)⇢ GGUF(Imatrix-Quants)

Using llama.cpp-b2277.

For --imatrix data, imatrix-Kunoichi-DPO-v2-7B-F16.dat was used.

🖼️ Waifu card

image/png

📊 Original model information

Performance Comparison 1

Model	MT Bench	EQ Bench	MMLU	Logic Test
GPT-4-Turbo	9.32	-	-	-
GPT-4	8.99	62.52	86.4	0.86
Kunoichi-DPO-v2-7B	8.51	42.18	64.94	0.58
Mixtral-8x7B-Instruct	8.30	44.81	70.6	0.75
Kunoichi-DPO-7B	8.29	41.60	64.83	0.59
Kunoichi-7B	8.14	44.32	64.9	0.58
Starling-7B	8.09	-	63.9	0.51
Claude-2	8.06	52.14	78.5	-
Silicon-Maid-7B	7.96	40.44	64.7	0.54
Loyal-Macaroni-Maid-7B	7.95	38.66	64.9	0.57
GPT-3.5-Turbo	7.94	50.28	70	0.57
Claude-1	7.9	-	77	-
Openchat-3.5	7.81	37.08	64.3	0.39
Dolphin-2.6-DPO	7.74	42.88	61.9	0.53
Zephyr-7B-beta	7.34	38.71	61.4	0.30
Llama-2-70b-chat-hf	6.86	51.56	63	-
Neural-chat-7b-v3-1	6.84	43.61	62.4	0.30

Performance Comparison 2

Model	Average	AGIEval	GPT4All	TruthfulQA	Bigbench
Kunoichi-DPO-7B	58.4	45.08	74	66.99	47.52
Kunoichi-DPO-v2-7B	58.31	44.85	75.05	65.69	47.65
Kunoichi-7B	57.54	44.99	74.86	63.72	46.58
OpenPipe/mistral-ft-optimized-1218	56.85	44.74	75.6	59.89	47.17
Silicon-Maid-7B	56.45	44.74	74.26	61.5	45.32
mlabonne/NeuralHermes-2.5-Mistral-7B	53.51	43.67	73.24	55.37	41.76
teknium/OpenHermes-2.5-Mistral-7B	52.42	42.75	72.99	52.99	40.94
openchat/openchat_3.5	51.34	42.67	72.92	47.27	42.51
berkeley-nest/Starling-LM-7B-alpha	51.16	42.06	72.72	47.33	42.53
HuggingFaceH4/zephyr-7b-beta	50.99	37.33	71.83	55.1	39.7

Performance Comparison 3

Model	AlpacaEval2	Length
GPT-4	23.58%	1365
GPT-4 0314	22.07%	1371
Mistral Medium	21.86%	1500
Mixtral 8x7B v0.1	18.26%	1465
Kunoichi-DPO-v2	17.19%	1785
Claude 2	17.19%	1069
Claude	16.99%	1082
Gemini Pro	16.85%	1315
GPT-4 0613	15.76%	1140
Claude 2.1	15.73%	1096
Mistral 7B v0.2	14.72%	1676
GPT 3.5 Turbo 0613	14.13%	1328
LLaMA2 Chat 70B	13.87%	1790
LMCocktail-10.7B-v1	13.15%	1203
WizardLM 13B V1.1	11.23%	1525
Zephyr 7B Beta	10.99%	1444
OpenHermes-2.5-Mistral (7B)	10.34%	1107
GPT 3.5 Turbo 0301	9.62%	827
Kunoichi-7B	9.38%	1492
GPT 3.5 Turbo 1106	9.18%	796
GPT-3.5	8.56%	1018
Phi-2 DPO	7.76%	1687
LLaMA2 Chat 13B	7.70%	1513

📄 License

This project is licensed under the cc-by-nc-4.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご