C4ai Command R Plus GGUF

🚀 Command R+ GGUF

This repository offers GGUF weights for llama.cpp, enhancing text - generation capabilities.

🚀 Quick Start

Prerequisites

1. Make sure you have release b2636 or a newer version.

Execution

Start with the following command:

./main -p "<|START_OF_TURN_TOKEN|><|USER_TOKEN|>Who are you?<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>" --color -m /path/to/command-r-plus-Q3_K_L-00001-of-00002.gguf

✨ Features

This repository contains GGUF weights for llama.cpp. Support for these weights was added in release b2636. Since commit dd2d53a, all weights in this repo come with chat templates.

In the imatrix folder, you can find imatrix quants. The importance matrix was trained using kalomaze's groups_merged.txt.

📚 Documentation

Perplexity on wikitext-2-raw [WIP]

Merging Weights

After commit 8a28d12, weights are split with gguf-split. This means you don't need to merge weights. Just pass the first split, as shown in the quick - start example, and llama.cpp will automatically load all splits. However, if you want to merge the splits, you can use the following command:

./gguf-split --merge /path/to/command-r-plus-f16-00001-of-00005.gguf /path/to/command-r-plus-f16-combined.gguf

📄 License

This project is licensed under cc-by-nc-4.0.