🚀 Cohere Labs Command R+ Model Card

Cohere Labs Command R+ is a 104B parameter model with advanced capabilities. It supports Retrieval Augmented Generation (RAG) and multi - step tool use, excelling in tasks like reasoning, summarization, and question - answering. Evaluated in 10 languages, it's optimized for various real - world applications.

🚀 Quick Start

You can try out Cohere Labs Command R+ before downloading the weights in our hosted Hugging Face Space.

Please install transformers from the source repository that includes the necessary changes for this model.

# pip install 'git+https://github.com/huggingface/transformers.git'
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "CohereLabs/c4ai-command-r-plus"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Format message with the command-r-plus chat template
messages = [{"role": "user", "content": "Hello, how are you?"}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
## <BOS_TOKEN><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Hello, how are you?<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>

gen_tokens = model.generate(
    input_ids, 
    max_new_tokens=100, 
    do_sample=True, 
    temperature=0.3,
    )

gen_text = tokenizer.decode(gen_tokens[0])
print(gen_text)

✨ Features

• Advanced Capabilities: Cohere Labs Command R+ has highly advanced capabilities, including Retrieval Augmented Generation (RAG) and multi - step tool use to automate sophisticated tasks.
• Multilingual Support: Evaluated in 10 languages (English, French, Spanish, Italian, German, Brazilian Portuguese, Japanese, Korean, Arabic, and Simplified Chinese), it can handle diverse language needs.
• Optimized for Use Cases: Optimized for reasoning, summarization, and question - answering, making it suitable for a variety of real - world applications.
• Grounded Generation: Specifically trained with grounded generation capabilities, it can generate responses based on supplied document snippets and include citations.

📦 Installation

Please install transformers from the source repository that includes the necessary changes for this model:

pip install 'git+https://github.com/huggingface/transformers.git'

💻 Usage Examples

Basic Usage

# pip install 'git+https://github.com/huggingface/transformers.git'
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "CohereLabs/c4ai-command-r-plus"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Format message with the command-r-plus chat template
messages = [{"role": "user", "content": "Hello, how are you?"}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
## <BOS_TOKEN><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Hello, how are you?<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>

gen_tokens = model.generate(
    input_ids, 
    max_new_tokens=100, 
    do_sample=True, 
    temperature=0.3,
    )

gen_text = tokenizer.decode(gen_tokens[0])
print(gen_text)

Advanced Usage - Quantized model through bitsandbytes, 8 - bit precision

# pip install 'git+https://github.com/huggingface/transformers.git' bitsandbytes accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

bnb_config = BitsAndBytesConfig(load_in_8bit=True)

model_id = "CohereLabs/c4ai-command-r-plus"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config)

# Format message with the command-r-plus chat template
messages = [{"role": "user", "content": "Hello, how are you?"}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
## <BOS_TOKEN><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Hello, how are you?<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>

gen_tokens = model.generate(
    input_ids, 
    max_new_tokens=100, 
    do_sample=True, 
    temperature=0.3,
    )

gen_text = tokenizer.decode(gen_tokens[0])
print(gen_text)

📚 Documentation

Model Details

• Input: The model takes text as input.
• Output: It generates text as output.
• Model Architecture: It is an auto - regressive language model using an optimized transformer architecture. After pretraining, it uses supervised fine - tuning (SFT) and preference training to align with human preferences for helpfulness and safety.
• Languages covered: Optimized for 10 languages (English, French, Spanish, Italian, German, Brazilian Portuguese, Japanese, Korean, Arabic, and Simplified Chinese). Pre - training data also included 13 other languages (Russian, Polish, Turkish, Vietnamese, Dutch, Czech, Indonesian, Ukrainian, Romanian, Greek, Hindi, Hebrew, Persian).
• Context length: Command R+ supports a context length of 128K.

Evaluations

Command R+ has been submitted to the Open LLM leaderboard. The evaluation results are as follows:

Note that these metrics do not fully capture RAG, multilingual, tooling performance or the evaluation of open - ended generations. For more evaluations, refer to here for RAG, multilingual and tooling, and chatbot arena for open - ended generation.

Grounded Generation and RAG Capabilities

Command R+ has been specifically trained with grounded generation capabilities. It can generate responses based on supplied document snippets and include citations. The code example for rendering a grounded generation prompt is as follows:

</details>

### Single - Step Tool Use Capabilities ("Function Calling")
[The original text did not provide complete content for this part. If there is more information, it can be further supplemented here.]

## 🔧 Technical Details
The model is an auto - regressive language model using an optimized transformer architecture. After pretraining, it uses supervised fine - tuning (SFT) and preference training to align with human preferences for helpfulness and safety. The grounded generation behavior is trained via a mixture of supervised fine - tuning and preference fine - tuning with a specific prompt template.

## 📄 License
The model is under the [CC - BY - NC](https://cohere.com/c4ai-cc-by-nc-license) license and requires adhering to [Cohere Lab's Acceptable Use Policy](https://docs.cohere.com/docs/c4ai-acceptable-use-policy).

> ⚠️ **Important Note**
> 
> This model is non - quantized version of Cohere Labs Command R+. You can find the quantized version of Cohere Labs Command R+ using bitsandbytes [here](https://huggingface.co/CohereLabs/c4ai-command-r-plus-4bit).

> 💡 **Usage Tip**
> 
> Deviating from the specific prompt template for grounded generation may reduce performance, but experimentation is encouraged.