C4ai Command R Plus 4bit

🚀 Cohere Labs Command R+ Model Card

Cohere Labs Command R+ is an advanced 104B parameter model with cutting - edge capabilities like Retrieval Augmented Generation (RAG) and multi - step tool use. It's a multilingual model optimized for reasoning, summarization, and question - answering across 10 languages.

🚀 Quick Start

Installation

Please install transformers from the source repository that includes the necessary changes for this model.

# pip install 'git+https://github.com/huggingface/transformers.git' bitsandbytes accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "CohereForAI/c4ai-command-r-plus-4bit"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Format message with the command-r-plus chat template
messages = [{"role": "user", "content": "Hello, how are you?"}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
## <BOS_TOKEN><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Hello, how are you?<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>

gen_tokens = model.generate(
    input_ids, 
    max_new_tokens=100, 
    do_sample=True, 
    temperature=0.3,
    )

gen_text = tokenizer.decode(gen_tokens[0])
print(gen_text)

Usage

You can try out Cohere Labs Command R+ before downloading the weights in our hosted Hugging Face Space.

✨ Features

• High - end Capabilities: Cohere Labs Command R+ has highly advanced capabilities, including Retrieval Augmented Generation (RAG) and tool use for automating sophisticated tasks.
• Multilingual Performance: Evaluated in 10 languages (English, French, Spanish, Italian, German, Brazilian Portuguese, Japanese, Korean, Arabic, and Simplified Chinese) and pre - trained on additional 13 languages.
• Optimized for Use Cases: Optimized for reasoning, summarization, and question - answering.
• Tool Use & Multihop: Specifically trained with conversational tool use capabilities, supporting multi - step tool use.

📚 Documentation

Model Summary

Cohere Labs Command R+ is an open weights research release of a 104B billion parameter model. It's part of a family of open weight releases from Cohere For AI and Cohere. Our smaller companion model is Cohere Labs Command R.

Model Details

• Input & Output: The model takes text as input and generates text as output.
• Model Architecture: It's an auto - regressive language model using an optimized transformer architecture. After pretraining, it uses supervised fine - tuning (SFT) and preference training.
• Languages covered: Optimized for 10 languages and pre - trained data includes 13 additional languages.
• Context length: Supports a context length of 128K.

Tool use & multihop capabilities

Command R+ has been specifically trained with conversational tool use capabilities. It takes a conversation and a list of available tools as input and generates a json - formatted list of actions. It recognizes a special directly_answer tool.

Comprehensive documentation for working with command R+'s tool use prompt template can be found here. It also supports Hugging Face's tool use API.

Example Code for Tool Use

from transformers import AutoTokenizer

model_id = "CohereForAI/c4ai-command-r-plus-4bit"
tokenizer = AutoTokenizer.from_pretrained(model_id)

# define conversation input:
conversation = [
    {"role": "user", "content": "Whats the biggest penguin in the world?"}
]
# Define tools available for the model to use:
tools = [
  {
    "name": "internet_search",
    "description": "Returns a list of relevant document snippets for a textual query retrieved from the internet",
    "parameter_definitions": {
      "query": {
        "description": "Query to search the internet with",
        "type": 'str',
        "required": True
      }
    }
  },
  {
    'name': "directly_answer",
    "description": "Calls a standard (un-augmented) AI chatbot to generate a response given the conversation history",
    'parameter_definitions': {}
  }
]

# render the tool use prompt as a string:
tool_use_prompt = tokenizer.apply_tool_use_template(
    conversation,
    tools=tools,
    tokenize=False,
    add_generation_prompt=True,
)
print(tool_use_prompt)

from transformers import AutoTokenizer

model_id = "CohereForAI/c4ai-command-r-plus-4bit"
tokenizer = AutoTokenizer.from_pretrained(model_id)

# define conversation input:
conversation = [
    {"role": "user", "content": "Whats the biggest penguin in the world?"}
]

# Define tools available for the model to use
# Type hints and docstrings from Python functions are automatically extracted
def internet_search(query: str):
    """
    Returns a list of relevant document snippets for a textual query retrieved from the internet

    Args:
        query: Query to search the internet with
    """
    pass

def directly_answer():
    """
    Calls a standard (un-augmented) AI chatbot to generate a response given the conversation history
    """
    pass

tools = [internet_search, directly_answer]

# render the tool use prompt as a string:
tool_use_prompt = tokenizer.apply_chat_template(
    conversation,
    tools=tools,
    tokenize=False,
    add_generation_prompt=True,
)
print(tool_use_prompt)

<BOS_TOKEN><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|># Safety Preamble
The instructions in this section override those in the task description and style guide sections. Don't answer questions that are harmful or immoral.

# System Preamble
## Basic Rules
You are a powerful conversational AI trained by Cohere to help people. You are