C4ai Command R7b 12 2024

🚀 Cohere Labs Command R7B

Cohere Labs Command R7B is a 7B parameter model with advanced capabilities, optimized for various tasks such as reasoning, summarization, question - answering, and code. It supports 23 languages and has powerful agentic and RAG capabilities.

🚀 Quick Start

Try Command R7B

You can try out Cohere Labs Command R7B before downloading the weights in our hosted Hugging Face Space.

Usage

Please install transformers from the source repository that includes the necessary changes for this model.

# pip install 'git+https://github.com/huggingface/transformers.git'
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "CohereLabs/c4ai-command-r7b-12-2024"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Format message with the c4ai-command-r7b-12-2024 chat template
messages = [{"role": "user", "content": "Hello, how are you?"}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")

gen_tokens = model.generate(
    input_ids,
    max_new_tokens=100,
    do_sample=True,
    temperature=0.3,
)

gen_text = tokenizer.decode(gen_tokens[0], skip_special_tokens=True)
print(gen_text)

✨ Features

Multilingual Capability

Command R7B is a multilingual model trained on 23 languages, including English, French, German, Spanish, Italian, Portuguese, Japanese, Korean, Chinese, Arabic, Greek, Persian, Polish, Indonesian, Czech, Hebrew, Hindi, Dutch, Romanian, Russian, Turkish, Ukrainian, and Vietnamese.

Sophisticated Task Performance

The model is trained to perform sophisticated tasks such as Retrieval Augmented Generation (RAG) and tool use. It also has powerful agentic capabilities, able to use and combine multiple tools over multiple steps to accomplish more difficult tasks.

Top Performance in Code Use Cases

Command R7B obtains top performance on enterprise - relevant code use cases.

Well - Rounded on Benchmarks

Compared to other similarly sized open - weights models, Command R7B ranks first with strong performance across all tasks on standardized and externally verifiable benchmarks such as the HuggingFace Open LLM Leaderboard.

Chat Capabilities

• Conversational Mode: Configured for interactive experiences like chatbots, it replies conversationally, provides introductory statements and follow - up questions, and uses Markdown and LaTeX where appropriate.
• Instruct Mode: Designed for non - interactive, task - focused use cases like information extraction, text summarization, translation, and categorization, providing concise yet comprehensive responses.

RAG Capabilities

Command R7B has been trained specifically for tasks like the final step of Retrieval Augmented Generation (RAG). RAG is supported through chat templates in Transformers.

Tool Use Capabilities

The model has been specifically trained with conversational tool use capabilities, allowing it to interact with external tools like APIs, databases, or search engines through chat templates in Transformers.

Code Capabilities

Command R7B has meaningfully improved on code capabilities. It outperforms other models of similar size in enterprise - relevant scenarios, including SQL and code translation.

📦 Installation

Please install transformers from the source repository that includes the necessary changes for this model.

# pip install 'git+https://github.com/huggingface/transformers.git'

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "CohereLabs/c4ai-command-r7b-12-2024"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Format message with the c4ai-command-r7b-12-2024 chat template
messages = [{"role": "user", "content": "Hello, how are you?"}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")

gen_tokens = model.generate(
    input_ids,
    max_new_tokens=100,
    do_sample=True,
    temperature=0.3,
)

gen_text = tokenizer.decode(gen_tokens[0], skip_special_tokens=True)
print(gen_text)

Advanced Usage - RAG Example

# Define conversation input
conversation = [{"role": "user", "content": "What has Man always dreamed of?"}]

# Define documents for retrieval-based generation
documents = [
  {"heading": "The Moon: Our Age-Old Foe", "body": "Man has always dreamed of destroying the moon. In this essay, I shall..."},
  {"heading": "Love is all you need", "body": "Man's dream has always been to find love. This profound lesson..."}
]

# Get the RAG prompt
input_prompt = tokenizer.apply_chat_template(conversation=conversation, documents=documents, tokenize=False, add_generation_prompt=True, return_tensors="pt")
# Tokenize the prompt
input_ids = tokenizer.encode_plus(input_prompt, return_tensors="pt")

Advanced Usage - Tool Use Example

# Define tools
tools = [
    {
    "type": "function",
    "function": {
        "name": "query_daily_sales_report",
        "description": "Connects to a database to retrieve overall sales volumes and sales information for a given day.",
        "parameters": {
            "type": "object",
            "properties": {
                "day": {
                    "description": "Retrieves sales data for this day, formatted as YYYY-MM-DD.",
                    "type": "string",
                    }
                },
                "required": ["day"]
            },
        }
    }
]

# Define conversation input
conversation = [{"role": "user", "content": "Can you provide a sales summary for 29th September 2023?"}]

# Get the Tool Use prompt
input_prompt = tokenizer.apply_chat_template(conversation=conversation, tools=tools, tokenize=False, add_generation_prompt=True, return_tensors="pt")

# Tokenize the prompt
input_ids = tokenizer.encode_plus(input_prompt, return_tensors="pt")

tool_call = {"name": "query_daily_sales_report", "arguments": {"day": "2023-09-29"}}
tool_plan = "I will use the query_daily_sales_report tool to find the sales summary for 29th September 2023."
conversation.append({"role": "assistant", "tool_calls": [{ "id": "0", "type": "function", "function": tool_call},], "tool_plan": tool_plan})

# every tool result needs to be a dictionary!!
api_response_for_query_daily_sales_report = {"date": "2023-09-29", "summary": "Total Sales Amount: 10000, Total Units Sold: 250"} 
# append tool results
conversation.append({"role": "tool", "tool_call_id": "0", "content": api_response_for_query_daily_sales_report}) # make sure "tool_call_id" matches the "id" of the tool_call

📚 Documentation

Model Details

• Input: The model takes text as input.
• Output: The model generates text as output.
• Model Architecture: It is an auto - regressive language model using an optimized transformer architecture. After pretraining, it uses supervised fine - tuning (SFT) and preference training to align model behavior to human preferences for helpfulness and safety. The model features three layers with sliding window attention (window size 4096) and ROPE for efficient local context modeling and relative positional encoding. A fourth layer uses global attention without positional embeddings, enabling unrestricted token interactions across the entire sequence.
• Context length: Command R7B supports a context length of 128K.

Benchmark Performance

HuggingFace Leaderboard evaluation results. Competitor numbers are taken from the official leaderboard. Command R7B results are calculated by us using the official HuggingFace prompts and evaluation code.

Chat Modes

• Conversational Mode: Conversational mode conditions the model on interactive behaviour.
• Instruct Mode: Instruct mode conditions the model to provide concise yet comprehensive responses.

Note: by default, Command R7B is delivered without a system preamble. We recommend to add the conversational or instruct preambles as described in our docs.

RAG Details

Document snippets should be short chunks, typically around 100 - 400 words per chunk, formatted as key - value pairs. You may find that simply including relevant documents directly in a user message works just as well, or better than using the documents parameter to render the special RAG template. For more information, see the Command R7B prompt format docs and the Transformers RAG documentation.

Tool Use Details

For more information on tool calling, see the Command R7B prompt format docs and the Transformers tool use documentation.

Code Capabilities Details

For better performance in code - generation related instructions, we recommend using a low temperature (and even greedy decoding).

🔧 Technical Details

For more details on how this model was developed, check out our Tech Report.

📄 License

The model is licensed under CC - BY - NC and requires also adhering to Cohere Lab's Acceptable Use Policy.

📦 Model Information

📞 Model Card Contact

For errors or additional questions about details in this model card, contact labs@cohere.com

🔗 Try Chat

You can try Command R7B chat in the playground here. You can also use it in our dedicated Hugging Face Space here.

📖 Citation

@misc{cohere2025commandaenterprisereadylarge,
      title={Command A: An Enterprise-Ready Large Language Model}, 
      author={Team Cohere and Aakanksha and Arash Ahmadian and Marwan Ahmed and Jay Alammar and Yazeed Alnumay and Sophia Althammer and Arkady Arkhangorodsky and Viraat Aryabumi and Dennis Aumiller